Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/40561/?format=api
{ "id": 40561, "url": "https://patches.dpdk.org/api/patches/40561/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/2d97a30636fdaece6f64eabb597f2f892e3e93ad.1527787910.git.anatoly.burakov@intel.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<2d97a30636fdaece6f64eabb597f2f892e3e93ad.1527787910.git.anatoly.burakov@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/2d97a30636fdaece6f64eabb597f2f892e3e93ad.1527787910.git.anatoly.burakov@intel.com", "date": "2018-05-31T17:35:33", "name": "[dpdk-dev] eal: add option to limit memory allocation on sockets", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": true, "hash": "f1b5f72ba890fac8b7d27159f582bb673b8002b3", "submitter": { "id": 4, "url": "https://patches.dpdk.org/api/people/4/?format=api", "name": "Burakov, Anatoly", "email": "anatoly.burakov@intel.com" }, "delegate": { "id": 1, "url": "https://patches.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "https://patches.dpdk.org/project/dpdk/patch/2d97a30636fdaece6f64eabb597f2f892e3e93ad.1527787910.git.anatoly.burakov@intel.com/mbox/", "series": [], "comments": "https://patches.dpdk.org/api/patches/40561/comments/", "check": "warning", "checks": "https://patches.dpdk.org/api/patches/40561/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 8529069C6;\n\tThu, 31 May 2018 19:35:39 +0200 (CEST)", "from mga18.intel.com (mga18.intel.com [134.134.136.126])\n\tby dpdk.org (Postfix) with ESMTP id 91F21695D\n\tfor <dev@dpdk.org>; Thu, 31 May 2018 19:35:36 +0200 (CEST)", "from fmsmga008.fm.intel.com ([10.253.24.58])\n\tby orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t31 May 2018 10:35:34 -0700", "from irvmail001.ir.intel.com ([163.33.26.43])\n\tby fmsmga008.fm.intel.com with ESMTP; 31 May 2018 10:35:33 -0700", "from sivswdev01.ir.intel.com (sivswdev01.ir.intel.com\n\t[10.237.217.45])\n\tby irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id\n\tw4VHZXPj023062; Thu, 31 May 2018 18:35:33 +0100", "from sivswdev01.ir.intel.com (localhost [127.0.0.1])\n\tby sivswdev01.ir.intel.com with ESMTP id w4VHZX9X010212;\n\tThu, 31 May 2018 18:35:33 +0100", "(from aburakov@localhost)\n\tby sivswdev01.ir.intel.com with LOCAL id w4VHZXcL010208;\n\tThu, 31 May 2018 18:35:33 +0100" ], "X-Amp-Result": "SKIPPED(no attachment in message)", "X-Amp-File-Uploaded": "False", "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.49,463,1520924400\"; d=\"scan'208\";a=\"44301498\"", "From": "Anatoly Burakov <anatoly.burakov@intel.com>", "To": "dev@dpdk.org", "Cc": "John McNamara <john.mcnamara@intel.com>,\n\tMarko Kovacevic <marko.kovacevic@intel.com>", "Date": "Thu, 31 May 2018 18:35:33 +0100", "Message-Id": "<2d97a30636fdaece6f64eabb597f2f892e3e93ad.1527787910.git.anatoly.burakov@intel.com>", "X-Mailer": "git-send-email 1.7.0.7", "Subject": "[dpdk-dev] [PATCH] eal: add option to limit memory allocation on\n\tsockets", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "Previously, it was possible to limit maximum amount of memory\nallowed for allocation by creating validator callbacks. Although a\npowerful tool, it's a bit of a hassle and requires modifying the\napplication for it to work with DPDK example applications.\n\nFix this by adding a new parameter \"--socket-limit\", with syntax\nsimilar to \"--socket-mem\", which would set per-socket memory\nallocation limits, and set up a default validator callback to deny\nall allocations above the limit.\n\nThis option is incompatible with legacy mode, as validator callbacks\nare not supported there.\n\nSigned-off-by: Anatoly Burakov <anatoly.burakov@intel.com>\n---\n doc/guides/linux_gsg/build_sample_apps.rst | 4 +++\n .../prog_guide/env_abstraction_layer.rst | 4 +++\n lib/librte_eal/common/eal_common_options.c | 10 ++++++\n lib/librte_eal/common/eal_internal_cfg.h | 2 ++\n lib/librte_eal/common/eal_options.h | 2 ++\n lib/librte_eal/linuxapp/eal/eal.c | 36 +++++++++++++------\n lib/librte_eal/linuxapp/eal/eal_memory.c | 21 +++++++++++\n 7 files changed, 68 insertions(+), 11 deletions(-)", "diff": "diff --git a/doc/guides/linux_gsg/build_sample_apps.rst b/doc/guides/linux_gsg/build_sample_apps.rst\nindex 3623ddf46..332424e05 100644\n--- a/doc/guides/linux_gsg/build_sample_apps.rst\n+++ b/doc/guides/linux_gsg/build_sample_apps.rst\n@@ -114,6 +114,10 @@ The EAL options are as follows:\n this memory will also be pinned (i.e. not released back to the system until\n application closes).\n \n+* ``--socket-limit``:\n+ Limit maximum memory available for allocation on each socket. Does not support\n+ legacy memory mode.\n+\n * ``-d``:\n Add a driver or driver directory to be loaded.\n The application should use this option to load the pmd drivers\ndiff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst\nindex a22640d29..4c51efd42 100644\n--- a/doc/guides/prog_guide/env_abstraction_layer.rst\n+++ b/doc/guides/prog_guide/env_abstraction_layer.rst\n@@ -147,6 +147,10 @@ notified about memory allocations above specified threshold (and have a chance\n to deny them), allocation validator callbacks are also available via\n ``rte_mem_alloc_validator_callback_register()`` function.\n \n+A default validator callback is provided by EAL, which can be enabled with a\n+``--socket-limit`` command-line option, for a simple way to limit maximum amount\n+of memory that can be used by DPDK application.\n+\n .. note::\n \n In multiprocess scenario, all related processes (i.e. primary process, and\ndiff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c\nindex ecebb2923..c720efa86 100644\n--- a/lib/librte_eal/common/eal_common_options.c\n+++ b/lib/librte_eal/common/eal_common_options.c\n@@ -70,6 +70,7 @@ eal_long_options[] = {\n \t{OPT_PCI_WHITELIST, 1, NULL, OPT_PCI_WHITELIST_NUM },\n \t{OPT_PROC_TYPE, 1, NULL, OPT_PROC_TYPE_NUM },\n \t{OPT_SOCKET_MEM, 1, NULL, OPT_SOCKET_MEM_NUM },\n+\t{OPT_SOCKET_LIMIT, 1, NULL, OPT_SOCKET_LIMIT_NUM },\n \t{OPT_SYSLOG, 1, NULL, OPT_SYSLOG_NUM },\n \t{OPT_VDEV, 1, NULL, OPT_VDEV_NUM },\n \t{OPT_VFIO_INTR, 1, NULL, OPT_VFIO_INTR_NUM },\n@@ -179,6 +180,10 @@ eal_reset_internal_config(struct internal_config *internal_cfg)\n \t/* zero out the NUMA config */\n \tfor (i = 0; i < RTE_MAX_NUMA_NODES; i++)\n \t\tinternal_cfg->socket_mem[i] = 0;\n+\tinternal_cfg->force_socket_limits = 0;\n+\t/* zero out the NUMA limits config */\n+\tfor (i = 0; i < RTE_MAX_NUMA_NODES; i++)\n+\t\tinternal_cfg->socket_limit[i] = 0;\n \t/* zero out hugedir descriptors */\n \tfor (i = 0; i < MAX_HUGEPAGE_SIZES; i++) {\n \t\tmemset(&internal_cfg->hugepage_info[i], 0,\n@@ -1322,6 +1327,11 @@ eal_check_common_options(struct internal_config *internal_cfg)\n \t\t\t\"be specified together with --\"OPT_NO_HUGE\"\\n\");\n \t\treturn -1;\n \t}\n+\tif (internal_config.force_socket_limits && internal_config.legacy_mem) {\n+\t\tRTE_LOG(ERR, EAL, \"--\" OPT_SOCKET_LIMIT \" is only supported in \"\n+\t\t\t\"non-legacy memory mode\\n\");\n+\t\treturn -1;\n+\t}\n \n \treturn 0;\n }\ndiff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h\nindex c4cbf3acd..d66cd0313 100644\n--- a/lib/librte_eal/common/eal_internal_cfg.h\n+++ b/lib/librte_eal/common/eal_internal_cfg.h\n@@ -46,6 +46,8 @@ struct internal_config {\n \t/** true to try allocating memory on specific sockets */\n \tvolatile unsigned force_sockets;\n \tvolatile uint64_t socket_mem[RTE_MAX_NUMA_NODES]; /**< amount of memory per socket */\n+\tvolatile unsigned force_socket_limits;\n+\tvolatile uint64_t socket_limit[RTE_MAX_NUMA_NODES]; /**< limit amount of memory per socket */\n \tuintptr_t base_virtaddr; /**< base address to try and reserve memory from */\n \tvolatile unsigned legacy_mem;\n \t/**< true to enable legacy memory behavior (no dynamic allocation,\ndiff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h\nindex 211ae06ae..6d92f64a8 100644\n--- a/lib/librte_eal/common/eal_options.h\n+++ b/lib/librte_eal/common/eal_options.h\n@@ -47,6 +47,8 @@ enum {\n \tOPT_NO_SHCONF_NUM,\n #define OPT_SOCKET_MEM \"socket-mem\"\n \tOPT_SOCKET_MEM_NUM,\n+#define OPT_SOCKET_LIMIT \"socket-limit\"\n+\tOPT_SOCKET_LIMIT_NUM,\n #define OPT_SYSLOG \"syslog\"\n \tOPT_SYSLOG_NUM,\n #define OPT_VDEV \"vdev\"\ndiff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c\nindex 8655b8691..5d731724d 100644\n--- a/lib/librte_eal/linuxapp/eal/eal.c\n+++ b/lib/librte_eal/linuxapp/eal/eal.c\n@@ -405,6 +405,7 @@ eal_usage(const char *prgname)\n \teal_common_usage();\n \tprintf(\"EAL Linux options:\\n\"\n \t \" --\"OPT_SOCKET_MEM\" Memory to allocate on sockets (comma separated values)\\n\"\n+\t \" --\"OPT_SOCKET_LIMIT\" Limit memory allocation on sockets (comma separated values)\\n\"\n \t \" --\"OPT_HUGE_DIR\" Directory where hugetlbfs is mounted\\n\"\n \t \" --\"OPT_FILE_PREFIX\" Prefix for hugepage filenames\\n\"\n \t \" --\"OPT_BASE_VIRTADDR\" Base virtual address\\n\"\n@@ -434,46 +435,45 @@ rte_set_application_usage_hook( rte_usage_hook_t usage_func )\n }\n \n static int\n-eal_parse_socket_mem(char *socket_mem)\n+eal_parse_socket_arg(char *strval, volatile uint64_t *socket_arg)\n {\n \tchar * arg[RTE_MAX_NUMA_NODES];\n \tchar *end;\n \tint arg_num, i, len;\n \tuint64_t total_mem = 0;\n \n-\tlen = strnlen(socket_mem, SOCKET_MEM_STRLEN);\n+\tlen = strnlen(strval, SOCKET_MEM_STRLEN);\n \tif (len == SOCKET_MEM_STRLEN) {\n \t\tRTE_LOG(ERR, EAL, \"--socket-mem is too long\\n\");\n \t\treturn -1;\n \t}\n \n \t/* all other error cases will be caught later */\n-\tif (!isdigit(socket_mem[len-1]))\n+\tif (!isdigit(strval[len-1]))\n \t\treturn -1;\n \n \t/* split the optarg into separate socket values */\n-\targ_num = rte_strsplit(socket_mem, len,\n+\targ_num = rte_strsplit(strval, len,\n \t\t\targ, RTE_MAX_NUMA_NODES, ',');\n \n \t/* if split failed, or 0 arguments */\n \tif (arg_num <= 0)\n \t\treturn -1;\n \n-\tinternal_config.force_sockets = 1;\n-\n \t/* parse each defined socket option */\n \terrno = 0;\n \tfor (i = 0; i < arg_num; i++) {\n+\t\tuint64_t val;\n \t\tend = NULL;\n-\t\tinternal_config.socket_mem[i] = strtoull(arg[i], &end, 10);\n+\t\tval = strtoull(arg[i], &end, 10);\n \n \t\t/* check for invalid input */\n \t\tif ((errno != 0) ||\n \t\t\t\t(arg[i][0] == '\\0') || (end == NULL) || (*end != '\\0'))\n \t\t\treturn -1;\n-\t\tinternal_config.socket_mem[i] *= 1024ULL;\n-\t\tinternal_config.socket_mem[i] *= 1024ULL;\n-\t\ttotal_mem += internal_config.socket_mem[i];\n+\t\tval <<= 20;\n+\t\ttotal_mem += val;\n+\t\tsocket_arg[i] = val;\n \t}\n \n \t/* check if we have a positive amount of total memory */\n@@ -621,13 +621,27 @@ eal_parse_args(int argc, char **argv)\n \t\t\tbreak;\n \n \t\tcase OPT_SOCKET_MEM_NUM:\n-\t\t\tif (eal_parse_socket_mem(optarg) < 0) {\n+\t\t\tif (eal_parse_socket_arg(optarg,\n+\t\t\t\t\tinternal_config.socket_mem) < 0) {\n \t\t\t\tRTE_LOG(ERR, EAL, \"invalid parameters for --\"\n \t\t\t\t\t\tOPT_SOCKET_MEM \"\\n\");\n \t\t\t\teal_usage(prgname);\n \t\t\t\tret = -1;\n \t\t\t\tgoto out;\n \t\t\t}\n+\t\t\tinternal_config.force_sockets = 1;\n+\t\t\tbreak;\n+\n+\t\tcase OPT_SOCKET_LIMIT_NUM:\n+\t\t\tif (eal_parse_socket_arg(optarg,\n+\t\t\t\t\tinternal_config.socket_limit) < 0) {\n+\t\t\t\tRTE_LOG(ERR, EAL, \"invalid parameters for --\"\n+\t\t\t\t\t\tOPT_SOCKET_LIMIT \"\\n\");\n+\t\t\t\teal_usage(prgname);\n+\t\t\t\tret = -1;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t\tinternal_config.force_socket_limits = 1;\n \t\t\tbreak;\n \n \t\tcase OPT_BASE_VIRTADDR_NUM:\ndiff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c\nindex c917de1c2..89d2f4d4a 100644\n--- a/lib/librte_eal/linuxapp/eal/eal_memory.c\n+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c\n@@ -1604,6 +1604,15 @@ hugepage_count_walk(const struct rte_memseg_list *msl, void *arg)\n \treturn 0;\n }\n \n+static int\n+limits_callback(int socket_id, size_t cur_limit, size_t new_len)\n+{\n+\tRTE_SET_USED(socket_id);\n+\tRTE_SET_USED(cur_limit);\n+\tRTE_SET_USED(new_len);\n+\treturn -1;\n+}\n+\n static int\n eal_hugepage_init(void)\n {\n@@ -1687,6 +1696,18 @@ eal_hugepage_init(void)\n \t\t\tfree(pages);\n \t\t}\n \t}\n+\t/* if socket limits were specified, set them */\n+\tif (internal_config.force_socket_limits) {\n+\t\tunsigned int i;\n+\t\tfor (i = 0; i < RTE_MAX_NUMA_NODES; i++) {\n+\t\t\tuint64_t limit = internal_config.socket_limit[i];\n+\t\t\tif (limit == 0)\n+\t\t\t\tcontinue;\n+\t\t\tif (rte_mem_alloc_validator_register(\"socket-limit\",\n+\t\t\t\t\tlimits_callback, i, limit))\n+\t\t\t\tRTE_LOG(ERR, EAL, \"Failed to register socket limits validator callback\\n\");\n+\t\t}\n+\t}\n \treturn 0;\n }\n \n", "prefixes": [ "dpdk-dev" ] }