From patchwork Sat Jan 27 18:31:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 34566 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B11B61E35; Sat, 27 Jan 2018 19:31:27 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 8A4F21E35 for ; Sat, 27 Jan 2018 19:31:24 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Jan 2018 10:31:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,423,1511856000"; d="scan'208";a="199296827" Received: from silpixa00398672.ir.intel.com ([10.237.223.111]) by fmsmga006.fm.intel.com with ESMTP; 27 Jan 2018 10:31:21 -0800 From: Harry van Haaren To: dev@dpdk.org Cc: thomas@monjalon.net, Harry van Haaren , vipin.varghese@intel.com Date: Sat, 27 Jan 2018 18:31:14 +0000 Message-Id: <1517077877-53654-1-git-send-email-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [dpdk-dev] [PATCH 1/4] service: move finalize to internal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit moves the rte_service_finalize() function to be in the component header, and marks it as @internal. The function is only called internally by rte_eal_finalize(). Signed-off-by: Harry van Haaren --- Cc: thomas@monjalon.net Cc: vipin.varghese@intel.com --- lib/librte_eal/common/include/rte_service.h | 11 ----------- lib/librte_eal/common/include/rte_service_component.h | 11 +++++++++++ 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/lib/librte_eal/common/include/rte_service.h b/lib/librte_eal/common/include/rte_service.h index 5e3e3a6..02b1512 100644 --- a/lib/librte_eal/common/include/rte_service.h +++ b/lib/librte_eal/common/include/rte_service.h @@ -429,17 +429,6 @@ int32_t rte_service_attr_get(uint32_t id, uint32_t attr_id, */ int32_t rte_service_attr_reset_all(uint32_t id); -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Free up the memory that has been initialized. This routine - * is to be invoked prior to process termination. - * - * @retval None - */ -void rte_service_finalize(void); - #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/common/include/rte_service_component.h b/lib/librte_eal/common/include/rte_service_component.h index 849c71d..f881ac0 100644 --- a/lib/librte_eal/common/include/rte_service_component.h +++ b/lib/librte_eal/common/include/rte_service_component.h @@ -139,4 +139,15 @@ int32_t rte_service_component_runstate_set(uint32_t id, uint32_t runstate); */ int32_t rte_service_init(void); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * @internal Free up the memory that has been initialized. + * This routine is to be invoked prior to process termination. + * + * @retval None + */ +void rte_service_finalize(void); + #endif /* _RTE_SERVICE_PRIVATE_H_ */ From patchwork Sat Jan 27 18:31:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 34567 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B38031B171; Sat, 27 Jan 2018 19:31:29 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 8868719F5 for ; Sat, 27 Jan 2018 19:31:25 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Jan 2018 10:31:24 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,423,1511856000"; d="scan'208";a="199296832" Received: from silpixa00398672.ir.intel.com ([10.237.223.111]) by fmsmga006.fm.intel.com with ESMTP; 27 Jan 2018 10:31:23 -0800 From: Harry van Haaren To: dev@dpdk.org Cc: thomas@monjalon.net, Harry van Haaren , vipin.varghese@intel.com Date: Sat, 27 Jan 2018 18:31:15 +0000 Message-Id: <1517077877-53654-2-git-send-email-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1517077877-53654-1-git-send-email-harry.van.haaren@intel.com> References: <1517077877-53654-1-git-send-email-harry.van.haaren@intel.com> Subject: [dpdk-dev] [PATCH 2/4] eal: add function to release internal resources X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds a new function rte_eal_finalize(). The function serves as a hook to allow DPDK to release internal resources (e.g.: hugepage allocations). This function allows DPDK to become more like an ordinary library, where the library context itself can be initialized and finalized by the application. The rte_exit() and rte_panic() functions must be considered, particularly if they should call finalize() to cleanup any resources or not. This patch adds the cleanup to rte_exit(), but does not clean up on rte_panic(). The reason to not clean up on panicing is that the developer may wish to inspect the exact internal state of EAL. Signed-off-by: Harry van Haaren --- Cc: thomas@monjalon.net Cc: vipin.varghese@intel.com --- doc/guides/prog_guide/env_abstraction_layer.rst | 8 ++++++++ doc/guides/rel_notes/release_18_02.rst | 9 +++++++++ lib/librte_eal/bsdapp/eal/Makefile | 1 + lib/librte_eal/bsdapp/eal/eal_debug.c | 5 +++++ lib/librte_eal/common/include/rte_eal.h | 15 +++++++++++++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_debug.c | 5 +++++ lib/librte_eal/rte_eal_version.map | 1 + 8 files changed, 45 insertions(+) diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 34d871c..f020041 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -99,6 +99,14 @@ It consist of calls to the pthread library (more specifically, pthread_self(), p The creation and initialization functions for these objects are not multi-thread safe. However, once initialized, the objects themselves can safely be used in multiple threads simultaneously. +Finalizing and Cleanup +~~~~~~~~~~~~~~~~~~~~~~ + +During the initialization of EAL resources such as hugepage backed memory can be +allocated by core components. The memory allocated during ``rte_eal_init()`` +can be released by calling the ``rte_eal_finalize()`` function. Refer to the +API documentation for details. + Multi-process Support ~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/rel_notes/release_18_02.rst b/doc/guides/rel_notes/release_18_02.rst index 00b3224..5c7410e 100644 --- a/doc/guides/rel_notes/release_18_02.rst +++ b/doc/guides/rel_notes/release_18_02.rst @@ -41,6 +41,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **Add function to allow releasing internal EAL resources on exit** + + During ``rte_eal_init()`` EAL allocates memory from hugepages to enable its + core libraries to perform their tasks. The ``rte_eal_finalize()`` function + releases these resources, ensuring that no hugepage memory is leaked. It is + expected that all DPDK applications call ``rte_eal_finalize()`` before + exiting. Not calling this function could result in leaking hugepages, leading + to failure during initialization of secondary processes. + * **Added the ixgbe ethernet driver to support RSS with flow API.** Rte_flow actually defined to include RSS, but till now, RSS is out of diff --git a/lib/librte_eal/bsdapp/eal/Makefile b/lib/librte_eal/bsdapp/eal/Makefile index c694076..7480f98 100644 --- a/lib/librte_eal/bsdapp/eal/Makefile +++ b/lib/librte_eal/bsdapp/eal/Makefile @@ -34,6 +34,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_interrupts.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_alarm.c # from common dir +SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_lcore.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_timer.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_memzone.c diff --git a/lib/librte_eal/bsdapp/eal/eal_debug.c b/lib/librte_eal/bsdapp/eal/eal_debug.c index b0ae2b7..1a17ce3 100644 --- a/lib/librte_eal/bsdapp/eal/eal_debug.c +++ b/lib/librte_eal/bsdapp/eal/eal_debug.c @@ -14,6 +14,7 @@ #include #include #include +#include #define BACKTRACE_SIZE 256 @@ -79,6 +80,10 @@ rte_exit(int exit_code, const char *format, ...) va_end(ap); #ifndef RTE_EAL_ALWAYS_PANIC_ON_ERROR + int ret = rte_eal_finalize(); + if (ret) + RTE_LOG(CRIT, EAL, + "EAL could not release all resources, code %d\n", ret); exit(exit_code); #else rte_dump_stack(); diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h index 2aba2c8..b2f5869 100644 --- a/lib/librte_eal/common/include/rte_eal.h +++ b/lib/librte_eal/common/include/rte_eal.h @@ -170,6 +170,21 @@ int rte_eal_iopl_init(void); int rte_eal_init(int argc, char **argv); /** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * Finalize the Environment Abstraction Layer (EAL) + * + * This function must be called to release any internal resources that EAL has + * allocated during rte_eal_init(). After this call, no DPDK function calls may + * be made. It is expected that common usage of this function is to call it + * just before terminating the process. + * + * @return 0 Successfully released all internal EAL resources + * @return -EFAULT There was an error in releasing all resources. + */ +int rte_eal_finalize(void); + +/** * Check if a primary process is currently alive * * This function returns true when a primary process is currently diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index 7bf278f..ad20f2f 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -41,6 +41,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_interrupts.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_alarm.c # from common dir +SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_lcore.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_timer.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_memzone.c diff --git a/lib/librte_eal/linuxapp/eal/eal_debug.c b/lib/librte_eal/linuxapp/eal/eal_debug.c index b0ae2b7..1a17ce3 100644 --- a/lib/librte_eal/linuxapp/eal/eal_debug.c +++ b/lib/librte_eal/linuxapp/eal/eal_debug.c @@ -14,6 +14,7 @@ #include #include #include +#include #define BACKTRACE_SIZE 256 @@ -79,6 +80,10 @@ rte_exit(int exit_code, const char *format, ...) va_end(ap); #ifndef RTE_EAL_ALWAYS_PANIC_ON_ERROR + int ret = rte_eal_finalize(); + if (ret) + RTE_LOG(CRIT, EAL, + "EAL could not release all resources, code %d\n", ret); exit(exit_code); #else rte_dump_stack(); diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 1a8b1b5..c906305 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -215,6 +215,7 @@ EXPERIMENTAL { rte_eal_devargs_insert; rte_eal_devargs_parse; rte_eal_devargs_remove; + rte_eal_finalize; rte_eal_hotplug_add; rte_eal_hotplug_remove; rte_service_attr_get; From patchwork Sat Jan 27 18:31:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 34568 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 52AA91B1AF; Sat, 27 Jan 2018 19:31:35 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id E54A12BBD for ; Sat, 27 Jan 2018 19:31:26 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Jan 2018 10:31:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,423,1511856000"; d="scan'208";a="199296837" Received: from silpixa00398672.ir.intel.com ([10.237.223.111]) by fmsmga006.fm.intel.com with ESMTP; 27 Jan 2018 10:31:24 -0800 From: Harry van Haaren To: dev@dpdk.org Cc: thomas@monjalon.net, Harry van Haaren , vipin.varghese@intel.com Date: Sat, 27 Jan 2018 18:31:16 +0000 Message-Id: <1517077877-53654-3-git-send-email-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1517077877-53654-1-git-send-email-harry.van.haaren@intel.com> References: <1517077877-53654-1-git-send-email-harry.van.haaren@intel.com> Subject: [dpdk-dev] [PATCH 3/4] app/pdump: call eal finalize before exit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds a call to the newly introduced finalize() function just before quitting the pdump app. Adding this function call before quitting from a secondary processes is important, as otherwise it will leak hugepage memory. For a secondary process that is run multiple times, this could cause hugepage memory to become depleted and stop a secondary process from starting. Signed-off-by: Harry van Haaren --- Cc: thomas@monjalon.net Cc: vipin.varghese@intel.com --- app/pdump/main.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/app/pdump/main.c b/app/pdump/main.c index 0f70c75..6c58cda 100644 --- a/app/pdump/main.c +++ b/app/pdump/main.c @@ -882,5 +882,10 @@ main(int argc, char **argv) /* dump debug stats */ print_pdump_stats(); + ret = rte_eal_finalize(); + if (ret) + printf("Error from rte_eal_finalize(), %d\n", ret); + + return 0; } From patchwork Sat Jan 27 18:31:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 34569 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 804D01B1BA; Sat, 27 Jan 2018 19:31:36 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 31C9F1B04C for ; Sat, 27 Jan 2018 19:31:27 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Jan 2018 10:31:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,423,1511856000"; d="scan'208";a="199296843" Received: from silpixa00398672.ir.intel.com ([10.237.223.111]) by fmsmga006.fm.intel.com with ESMTP; 27 Jan 2018 10:31:26 -0800 From: Harry van Haaren To: dev@dpdk.org Cc: thomas@monjalon.net, Harry van Haaren , vipin.varghese@intel.com Date: Sat, 27 Jan 2018 18:31:17 +0000 Message-Id: <1517077877-53654-4-git-send-email-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1517077877-53654-1-git-send-email-harry.van.haaren@intel.com> References: <1517077877-53654-1-git-send-email-harry.van.haaren@intel.com> Subject: [dpdk-dev] [PATCH 4/4] app/proc_info: call eal finalize before exit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds a call to the newly introduced finalize() function just before quitting the app. Adding this function call before quitting from a secondary processes is important, as otherwise it will leak hugepage memory. For a secondary process that is run multiple times, this could cause hugepage memory to become depleted and stop a secondary process from starting. Signed-off-by: Harry van Haaren --- Cc: thomas@monjalon.net Cc: vipin.varghese@intel.com --- app/proc_info/main.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/proc_info/main.c b/app/proc_info/main.c index 94d53f5..44efbc4 100644 --- a/app/proc_info/main.c +++ b/app/proc_info/main.c @@ -660,5 +660,9 @@ main(int argc, char **argv) if (enable_metrics) metrics_display(RTE_METRICS_GLOBAL); + ret = rte_eal_finalize(); + if (ret) + printf("Error from rte_eal_finalize(), %d\n", ret); + return 0; }