diff mbox series

[v3,3/9] mem: raise maximum fd limit unconditionally

Message ID	53a660ffd746c81f59fc3e1f35ac2222a7237641.1536073997.git.anatoly.burakov@intel.com (mailing list archive)
State	Accepted, archived
Delegated to:	Thomas Monjalon
Headers	From: Anatoly Burakov <anatoly.burakov@intel.com> To: dev@dpdk.org Cc: tiwei.bie@intel.com, ray.kinsella@intel.com, zhihong.wang@intel.com, maxime.coquelin@redhat.com, kuralamudhan.ramakrishnan@intel.com Date: Tue, 4 Sep 2018 16:15:44 +0100 Message-Id: <53a660ffd746c81f59fc3e1f35ac2222a7237641.1536073997.git.anatoly.burakov@intel.com> In-Reply-To: <cover.1536073996.git.anatoly.burakov@intel.com> References: <cover.1536073996.git.anatoly.burakov@intel.com> In-Reply-To: <cover.1536073996.git.anatoly.burakov@intel.com> References: <cover.1536072550.git.anatoly.burakov@intel.com> <cover.1536073996.git.anatoly.burakov@intel.com> Subject: [dpdk-dev] [PATCH v3 3/9] mem: raise maximum fd limit unconditionally Precedence: list Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org>
Series	Improve running DPDK without hugetlbfs mounpoint \| [v3,0/9] Improve running DPDK without hugetlbfs mounpoint [v3,1/9] fbarray: fix detach in noshconf mode [v3,2/9] eal: don't allow legacy mode with in-memory mode [v3,3/9] mem: raise maximum fd limit unconditionally [v3,4/9] memalloc: rename lock list to fd list [v3,5/9] memalloc: track page fd's in non-single file mode [v3,6/9] memalloc: add EAL-internal API to get and set segment fd's [v3,7/9] mem: add external API to retrieve page fd from EAL [v3,8/9] mem: allow querying offset into segment fd [v3,9/9] mem: support using memfd segments for in-memory mode

Checks

Context	Check	Description
ci/checkpatch	success	coding style OK
ci/Intel-compilation	success	Compilation OK

Commit Message

Burakov, Anatoly Sept. 4, 2018, 3:15 p.m. UTC

  Previously, when we allocated hugepages, we closed the fd's corresponding
to them after we've done our mappings. Since we did mmap(), we didn't
actually lose the reference, but file descriptors used for mmap() do not
count against the fd limit. Since we are going to store all of our fd's,
we will hit the fd limit much more often when using smaller page sizes.

Fix this to raise the fd limit to maximum unconditionally.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 lib/librte_eal/linuxapp/eal/eal_memory.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

Comments

Maxime Coquelin Sept. 13, 2018, 1:12 p.m. UTC | #1

On 09/04/2018 05:15 PM, Anatoly Burakov wrote:
> Previously, when we allocated hugepages, we closed the fd's corresponding
> to them after we've done our mappings. Since we did mmap(), we didn't
> actually lose the reference, but file descriptors used for mmap() do not
> count against the fd limit. Since we are going to store all of our fd's,
> we will hit the fd limit much more often when using smaller page sizes.
> 
> Fix this to raise the fd limit to maximum unconditionally.
> 
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
>   lib/librte_eal/linuxapp/eal/eal_memory.c | 20 ++++++++++++++++++++
>   1 file changed, 20 insertions(+)
> 
> diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
> index dbf19499e..dfb537f59 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_memory.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
> @@ -17,6 +17,7 @@
>   #include <sys/stat.h>
>   #include <sys/queue.h>
>   #include <sys/file.h>
> +#include <sys/resource.h>
>   #include <unistd.h>
>   #include <limits.h>
>   #include <sys/ioctl.h>
> @@ -2204,6 +2205,25 @@ memseg_secondary_init(void)
>   int
>   rte_eal_memseg_init(void)
>   {
> +	/* increase rlimit to maximum */
> +	struct rlimit lim;
> +
> +	if (getrlimit(RLIMIT_NOFILE, &lim) == 0) {
> +		/* set limit to maximum */
> +		lim.rlim_cur = lim.rlim_max;
> +
> +		if (setrlimit(RLIMIT_NOFILE, &lim) < 0) {
> +			RTE_LOG(DEBUG, EAL, "Setting maximum number of open files failed: %s\n",
> +					strerror(errno));
> +		} else {
> +			RTE_LOG(DEBUG, EAL, "Setting maximum number of open files to %"
> +					PRIu64 "\n",
> +					(uint64_t)lim.rlim_cur);
> +		}
> +	} else {
> +		RTE_LOG(ERR, EAL, "Cannot get current resource limits\n");
> +	}
> +
>   	return rte_eal_process_type() == RTE_PROC_PRIMARY ?
>   #ifndef RTE_ARCH_64
>   			memseg_primary_init_32() :
> 

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

diff mbox series

Patch

diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
index dbf19499e..dfb537f59 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -17,6 +17,7 @@ 
 #include <sys/stat.h>
 #include <sys/queue.h>
 #include <sys/file.h>
+#include <sys/resource.h>
 #include <unistd.h>
 #include <limits.h>
 #include <sys/ioctl.h>
@@ -2204,6 +2205,25 @@  memseg_secondary_init(void)
 int
 rte_eal_memseg_init(void)
 {
+	/* increase rlimit to maximum */
+	struct rlimit lim;
+
+	if (getrlimit(RLIMIT_NOFILE, &lim) == 0) {
+		/* set limit to maximum */
+		lim.rlim_cur = lim.rlim_max;
+
+		if (setrlimit(RLIMIT_NOFILE, &lim) < 0) {
+			RTE_LOG(DEBUG, EAL, "Setting maximum number of open files failed: %s\n",
+					strerror(errno));
+		} else {
+			RTE_LOG(DEBUG, EAL, "Setting maximum number of open files to %"
+					PRIu64 "\n",
+					(uint64_t)lim.rlim_cur);
+		}
+	} else {
+		RTE_LOG(ERR, EAL, "Cannot get current resource limits\n");
+	}
+
 	return rte_eal_process_type() == RTE_PROC_PRIMARY ?
 #ifndef RTE_ARCH_64
 			memseg_primary_init_32() :