diff mbox

[dpdk-dev] rte_mempool_create fails with ENOMEM

Message ID CAF28U9ORGNY7=QUKrd-ZCGn6HqBw7h6NE7wxUszf6WxOY18geg@mail.gmail.com (mailing list archive)
State Not Applicable, archived
Headers show

Commit Message

Liran Zvibel Jan. 10, 2015, 7:26 p.m. UTC
Hi Newman,

There are two options, either one of your pools is very large, and
just does not fit in half of the memory,
so if the physical memory must be split it just can never work, or
what you’re seeing is localized to your
environment, and just when allocating from both NUMAs the huge pages
just happen to be to scattered
for your pools to be allocated.

In any case, we also have to deal with large pools that don’t always
fit into consecutive huge pages as
allocated by the kernel. I have created a small patch to DPDK itself,
then some more code that can live
as part of the dpdk application that does the scattered allocation.

I’m going to send both parts here (the change to the DPDK and the user
part). I don’t know what are the
rules that allow pushing to the repository, so I won’t try to do so.

First — the DPDK patch, that just makes sure that the huge pates are
mapped in a continuous virtual memory,
and then the memory segments are allocated continuously in virtual
memory: I’m attaching full mbox content to make it easier
for you to use if you’d like. I created it against 1.7.1, since that
is the version we’re  using. If you’d like, I can also create it
against 1.8.0

                        vma_len = num_pages * hugepage_sz;

                        /* get the biggest virtual memory area up to
@@ -1268,6 +1277,16 @@ rte_eal_hugepage_init(void)
                        new_memseg = 1;

                if (new_memseg) {
+                       if (0 <= j) {
+                               RTE_LOG(DEBUG, EAL, "Closing memory
segment #%d(%p) vaddr is %p phys is 0x%lx size is 0x%lx "
+                                       "which is #%ld pages next
vaddr will be at 0x%lx\n",
+                                       j,&mcfg->memseg[j],
+                                       mcfg->memseg[j].addr,
mcfg->memseg[j].phys_addr, mcfg->memseg[j].len,
+                                       mcfg->memseg[j].len /
+                                       mcfg->memseg[j].addr_64 +
+                       }
                        j += 1;
                        if (j == RTE_MAX_MEMSEG)
1.9.3 (Apple Git-50)


Then there is the dpdk-application library part that implements the
struct rte_mempool *scattered_mempool_create(uint32_t elt_size,
uint32_t elt_num, int32_t socket_id,
*mp_init, void *mp_init_arg,
*obj_init, void *obj_init_arg)

interface. If you would like, I can easily break the different
functions into their right place in the rte_memseg and rte_mempool
DPDK modules and have it included as another interface of the DPDK
library (as suggested by Konstantin below)

static inline int  is_memseg_valid(struct rte_memseg * free_memseg,
size_t requested_page_size,
                                   int socket_id)
        if (free_memseg->len == 0) {
                return 0;

        if (socket_id != SOCKET_ID_ANY &&
            free_memseg->socket_id != SOCKET_ID_ANY &&
            free_memseg->socket_id != socket_id) {
                RTE_LOG(DEBUG, USER1, "memseg goes not qualify for
socked_id, requested %d got %d",
                         socket_id, free_memseg->socket_id);
                return 0;

        if (free_memseg->len < requested_page_size) {
                RTE_LOG(DEBUG, USER1, "memseg too small. len %lu <
requested_page_size %lu",
                         free_memseg->len, requested_page_size);
                return 0;

        if (free_memseg->hugepage_sz != requested_page_size) {
                RTE_LOG(DEBUG, USER1, "memset hugepage size !=
requested page size %lu != %lu",
                return 0;

        return 1;

static int try_allocating_memseg_range(struct rte_memseg *
free_memseg, int start,
                                       int requested_page_size, size_t
len, int socket_id)
        int i;
        for (i = start; i < RTE_MAX_MEMSEG; i++) {
                if (free_memseg[i].addr == NULL) {
                        return -1;

                if (!is_memseg_valid(free_memseg +i,
requested_page_size, socket_id)) {
                        return -1;

                if ((start != i) &&
                    ((char *)free_memseg[i].addr !=
(char*)free_memseg[i-1].addr + free_memseg[i-1].len)) {
                        RTE_LOG(DEBUG, USER1, "Looking for cont memseg range. "
                                 "[%d].vaddr %p != [%d].vaddr %p +
[i-1].len %lu == %p",
                                 i, free_memseg[i].addr, i-1,
                                 (char *)(free_memseg[i-1].addr) +
                        return -1;

                if ((free_memseg[i].len < len) && ((free_memseg[i].len
% requested_page_size) != 0)) {
                RTE_LOG(DEBUG, USER1, "#%d memseg length not a
multplie of page size, or last."
                         " len %lu len %% requsted_pg_size %lu,
requested_pg_sz %d",
                         i, free_memseg[i].len, free_memseg[i].len %
requested_page_size, requested_page_size);
                return -1;

                if (len <= free_memseg[i].len) {
                        RTE_LOG(DEBUG, USER1, "Successfuly finished
lookng for memsegs. remaining req. "
                                 "len %lu seg_len %lu, start %d i %d",
                                 len, free_memseg[i].len, start, i);
                        return i - start +1;

                if (i == start)  {
                        // We may not start on the beginning, have to
move to next pagesize alignment...
                        char * aligned_vaddr =
RTE_PTR_ALIGN_CEIL(free_memseg[i].addr, requested_page_size);
                        size_t diff = (size_t)(aligned_vaddr - (char
                        if ((free_memseg[i].len - diff) %
requested_page_size != 0) {
                                RTE_LOG(ERR, USER1, "BUG! First
segment is not page aligned! vaddr %p aligned "
                                           "vaddr %p diff %lu len %lu,
len - diff %lu, "
                                           "(len%%diff)/%d == %lu",
aligned_vaddr, diff, free_memseg[i].len,
                                           free_memseg[i].len - diff,
                                           (free_memseg[i].len - diff)
% requested_page_size);
                                return -1;
                        } else if (0 == free_memseg[i].len - diff) {
                                RTE_LOG(DEBUG, USER1, "After
alignment, first memseg is empty!");
                                return -1;

                        RTE_LOG(DEBUG, USER1, "First memseg gives
(after alignment) len %lu out of potential %lu",
                                 (free_memseg[i].len - diff),
                        len -= (free_memseg[i].len - diff);
                len -= free_memseg[i].len;

        return -1;

 * Will register several memory zones, in continueues virtual
addresses of large size.
 * All first memzones will use full pages, only the last memzone may
request less than a full hugepage.
 * It will go through all the free memory segments, once it finds a
memsegment with full hugepages, it
 * will check wheter it can start allocating from that memory segment on.
static const  struct rte_memzone *
memzone_reserve_multiple_cont_mz(const char * basename, size_t *
zones_len, size_t len, int socket_id,
                                 unsigned flags, unsigned align)
struct rte_mem_config *mcfg;
        const struct rte_memzone * ret = NULL;
        size_t requested_page_size;
        int i;
        struct rte_memseg * free_memseg = NULL;
        int first_memseg = -1;
        int memseg_count = -1;

        mcfg = rte_eal_get_configuration()->mem_config;
        free_memseg = mcfg->free_memseg;

        RTE_LOG(DEBUG, USER1, "mcfg is at %p free_memseg at %p memseg
at %p", mcfg, mcfg->free_memseg, mcfg->memseg);

        for (i = 0; i  < 10 && (free_memseg[i].addr != NULL); i++) {
                RTE_LOG(DEBUG, USER1, "free_memseg[%d] : vaddr 0x%p
phys_addr 0x%p len %lu pages: %lu [0x%lu]", i,
free_memseg[i].len, free_memseg[i].len/free_memseg[i].hugepage_sz,

        for (i = 0; i  < 10 && (mcfg->memseg[i].addr != NULL); i++) {
                RTE_LOG(DEBUG, USER1, "memseg[%d] : vaddr 0x%p
phys_addr 0x%p len %lu pages: %lu [0x%lu]", i,
                         (void*)mcfg->memseg[i].phys_addr, mcfg->memseg[i].len,

        *zones_len = 0;

        if (mcfg->memzone_idx >= RTE_MAX_MEMZONE) {
                RTE_LOG(DEBUG, USER1, "No more room for new memzones");
                return NULL;

        if ((flags & (RTE_MEMZONE_2MB | RTE_MEMZONE_1GB)) == 0) {
                RTE_LOG(DEBUG, USER1, "Must request either 2MB or 1GB pages");
                return NULL;

        if ((flags & RTE_MEMZONE_1GB ) && (flags & RTE_MEMZONE_2MB)) {
                RTE_LOG(DEBUG, USER1, "Cannot request both 1GB and 2MB pages");
                return NULL;

        if (flags & RTE_MEMZONE_2MB) {
                requested_page_size = RTE_PGSIZE_2M;
        } else {
                requested_page_size = RTE_PGSIZE_1G;

        if (len < requested_page_size) {
                RTE_LOG(DEBUG, USER1, "Requested length %lu is smaller
than requested pages size %lu",
                         len , requested_page_size);
                return NULL;

        ret = rte_memzone_reserve_aligned(basename, len, socket_id,
flags, align);
        if (ret != NULL) {
                RTE_LOG(DEBUG, USER1, "Normal
rte_memzone_reserve_aligned worked!");
                *zones_len = 1;
                return ret;

        RTE_LOG(DEBUG, USER1, "rte_memzone_reserve_aligned failed.
Will have to allocate on our own");

        for (i = 0; i < RTE_MAX_MEMSEG; i++) {
                if (free_memseg[i].addr == NULL) {

                if (!is_memseg_valid(free_memseg +i,
requested_page_size, socket_id)) {

                memseg_count =
try_allocating_memseg_range(free_memseg, i, requested_page_size, len,
                if (0 < memseg_count ) {
                        RTE_LOG(DEBUG, USER1, "Was able to find
memsegments for zone! "
                                 "first segment: %d segment_count %d len %lu",
                                 i, memseg_count, len);
                        first_memseg = i;

                        // Fix first memseg -- make sure it's page aligned!
                        char * aligned_vaddr =

                        size_t diff = (size_t)(aligned_vaddr - (char
                        RTE_LOG(DEBUG, USER1, "Decreasing first
segment by %lu", diff);
                        free_memseg[i].addr = aligned_vaddr;
                        free_memseg[i].phys_addr += diff;
                        free_memseg[i].len -= diff;
                        if ((free_memseg[i].phys_addr %
requested_page_size != 0)) {
                                RTE_LOG(ERR, USER1, "After aligning
first free memseg, "
                                           "physical address NOT page
aligned! %p",


        if (first_memseg < 0) {
                RTE_LOG(DEBUG, USER1, "Could not find memsegs to
allocate enough memory");
                goto out;

        // now perform actual allocation.
        if (mcfg->memzone_idx + memseg_count >= RTE_MAX_MEMZONE) {
                RTE_LOG(DEBUG, USER1, "There are not enough memzones
to allocate. "
                         "memzone_idx %d memseg_count %d max %s=%d",
                         mcfg->memzone_idx, memseg_count,
                goto out;

        ret = &mcfg->memzone[mcfg->memzone_idx];
        *zones_len = memseg_count;
        for (i = first_memseg; i < first_memseg + memseg_count; i++) {
                size_t allocated_length;
                if (free_memseg[i].len <= len) {
                        allocated_length = free_memseg[i].len;
                } else {
                        allocated_length = len;

                struct rte_memzone * mz = &mcfg->memzone[mcfg->memzone_idx++];
                snprintf(mz->name, sizeof(mz->name), "%s%d", basename,
i - first_memseg);
                mz->phys_addr   = free_memseg[i].phys_addr;
                mz->addr        = free_memseg[i].addr;
                mz->len         = allocated_length;
                mz->hugepage_sz = free_memseg[i].hugepage_sz;
                mz->socket_id   = free_memseg[i].socket_id;
                mz->flags       = 0;
                mz->memseg_id   = i;

                free_memseg[i].len -= allocated_length;
                free_memseg[i].phys_addr += allocated_length;
                free_memseg[i].addr_64 += allocated_length;
                len -= allocated_length;

        if (len != 0) {
                RTE_LOG(DEBUG, USER1, "After registering all the
memzone, len is too small! Len is %lu", len);
                ret = NULL;
                goto out;
        return ret;

static inline void build_physical_pages(phys_addr_t * phys_pages, int
num_phys_pages, size_t sz,
                                        const struct rte_memzone * mz,
int num_zones)
        size_t accounted_for_size =0;
        int curr_page = 0;
        int i;
        unsigned j;

        RTE_LOG(DEBUG, USER1, "Phys pages are at %p 2M is %d mz
pagesize is %lu trailing zeros: %d",
                 phys_pages, RTE_PGSIZE_2M, mz->hugepage_sz,

        for (i = 0; i < num_zones; i++) {
                size_t mz_remaining_len = mz[i].len;
                for (j = 0; (j <= mz[i].len / RTE_PGSIZE_2M) && (0 <
mz_remaining_len) ; j++) {
                        phys_pages[curr_page++] = mz[i].phys_addr + j

                        size_t added_len =
RTE_MIN((size_t)RTE_PGSIZE_2M, mz_remaining_len);
                        accounted_for_size += added_len;
                        mz_remaining_len -= added_len;

                        if (sz <= accounted_for_size) {
                                RTE_LOG(DEBUG, USER1, "Filled in %d
pages of the physical pages array", curr_page);
                        if (num_phys_pages < curr_page) {
                                RTE_LOG(ERR, USER1, "When building
physcial pages array, "
                                           "used pages (%d) is more
than allocated pages %d. "
                                           "accounted size %lu size %lu",
                                           curr_page, num_phys_pages,
accounted_for_size, sz);

        if (accounted_for_size < sz) {
                RTE_LOG(ERR, USER1, "Finished going over %d memory
zones, and still accounted size is %lu "
                           "and requested size is %lu",
                           num_zones, accounted_for_size, sz);

struct rte_mempool *scattered_mempool_create(uint32_t elt_size,
uint32_t elt_num, int32_t socket_id,
*mp_init, void *mp_init_arg,
*obj_init, void *obj_init_arg)
        struct rte_mempool *mp;
        const struct rte_memzone *mz;
        size_t                          num_zones;
        struct rte_mempool_objsz obj_sz;
        uint32_t flags, total_size;
        size_t sz;


        total_size = rte_mempool_calc_obj_size(elt_size, flags, &obj_sz);

        sz = elt_num * total_size;
        /* We now have to account for the "gaps" at the end of each
page. Worst case is that we get
         * all distinct pages, so we have to add the gap for each
possible page */
        int pages_num = (sz + RTE_PGSIZE_2M -1) / RTE_PGSIZE_2M;
        int page_gap = RTE_PGSIZE_2M % elt_size;
        sz += pages_num + page_gap;

        RTE_LOG(DEBUG, USER1, "Will have to allocate %d 2M pages for
the page table.", pages_num);

        if ((mz = memzone_reserve_multiple_cont_mz("data_obj",
&num_zones, sz, socket_id,
                RTE_LOG(WARNING, USER1, "memzone reserve multi mz
returned NULL for socket id %d, will try ANY",
                if ((mz =
&num_zones, sz, socket_id,
                        RTE_LOG(ERR, USER1, "memzone reserve multi mz
returned NULL even for any socket");
                        return NULL;
                } else {
                        RTE_LOG(DEBUG, USER1, "memzone reserve multi
mz returne %p with %lu zones for SOCKET_ID_ANY",
                                 mz, num_zones);
        } else {
                RTE_LOG(DEBUG, USER1, "memzone reserve multi mz
returned %p with %lu zones for size %lu  socket %d",
                         mz, num_zones, sz, socket_id);

        // Now will "break" the pages into smaller ones
        phys_addr_t * phys_pages = malloc(sizeof(phys_addr_t)*pages_num);
        if(phys_pages == NULL) {
            RTE_LOG(DEBUG, USER1, "phys_pages is null. aborting");

        build_physical_pages(phys_pages, pages_num, sz, mz, num_zones);
        RTE_LOG(DEBUG, USER1, "Beginning of vaddr is %p beginning of
physical addr is 0x%lx", mz->addr, mz->phys_addr);
        mp = rte_mempool_xmem_create("data_pool", elt_num, elt_size,
                                     257 , sizeof(struct
                                     mp_init, mp_init_arg, obj_init,
                                     socket_id, flags, (char *)mz[0].addr,
                                     phys_pages, pages_num,

        RTE_LOG(DEBUG, USER1, "rte_mempool_xmem_create returned %p", mp);
        return mp;


Please let me know if you have any questions/comments about this code.

Best Regards,


On Jan 8, 2015, at 10:19, Newman Poborsky <newman555p@gmail.com> wrote:

I finally found the time to try this and I noticed that on a server
with 1 NUMA node, this works, but if  server has 2 NUMA nodes than by
default memory policy, reserved hugepages are divided on each node and
again DPDK test app fails for the reason already mentioned. I found
out that 'solution' for this is to deallocate hugepages on node1
(after boot) and leave them only on node0:
echo 0 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages

Could someone please explain what changes when there are hugepages on
both nodes? Does this cause some memory fragmentation so that there
aren't enough contiguous segments? If so, how?



On Mon, Dec 22, 2014 at 11:48 AM, Newman Poborsky <newman555p@gmail.com> wrote:

On Sat, Dec 20, 2014 at 2:34 AM, Stephen Hemminger
<stephen@networkplumber.org> wrote:

You can reserve hugepages on the kernel cmdline (GRUB).

Great, thanks, I'll try that!


On Fri, Dec 19, 2014 at 12:13 PM, Newman Poborsky <newman555p@gmail.com>

On Thu, Dec 18, 2014 at 9:03 PM, Ananyev, Konstantin <
konstantin.ananyev@intel.com> wrote:

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ananyev,
Sent: Thursday, December 18, 2014 5:43 PM
To: Newman Poborsky; dev@dpdk.org
Subject: Re: [dpdk-dev] rte_mempool_create fails with ENOMEM


-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Newman Poborsky
Sent: Thursday, December 18, 2014 1:26 PM
To: dev@dpdk.org
Subject: [dpdk-dev] rte_mempool_create fails with ENOMEM


could someone please provide any explanation why sometimes mempool


fails with ENOMEM?

I run my test app several times without any problems and then I
getting ENOMEM error when creating mempool that are used for

I try

to delete everything from /mnt/huge, I increase the number of huge


remount /mnt/huge but nothing helps.

There is more than enough memory on server. I tried to debug
rte_mempool_create() call and it seems that after server is


mem segments are bigger than 2MB, but after running test app for


times, it seems that all free mem segments have a size of 2MB, and

since I

am requesting 8MB for my packet mempool, this fails.  I'm not really


that this conclusion is correct.

Yes,rte_mempool_create uses  rte_memzone_reserve() to allocate
single physically continuous chunk of memory.
If no such chunk exist, then it would fail.
Why physically continuous?
Main reason - to make things easier for us, as in that case we don't

have to worry

about situation when mbuf crosses page boundary.
So you can overcome that problem like that:
Allocate max amount of memory you would need to hold all mbufs in

case (all pages physically disjoint)

using rte_malloc().

Actually my wrong: rte_malloc()s wouldn't help you here.
You probably need to allocate some external (not managed by EAL) memory
that case,
may be mmap() with MAP_HUGETLB, or something similar.

Figure out it's physical mappings.
Call  rte_mempool_xmem_create().
You can look at: app/test-pmd/mempool_anon.c as a reference.
It uses same approach to create mempool over 4K pages.

We probably add similar function into mempool API

(create_scatter_mempool or something)

or just add a new flag (USE_SCATTER_MEM) into rte_mempool_create().
Though right now it is not there.

Another quick alternative - use 1G pages.


Ok, thanks for the explanation. I understand that this is probably an OS
question more than DPDK, but is there a way to again allocate a contiguous
memory for n-th run of my test app?  It seems that hugepages get
divded/separated to individual 2MB hugepage. Shouldn't OS's memory
management system try to group those hupages back to one contiguous chunk
once my app/process is done?   Again, I know very little about Linux
management and hugepages, so forgive me if this is a stupid question.
Is rebooting the OS the only way to deal with this problem?  Or should I
just try to use 1GB hugepages?

p.s. Konstantin, sorry for the double reply, I accidentally forgot to
include dev list in my first reply  :)


Does anybody have any idea what to check and how running my test app
several times affects hugepages?

For me, this doesn't make any since because after test app exits,


should be freed, right?

This has been driving me crazy for days now. I tried reading a bit
theory about hugepages, but didn't find out anything that could help


Maybe it's something else and completely trivial, but I can't figure
out, so any help is appreciated.

Thank you!

Newman P.
diff mbox



From 10ebc74eda2c3fe9e5a34815e0f7ee1f44d99aa3 Mon Sep 17 00:00:00 2001
From: Liran Zvibel <liran@weka.io>
Date: Sat, 10 Jan 2015 12:46:54 +0200
Subject: [PATCH] Add an option to allocate huge pages in contiunous virtual
To: dev@dpdk.org

that advises the memory sengment allocation code to allocate as many
hugemages in a continuous way in virtual addresses as possible.

This way, a mempool may be created out of disparsed memzones allocated
from these new continuos memory segments.
 lib/librte_eal/linuxapp/eal/eal_memory.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c
index f2454f4..b8d68b0 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -329,6 +329,7 @@  map_all_hugepages(struct hugepage_file *hugepg_tbl,

                else if (vma_len == 0) {
                        unsigned j, num_pages;

                        /* reserve a virtual area for next contiguous
@@ -340,6 +341,14 @@  map_all_hugepages(struct hugepage_file *hugepg_tbl,
                        num_pages = j - i;
+#else // hugepages are will be allocated in a continous virtual address way
+                       unsigned num_pages;
+                       /* We will reserve a virtual area large enough
to fit ALL
+                        * physical blocks.
+                        * This way we can have bigger mempools even
if there is no
+                        * continuos physcial region.
+                       num_pages = hpi->num_pages[0] - i;