[v3,0/9] Improve running DPDK without hugetlbfs mounpoint

  This patchset further improves DPDK support for running
without hugetlbfs mountpoints.

First of all, it enables using memfd-created hugepages in
in-memory mode. This way, instead of anonymous hugepages, we
can get proper fd's for each page (or for the entire segment,
if we're using single-file segments). Memfd will be used
automatically if support for it was compiled and is available
at runtime, however DPDK will fall back to using anonymous
hugepages if such support is not available.

The other thing this patchset does is exposing segment fd's
through an external API. There is a lot of ugliness in current
virtio/vhost code that deals with finding hugepage files
through procfs, while all virtio really needs are fd's
referring to the pages, and their offsets. Using this API,
virtio will be able to access segment fd's directly, without
the procfs magic.

As a bonus, because we enabled use of memfd (given that
sufficiently recent kernel version is used), once virtio
support for getting segment fd's using the new API is
implemented, virtio will also be able to work without having
hugetlbfs mountpoints.

Virtio support is not provided in this patchset, coordination
and implementation of it is up to virtio maintainers.

Once virtio support for this is in place, DPDK will have one
less barrier for adoption in container space.

v2->v3:
- Fixed single file segments mode for fd offsets

v1->v2:
- Added a new API to retrieve segment offset into its fd

Anatoly Burakov (9):
  fbarray: fix detach in noshconf mode
  eal: don't allow legacy mode with in-memory mode
  mem: raise maximum fd limit unconditionally
  memalloc: rename lock list to fd list
  memalloc: track page fd's in non-single file mode
  memalloc: add EAL-internal API to get and set segment fd's
  mem: add external API to retrieve page fd from EAL
  mem: allow querying offset into segment fd
  mem: support using memfd segments for in-memory mode

 lib/librte_eal/bsdapp/eal/eal_memalloc.c   |  19 +
 lib/librte_eal/common/eal_common_fbarray.c |   4 +
 lib/librte_eal/common/eal_common_memory.c  | 107 ++++-
 lib/librte_eal/common/eal_common_options.c |  12 +-
 lib/librte_eal/common/eal_memalloc.h       |  11 +
 lib/librte_eal/common/include/rte_memory.h |  97 +++++
 lib/librte_eal/linuxapp/eal/eal_memalloc.c | 452 +++++++++++++++++----
 lib/librte_eal/linuxapp/eal/eal_memory.c   |  64 ++-
 lib/librte_eal/rte_eal_version.map         |   4 +
 9 files changed, 672 insertions(+), 98 deletions(-)
  

> Anatoly Burakov (9):
>   fbarray: fix detach in noshconf mode
>   eal: don't allow legacy mode with in-memory mode
>   mem: raise maximum fd limit unconditionally
>   memalloc: rename lock list to fd list
>   memalloc: track page fd's in non-single file mode
>   memalloc: add EAL-internal API to get and set segment fd's
>   mem: add external API to retrieve page fd from EAL
>   mem: allow querying offset into segment fd
>   mem: support using memfd segments for in-memory mode

Applied, tha	nks
  

On 19-Sep-18 2:04 PM, Thomas Monjalon wrote:
>> Anatoly Burakov (9):
>>    fbarray: fix detach in noshconf mode
>>    eal: don't allow legacy mode with in-memory mode
>>    mem: raise maximum fd limit unconditionally
>>    memalloc: rename lock list to fd list
>>    memalloc: track page fd's in non-single file mode
>>    memalloc: add EAL-internal API to get and set segment fd's
>>    mem: add external API to retrieve page fd from EAL
>>    mem: allow querying offset into segment fd
>>    mem: support using memfd segments for in-memory mode
> 
> Applied, tha	nks
> 

You're welc	ome!
  

19/09/2018 15:55, Burakov, Anatoly:
> On 19-Sep-18 2:04 PM, Thomas Monjalon wrote:
> >> Anatoly Burakov (9):
> >>    fbarray: fix detach in noshconf mode
> >>    eal: don't allow legacy mode with in-memory mode
> >>    mem: raise maximum fd limit unconditionally
> >>    memalloc: rename lock list to fd list
> >>    memalloc: track page fd's in non-single file mode
> >>    memalloc: add EAL-internal API to get and set segment fd's
> >>    mem: add external API to retrieve page fd from EAL
> >>    mem: allow querying offset into segment fd
> >>    mem: support using memfd segments for in-memory mode
> > 
> > Applied, tha	nks
> > 
> 
> You're welc	ome!

On a french keyboard, "tab" is the key next to "a", so I may forgive myself.
But looking at "c" and "o" layout, I think your fingers are way bigger than mines :)