[v3,0/4] Allow using external memory without malloc

Currently, the only way to use externally allocated memory
is through rte_malloc API's. While this is fine for a lot
of use cases, it may not be suitable for certain other use
cases like manual memory management, etc.

This patchset adds another API to register memory segments
with DPDK (so that API's like ``rte_mem_virt2memseg`` could
be relied on by PMD's and such), but not create a malloc
heap out of them.

Aside from the obvious (not adding memory to a heap), the
other major difference between this API and the
``rte_malloc_heap_*`` external memory functions is the fact
that no DMA mapping is performed automatically, as well as
no mem event callbacks are triggered.

This really draws a line in the sand, and there are now two
ways of doing things - do everything automatically (using
the ``rte_malloc_heap_*`` API's), or do everything manually
(``rte_extmem_*`` and future DMA mapping API [1] that would
replace ``rte_vfio_dma_map``). This way, the consistency of
API is kept, and flexibility is also allowed.

[1] https://mails.dpdk.org/archives/dev/2018-November/118175.html

v3:
- Rebase on latest master

v2:
- More sanity checking of parameters

Anatoly Burakov (4):
  malloc: separate creating memseg list and malloc heap
  malloc: separate destroying memseg list and heap data
  mem: allow registering external memory areas
  mem: allow usage of non-heap external memory in multiprocess

 .../prog_guide/env_abstraction_layer.rst      |  63 +++++++--
 doc/guides/rel_notes/release_19_02.rst        |   7 +
 lib/librte_eal/common/eal_common_memory.c     | 119 +++++++++++++++++
 lib/librte_eal/common/include/rte_memory.h    | 122 ++++++++++++++++++
 lib/librte_eal/common/malloc_heap.c           | 104 +++++++++++----
 lib/librte_eal/common/malloc_heap.h           |  15 ++-
 lib/librte_eal/common/rte_malloc.c            | 116 +++++++----------
 lib/librte_eal/rte_eal_version.map            |   4 +
 8 files changed, 447 insertions(+), 103 deletions(-)

On Thu, 20 Dec 2018 15:32:37 +0000
Anatoly Burakov <anatoly.burakov@intel.com> wrote:

> Currently, the only way to use externally allocated memory
> is through rte_malloc API's. While this is fine for a lot
> of use cases, it may not be suitable for certain other use
> cases like manual memory management, etc.
> 
> This patchset adds another API to register memory segments
> with DPDK (so that API's like ``rte_mem_virt2memseg`` could
> be relied on by PMD's and such), but not create a malloc
> heap out of them.
> 
> Aside from the obvious (not adding memory to a heap), the
> other major difference between this API and the
> ``rte_malloc_heap_*`` external memory functions is the fact
> that no DMA mapping is performed automatically, as well as
> no mem event callbacks are triggered.
> 
> This really draws a line in the sand, and there are now two
> ways of doing things - do everything automatically (using
> the ``rte_malloc_heap_*`` API's), or do everything manually
> (``rte_extmem_*`` and future DMA mapping API [1] that would
> replace ``rte_vfio_dma_map``). This way, the consistency of
> API is kept, and flexibility is also allowed.
> 
> [1] https://mails.dpdk.org/archives/dev/2018-November/118175.html

Where are the examples for this? Give a sample application maybe.

Also there are no test cases.

> Anatoly Burakov (4):
>   malloc: separate creating memseg list and malloc heap
>   malloc: separate destroying memseg list and heap data
>   mem: allow registering external memory areas
>   mem: allow usage of non-heap external memory in multiprocess

Applied, thanks

20/12/2018 17:16, Stephen Hemminger:
> On Thu, 20 Dec 2018 15:32:37 +0000
> Anatoly Burakov <anatoly.burakov@intel.com> wrote:
> 
> > Currently, the only way to use externally allocated memory
> > is through rte_malloc API's. While this is fine for a lot
> > of use cases, it may not be suitable for certain other use
> > cases like manual memory management, etc.
> > 
> > This patchset adds another API to register memory segments
> > with DPDK (so that API's like ``rte_mem_virt2memseg`` could
> > be relied on by PMD's and such), but not create a malloc
> > heap out of them.
> > 
> > Aside from the obvious (not adding memory to a heap), the
> > other major difference between this API and the
> > ``rte_malloc_heap_*`` external memory functions is the fact
> > that no DMA mapping is performed automatically, as well as
> > no mem event callbacks are triggered.
> > 
> > This really draws a line in the sand, and there are now two
> > ways of doing things - do everything automatically (using
> > the ``rte_malloc_heap_*`` API's), or do everything manually
> > (``rte_extmem_*`` and future DMA mapping API [1] that would
> > replace ``rte_vfio_dma_map``). This way, the consistency of
> > API is kept, and flexibility is also allowed.
> > 
> > [1] https://mails.dpdk.org/archives/dev/2018-November/118175.html
> 
> Where are the examples for this? Give a sample application maybe.
> 
> Also there are no test cases.

It looks to be a big task, but yes, would be nice to have test
of external memory allocation in DPDK unit tests.

On 20-Dec-18 5:18 PM, Thomas Monjalon wrote:
> 20/12/2018 17:16, Stephen Hemminger:
>> On Thu, 20 Dec 2018 15:32:37 +0000
>> Anatoly Burakov <anatoly.burakov@intel.com> wrote:
>>
>>> Currently, the only way to use externally allocated memory
>>> is through rte_malloc API's. While this is fine for a lot
>>> of use cases, it may not be suitable for certain other use
>>> cases like manual memory management, etc.
>>>
>>> This patchset adds another API to register memory segments
>>> with DPDK (so that API's like ``rte_mem_virt2memseg`` could
>>> be relied on by PMD's and such), but not create a malloc
>>> heap out of them.
>>>
>>> Aside from the obvious (not adding memory to a heap), the
>>> other major difference between this API and the
>>> ``rte_malloc_heap_*`` external memory functions is the fact
>>> that no DMA mapping is performed automatically, as well as
>>> no mem event callbacks are triggered.
>>>
>>> This really draws a line in the sand, and there are now two
>>> ways of doing things - do everything automatically (using
>>> the ``rte_malloc_heap_*`` API's), or do everything manually
>>> (``rte_extmem_*`` and future DMA mapping API [1] that would
>>> replace ``rte_vfio_dma_map``). This way, the consistency of
>>> API is kept, and flexibility is also allowed.
>>>
>>> [1] https://mails.dpdk.org/archives/dev/2018-November/118175.html
>>
>> Where are the examples for this? Give a sample application maybe.
>>
>> Also there are no test cases.
> 
> It looks to be a big task, but yes, would be nice to have test
> of external memory allocation in DPDK unit tests.
> 

I imagine if i submitted patches for this, since it's test code, it can 
go into rc1? Or is that considered a "feature"?

I don't think it will be a lot of code, there are only 4 new API calls. 
Extending extmem autotest should do the trick. Adding a new testpmd mode 
is also possible but less trivial, and can be postponed to 19.05.