[v2,5/7] mem: modify error message for DMA mask check
Checks
Commit Message
If DMA mask checks shows mapped memory out of the supported range
specified by the DMA mask, nothing can be done but return an error
an report the error. This can imply the app not being executed at
all or precluding dynamic memory allocation once the app is running.
In any case, we can advice the user to force IOVA as PA if currently
IOVA being VA and user being root.
Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
---
lib/librte_eal/common/malloc_heap.c | 41 +++++++++++++++++++++++++----
1 file changed, 36 insertions(+), 5 deletions(-)
Comments
1. With GCC32, testpmd could not startup without '--iova-mode pa'.
./i686-native-linuxapp-gcc/app/testpmd -c f -n 4 -- -i
The output is:
EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Some devices want iova as va but pa will be used because.. EAL: few device bound to UIO
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: wrong dma mask size 48 (Max: 31)
EAL: alloc_pages_on_heap(): couldn't allocate memory due to IOVA exceeding limits of current DMA mask
error allocating rte services array
EAL: FATAL: rte_service_init() failed
EAL: rte_service_init() failed
PANIC in main():
Cannot init EAL
5: [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda) [0x56606fda]]
4: [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6) [0xf74d1276]]
3: [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21) [0x565fcee1]]
2: [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d) [0x565edc68]]
1: [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33) [0x5675f333]]
Aborted
2. With '--iova-mode pa', testpmd could startup.
3. With GCC64, there is no such issue.
Thanks!
-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alejandro Lucero
Sent: Friday, November 2, 2018 3:53 AM
To: dev@dpdk.org
Subject: [dpdk-dev] [PATCH v2 5/7] mem: modify error message for DMA mask check
If DMA mask checks shows mapped memory out of the supported range specified by the DMA mask, nothing can be done but return an error an report the error. This can imply the app not being executed at all or precluding dynamic memory allocation once the app is running.
In any case, we can advice the user to force IOVA as PA if currently IOVA being VA and user being root.
Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
Tested-by: Li, WenjieX A <wenjiex.a.li@intel.com>
---
lib/librte_eal/common/malloc_heap.c | 41 +++++++++++++++++++++++++----
1 file changed, 36 insertions(+), 5 deletions(-)
diff --git a/lib/librte_eal/common/malloc_heap.c b/lib/librte_eal/common/malloc_heap.c
index 4997c5ef5..c2c112aa6 100644
--- a/lib/librte_eal/common/malloc_heap.c
+++ b/lib/librte_eal/common/malloc_heap.c
@@ -321,13 +321,44 @@ alloc_pages_on_heap(struct malloc_heap *heap, uint64_t pg_sz, size_t elt_size,
goto fail;
}
- if (mcfg->dma_maskbits) {
- if (rte_mem_check_dma_mask(mcfg->dma_maskbits)) {
+ /*
+ * Once we have all the memseg lists configured, if there is a dma mask
+ * set, check iova addresses are not out of range. Otherwise the device
+ * setting the dma mask could have problems with the mapped memory.
+ *
+ * There are two situations when this can happen:
+ * 1) memory initialization
+ * 2) dynamic memory allocation
+ *
+ * For 1), an error when checking dma mask implies app can not be
+ * executed. For 2) implies the new memory can not be added.
+ */
+ if (mcfg->dma_maskbits &&
+ rte_mem_check_dma_mask(mcfg->dma_maskbits)) {
+ /*
+ * Currently this can only happen if IOMMU is enabled
+ * and the address width supported by the IOMMU hw is
+ * not enough for using the memory mapped IOVAs.
+ *
+ * If IOVA is VA, advice to try with '--iova-mode pa'
+ * which could solve some situations when IOVA VA is not
+ * really needed.
+ */
+ RTE_LOG(ERR, EAL,
+ "%s(): couldn't allocate memory due to IOVA exceeding limits of current DMA mask\n",
+ __func__);
+
+ /*
+ * If IOVA is VA and it is possible to run with IOVA PA,
+ * because user is root, give and advice for solving the
+ * problem.
+ */
+ if ((rte_eal_iova_mode() == RTE_IOVA_VA) &&
+ rte_eal_using_phys_addrs())
RTE_LOG(ERR, EAL,
- "%s(): couldn't allocate memory due to DMA mask\n",
+ "%s(): Please try initializing EAL with --iova-mode=pa
+parameter\n",
__func__);
- goto fail;
- }
+ goto fail;
}
/* add newly minted memsegs to malloc heap */
--
2.17.1
On Mon, Nov 5, 2018 at 10:01 AM Li, WenjieX A <wenjiex.a.li@intel.com>
wrote:
> 1. With GCC32, testpmd could not startup without '--iova-mode pa'.
> ./i686-native-linuxapp-gcc/app/testpmd -c f -n 4 -- -i
> The output is:
> EAL: Detected 16 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Some devices want iova as va but pa will be used because.. EAL: few
> device bound to UIO
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: wrong dma mask size 48 (Max: 31)
> EAL: alloc_pages_on_heap(): couldn't allocate memory due to IOVA exceeding
> limits of current DMA mask
> error allocating rte services array
> EAL: FATAL: rte_service_init() failed
> EAL: rte_service_init() failed
> PANIC in main():
> Cannot init EAL
> 5: [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda) [0x56606fda]]
> 4: [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6) [0xf74d1276]]
> 3: [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21) [0x565fcee1]]
> 2: [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d) [0x565edc68]]
> 1: [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33)
> [0x5675f333]]
> Aborted
>
> 2. With '--iova-mode pa', testpmd could startup.
> 3. With GCC64, there is no such issue.
> Thanks!
>
>
Does 32 bits support require IOMMU? It would be a surprise. If there is no
IOMMU hardware, no dma mask should be there at all.
>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alejandro Lucero
> Sent: Friday, November 2, 2018 3:53 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v2 5/7] mem: modify error message for DMA mask
> check
>
> If DMA mask checks shows mapped memory out of the supported range
> specified by the DMA mask, nothing can be done but return an error an
> report the error. This can imply the app not being executed at all or
> precluding dynamic memory allocation once the app is running.
> In any case, we can advice the user to force IOVA as PA if currently IOVA
> being VA and user being root.
>
> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
> Tested-by: Li, WenjieX A <wenjiex.a.li@intel.com>
> ---
> lib/librte_eal/common/malloc_heap.c | 41 +++++++++++++++++++++++++----
> 1 file changed, 36 insertions(+), 5 deletions(-)
>
> diff --git a/lib/librte_eal/common/malloc_heap.c
> b/lib/librte_eal/common/malloc_heap.c
> index 4997c5ef5..c2c112aa6 100644
> --- a/lib/librte_eal/common/malloc_heap.c
> +++ b/lib/librte_eal/common/malloc_heap.c
> @@ -321,13 +321,44 @@ alloc_pages_on_heap(struct malloc_heap *heap,
> uint64_t pg_sz, size_t elt_size,
> goto fail;
> }
>
> - if (mcfg->dma_maskbits) {
> - if (rte_mem_check_dma_mask(mcfg->dma_maskbits)) {
> + /*
> + * Once we have all the memseg lists configured, if there is a dma
> mask
> + * set, check iova addresses are not out of range. Otherwise the
> device
> + * setting the dma mask could have problems with the mapped memory.
> + *
> + * There are two situations when this can happen:
> + * 1) memory initialization
> + * 2) dynamic memory allocation
> + *
> + * For 1), an error when checking dma mask implies app can not be
> + * executed. For 2) implies the new memory can not be added.
> + */
> + if (mcfg->dma_maskbits &&
> + rte_mem_check_dma_mask(mcfg->dma_maskbits)) {
> + /*
> + * Currently this can only happen if IOMMU is enabled
> + * and the address width supported by the IOMMU hw is
> + * not enough for using the memory mapped IOVAs.
> + *
> + * If IOVA is VA, advice to try with '--iova-mode pa'
> + * which could solve some situations when IOVA VA is not
> + * really needed.
> + */
> + RTE_LOG(ERR, EAL,
> + "%s(): couldn't allocate memory due to IOVA
> exceeding limits of current DMA mask\n",
> + __func__);
> +
> + /*
> + * If IOVA is VA and it is possible to run with IOVA PA,
> + * because user is root, give and advice for solving the
> + * problem.
> + */
> + if ((rte_eal_iova_mode() == RTE_IOVA_VA) &&
> + rte_eal_using_phys_addrs())
> RTE_LOG(ERR, EAL,
> - "%s(): couldn't allocate memory due to DMA
> mask\n",
> + "%s(): Please try initializing EAL with
> --iova-mode=pa
> +parameter\n",
> __func__);
> - goto fail;
> - }
> + goto fail;
> }
>
> /* add newly minted memsegs to malloc heap */
> --
> 2.17.1
>
>
On 05-Nov-18 10:13 AM, Alejandro Lucero wrote:
> On Mon, Nov 5, 2018 at 10:01 AM Li, WenjieX A <wenjiex.a.li@intel.com>
> wrote:
>
>> 1. With GCC32, testpmd could not startup without '--iova-mode pa'.
>> ./i686-native-linuxapp-gcc/app/testpmd -c f -n 4 -- -i
>> The output is:
>> EAL: Detected 16 lcore(s)
>> EAL: Detected 1 NUMA nodes
>> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>> EAL: Some devices want iova as va but pa will be used because.. EAL: few
>> device bound to UIO
>> EAL: No free hugepages reported in hugepages-1048576kB
>> EAL: Probing VFIO support...
>> EAL: VFIO support initialized
>> EAL: wrong dma mask size 48 (Max: 31)
>> EAL: alloc_pages_on_heap(): couldn't allocate memory due to IOVA exceeding
>> limits of current DMA mask
>> error allocating rte services array
>> EAL: FATAL: rte_service_init() failed
>> EAL: rte_service_init() failed
>> PANIC in main():
>> Cannot init EAL
>> 5: [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda) [0x56606fda]]
>> 4: [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6) [0xf74d1276]]
>> 3: [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21) [0x565fcee1]]
>> 2: [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d) [0x565edc68]]
>> 1: [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33)
>> [0x5675f333]]
>> Aborted
>>
>> 2. With '--iova-mode pa', testpmd could startup.
>> 3. With GCC64, there is no such issue.
>> Thanks!
>>
>>
> Does 32 bits support require IOMMU? It would be a surprise. If there is no
> IOMMU hardware, no dma mask should be there at all.
IOMMU is supported on 32-bits, however limited the address space might
be. Maybe limit IOMMU width to RTE_MIN(31, value) bits for everything on
32-bit?
On Mon, Nov 5, 2018 at 3:12 PM Burakov, Anatoly <anatoly.burakov@intel.com>
wrote:
> On 05-Nov-18 10:13 AM, Alejandro Lucero wrote:
> > On Mon, Nov 5, 2018 at 10:01 AM Li, WenjieX A <wenjiex.a.li@intel.com>
> > wrote:
> >
> >> 1. With GCC32, testpmd could not startup without '--iova-mode pa'.
> >> ./i686-native-linuxapp-gcc/app/testpmd -c f -n 4 -- -i
> >> The output is:
> >> EAL: Detected 16 lcore(s)
> >> EAL: Detected 1 NUMA nodes
> >> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> >> EAL: Some devices want iova as va but pa will be used because.. EAL: few
> >> device bound to UIO
> >> EAL: No free hugepages reported in hugepages-1048576kB
> >> EAL: Probing VFIO support...
> >> EAL: VFIO support initialized
> >> EAL: wrong dma mask size 48 (Max: 31)
> >> EAL: alloc_pages_on_heap(): couldn't allocate memory due to IOVA
> exceeding
> >> limits of current DMA mask
> >> error allocating rte services array
> >> EAL: FATAL: rte_service_init() failed
> >> EAL: rte_service_init() failed
> >> PANIC in main():
> >> Cannot init EAL
> >> 5: [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda) [0x56606fda]]
> >> 4: [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6) [0xf74d1276]]
> >> 3: [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21) [0x565fcee1]]
> >> 2: [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d)
> [0x565edc68]]
> >> 1: [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33)
> >> [0x5675f333]]
> >> Aborted
> >>
> >> 2. With '--iova-mode pa', testpmd could startup.
> >> 3. With GCC64, there is no such issue.
> >> Thanks!
> >>
> >>
> > Does 32 bits support require IOMMU? It would be a surprise. If there is
> no
> > IOMMU hardware, no dma mask should be there at all.
>
> IOMMU is supported on 32-bits, however limited the address space might
> be. Maybe limit IOMMU width to RTE_MIN(31, value) bits for everything on
> 32-bit?
>
>
If IOMMU is supported in 32 bits, then the DMA mask check should not be
happening. AFAIK, the IOMMU hardware addressing limitations is a problem
only in 64 bits systems. The worst situation I have head of is 39 bits for
virtualized IOMMU with QEMU.
I would prefer not to invoke rte_mem_set_dma_mask for 32 bits system for
the Intel IOMMU case. The only other dma mask client is the NFP PMD and we
do not support 32 bits systems.
> --
> Thanks,
> Anatoly
>
On 05-Nov-18 3:33 PM, Alejandro Lucero wrote:
>
>
> On Mon, Nov 5, 2018 at 3:12 PM Burakov, Anatoly
> <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
>
> On 05-Nov-18 10:13 AM, Alejandro Lucero wrote:
> > On Mon, Nov 5, 2018 at 10:01 AM Li, WenjieX A
> <wenjiex.a.li@intel.com <mailto:wenjiex.a.li@intel.com>>
> > wrote:
> >
> >> 1. With GCC32, testpmd could not startup without '--iova-mode pa'.
> >> ./i686-native-linuxapp-gcc/app/testpmd -c f -n 4 -- -i
> >> The output is:
> >> EAL: Detected 16 lcore(s)
> >> EAL: Detected 1 NUMA nodes
> >> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> >> EAL: Some devices want iova as va but pa will be used because..
> EAL: few
> >> device bound to UIO
> >> EAL: No free hugepages reported in hugepages-1048576kB
> >> EAL: Probing VFIO support...
> >> EAL: VFIO support initialized
> >> EAL: wrong dma mask size 48 (Max: 31)
> >> EAL: alloc_pages_on_heap(): couldn't allocate memory due to IOVA
> exceeding
> >> limits of current DMA mask
> >> error allocating rte services array
> >> EAL: FATAL: rte_service_init() failed
> >> EAL: rte_service_init() failed
> >> PANIC in main():
> >> Cannot init EAL
> >> 5: [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda) [0x56606fda]]
> >> 4: [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6)
> [0xf74d1276]]
> >> 3: [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21) [0x565fcee1]]
> >> 2: [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d)
> [0x565edc68]]
> >> 1: [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33)
> >> [0x5675f333]]
> >> Aborted
> >>
> >> 2. With '--iova-mode pa', testpmd could startup.
> >> 3. With GCC64, there is no such issue.
> >> Thanks!
> >>
> >>
> > Does 32 bits support require IOMMU? It would be a surprise. If
> there is no
> > IOMMU hardware, no dma mask should be there at all.
>
> IOMMU is supported on 32-bits, however limited the address space might
> be. Maybe limit IOMMU width to RTE_MIN(31, value) bits for
> everything on
> 32-bit?
>
>
> If IOMMU is supported in 32 bits, then the DMA mask check should not be
> happening. AFAIK, the IOMMU hardware addressing limitations is a problem
> only in 64 bits systems. The worst situation I have head of is 39 bits
> for virtualized IOMMU with QEMU.
>
> I would prefer not to invoke rte_mem_set_dma_mask for 32 bits system for
> the Intel IOMMU case. The only other dma mask client is the NFP PMD and
> we do not support 32 bits systems.
>
I don't think not invoking DMA mask check is the right choice here. In
practice it may be, but i'd rather the behavior to be "correct", if at
all possible :) It is theoretically possible to have an IOMMU with an
addressing limitation of, say, 30 bits (even though they don't exist in
reality), so therefore our code should handle it, should it encounter
one, and it should also handle the "proper" ones correctly (as in, treat
them as 32-bit-limited instead of 39- or 48-bit-limited).
>
> --
> Thanks,
> Anatoly
>
On Mon, Nov 5, 2018 at 4:35 PM Burakov, Anatoly <anatoly.burakov@intel.com>
wrote:
> On 05-Nov-18 3:33 PM, Alejandro Lucero wrote:
> >
> >
> > On Mon, Nov 5, 2018 at 3:12 PM Burakov, Anatoly
> > <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
> >
> > On 05-Nov-18 10:13 AM, Alejandro Lucero wrote:
> > > On Mon, Nov 5, 2018 at 10:01 AM Li, WenjieX A
> > <wenjiex.a.li@intel.com <mailto:wenjiex.a.li@intel.com>>
> > > wrote:
> > >
> > >> 1. With GCC32, testpmd could not startup without '--iova-mode
> pa'.
> > >> ./i686-native-linuxapp-gcc/app/testpmd -c f -n 4 -- -i
> > >> The output is:
> > >> EAL: Detected 16 lcore(s)
> > >> EAL: Detected 1 NUMA nodes
> > >> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > >> EAL: Some devices want iova as va but pa will be used because..
> > EAL: few
> > >> device bound to UIO
> > >> EAL: No free hugepages reported in hugepages-1048576kB
> > >> EAL: Probing VFIO support...
> > >> EAL: VFIO support initialized
> > >> EAL: wrong dma mask size 48 (Max: 31)
> > >> EAL: alloc_pages_on_heap(): couldn't allocate memory due to IOVA
> > exceeding
> > >> limits of current DMA mask
> > >> error allocating rte services array
> > >> EAL: FATAL: rte_service_init() failed
> > >> EAL: rte_service_init() failed
> > >> PANIC in main():
> > >> Cannot init EAL
> > >> 5: [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda)
> [0x56606fda]]
> > >> 4: [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6)
> > [0xf74d1276]]
> > >> 3: [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21)
> [0x565fcee1]]
> > >> 2: [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d)
> > [0x565edc68]]
> > >> 1: [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33)
> > >> [0x5675f333]]
> > >> Aborted
> > >>
> > >> 2. With '--iova-mode pa', testpmd could startup.
> > >> 3. With GCC64, there is no such issue.
> > >> Thanks!
> > >>
> > >>
> > > Does 32 bits support require IOMMU? It would be a surprise. If
> > there is no
> > > IOMMU hardware, no dma mask should be there at all.
> >
> > IOMMU is supported on 32-bits, however limited the address space
> might
> > be. Maybe limit IOMMU width to RTE_MIN(31, value) bits for
> > everything on
> > 32-bit?
> >
> >
> > If IOMMU is supported in 32 bits, then the DMA mask check should not be
> > happening. AFAIK, the IOMMU hardware addressing limitations is a problem
> > only in 64 bits systems. The worst situation I have head of is 39 bits
> > for virtualized IOMMU with QEMU.
> >
> > I would prefer not to invoke rte_mem_set_dma_mask for 32 bits system for
> > the Intel IOMMU case. The only other dma mask client is the NFP PMD and
> > we do not support 32 bits systems.
> >
>
> I don't think not invoking DMA mask check is the right choice here. In
> practice it may be, but i'd rather the behavior to be "correct", if at
> all possible :) It is theoretically possible to have an IOMMU with an
> addressing limitation of, say, 30 bits (even though they don't exist in
> reality), so therefore our code should handle it, should it encounter
> one, and it should also handle the "proper" ones correctly (as in, treat
> them as 32-bit-limited instead of 39- or 48-bit-limited).
>
>
Fine.
The problem is the current sanity check about the dma mask width, what is
31 for 32 bits systems.
Should we just leave a single max dma width to 63? This covers the
possibility of 32 bit systems integrating an IOMMU designed for 64 bits. I
really doubt this is a real possibility in x86, although I can see it more
likely in embedded systems where this sort of hardware components
integration happens.
>
> > --
> > Thanks,
> > Anatoly
> >
>
>
> --
> Thanks,
> Anatoly
>
On 06-Nov-18 9:32 AM, Alejandro Lucero wrote:
>
>
> On Mon, Nov 5, 2018 at 4:35 PM Burakov, Anatoly
> <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
>
> On 05-Nov-18 3:33 PM, Alejandro Lucero wrote:
> >
> >
> > On Mon, Nov 5, 2018 at 3:12 PM Burakov, Anatoly
> > <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>
> <mailto:anatoly.burakov@intel.com
> <mailto:anatoly.burakov@intel.com>>> wrote:
> >
> > On 05-Nov-18 10:13 AM, Alejandro Lucero wrote:
> > > On Mon, Nov 5, 2018 at 10:01 AM Li, WenjieX A
> > <wenjiex.a.li@intel.com <mailto:wenjiex.a.li@intel.com>
> <mailto:wenjiex.a.li@intel.com <mailto:wenjiex.a.li@intel.com>>>
> > > wrote:
> > >
> > >> 1. With GCC32, testpmd could not startup without
> '--iova-mode pa'.
> > >> ./i686-native-linuxapp-gcc/app/testpmd -c f -n 4 -- -i
> > >> The output is:
> > >> EAL: Detected 16 lcore(s)
> > >> EAL: Detected 1 NUMA nodes
> > >> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > >> EAL: Some devices want iova as va but pa will be used
> because..
> > EAL: few
> > >> device bound to UIO
> > >> EAL: No free hugepages reported in hugepages-1048576kB
> > >> EAL: Probing VFIO support...
> > >> EAL: VFIO support initialized
> > >> EAL: wrong dma mask size 48 (Max: 31)
> > >> EAL: alloc_pages_on_heap(): couldn't allocate memory due
> to IOVA
> > exceeding
> > >> limits of current DMA mask
> > >> error allocating rte services array
> > >> EAL: FATAL: rte_service_init() failed
> > >> EAL: rte_service_init() failed
> > >> PANIC in main():
> > >> Cannot init EAL
> > >> 5: [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda)
> [0x56606fda]]
> > >> 4: [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6)
> > [0xf74d1276]]
> > >> 3: [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21)
> [0x565fcee1]]
> > >> 2: [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d)
> > [0x565edc68]]
> > >> 1:
> [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33)
> > >> [0x5675f333]]
> > >> Aborted
> > >>
> > >> 2. With '--iova-mode pa', testpmd could startup.
> > >> 3. With GCC64, there is no such issue.
> > >> Thanks!
> > >>
> > >>
> > > Does 32 bits support require IOMMU? It would be a surprise. If
> > there is no
> > > IOMMU hardware, no dma mask should be there at all.
> >
> > IOMMU is supported on 32-bits, however limited the address
> space might
> > be. Maybe limit IOMMU width to RTE_MIN(31, value) bits for
> > everything on
> > 32-bit?
> >
> >
> > If IOMMU is supported in 32 bits, then the DMA mask check should
> not be
> > happening. AFAIK, the IOMMU hardware addressing limitations is a
> problem
> > only in 64 bits systems. The worst situation I have head of is 39
> bits
> > for virtualized IOMMU with QEMU.
> >
> > I would prefer not to invoke rte_mem_set_dma_mask for 32 bits
> system for
> > the Intel IOMMU case. The only other dma mask client is the NFP
> PMD and
> > we do not support 32 bits systems.
> >
>
> I don't think not invoking DMA mask check is the right choice here. In
> practice it may be, but i'd rather the behavior to be "correct", if at
> all possible :) It is theoretically possible to have an IOMMU with an
> addressing limitation of, say, 30 bits (even though they don't exist in
> reality), so therefore our code should handle it, should it encounter
> one, and it should also handle the "proper" ones correctly (as in,
> treat
> them as 32-bit-limited instead of 39- or 48-bit-limited).
>
>
> Fine.
>
> The problem is the current sanity check about the dma mask width, what
> is 31 for 32 bits systems.
> Should we just leave a single max dma width to 63? This covers the
> possibility of 32 bit systems integrating an IOMMU designed for 64
> bits. I really doubt this is a real possibility in x86, although I can
> see it more likely in embedded systems where this sort of hardware
> components integration happens.
Actually (and after a quick chat with Ferruh), is this even needed? IOVA
addresses are independent from VA width, IOVA can happily be bigger than
32-bits if i understand things correctly. All of our IOVA addresses are
always 64-bit throughout DPDK. I don't think this check is even valid.
>
> >
> > --
> > Thanks,
> > Anatoly
> >
>
>
> --
> Thanks,
> Anatoly
>
On Tue, Nov 6, 2018 at 10:31 AM Burakov, Anatoly <anatoly.burakov@intel.com>
wrote:
> On 06-Nov-18 9:32 AM, Alejandro Lucero wrote:
> >
> >
> > On Mon, Nov 5, 2018 at 4:35 PM Burakov, Anatoly
> > <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
> >
> > On 05-Nov-18 3:33 PM, Alejandro Lucero wrote:
> > >
> > >
> > > On Mon, Nov 5, 2018 at 3:12 PM Burakov, Anatoly
> > > <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>
> > <mailto:anatoly.burakov@intel.com
> > <mailto:anatoly.burakov@intel.com>>> wrote:
> > >
> > > On 05-Nov-18 10:13 AM, Alejandro Lucero wrote:
> > > > On Mon, Nov 5, 2018 at 10:01 AM Li, WenjieX A
> > > <wenjiex.a.li@intel.com <mailto:wenjiex.a.li@intel.com>
> > <mailto:wenjiex.a.li@intel.com <mailto:wenjiex.a.li@intel.com>>>
> > > > wrote:
> > > >
> > > >> 1. With GCC32, testpmd could not startup without
> > '--iova-mode pa'.
> > > >> ./i686-native-linuxapp-gcc/app/testpmd -c f -n 4 -- -i
> > > >> The output is:
> > > >> EAL: Detected 16 lcore(s)
> > > >> EAL: Detected 1 NUMA nodes
> > > >> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > > >> EAL: Some devices want iova as va but pa will be used
> > because..
> > > EAL: few
> > > >> device bound to UIO
> > > >> EAL: No free hugepages reported in hugepages-1048576kB
> > > >> EAL: Probing VFIO support...
> > > >> EAL: VFIO support initialized
> > > >> EAL: wrong dma mask size 48 (Max: 31)
> > > >> EAL: alloc_pages_on_heap(): couldn't allocate memory due
> > to IOVA
> > > exceeding
> > > >> limits of current DMA mask
> > > >> error allocating rte services array
> > > >> EAL: FATAL: rte_service_init() failed
> > > >> EAL: rte_service_init() failed
> > > >> PANIC in main():
> > > >> Cannot init EAL
> > > >> 5: [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda)
> > [0x56606fda]]
> > > >> 4: [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6)
> > > [0xf74d1276]]
> > > >> 3: [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21)
> > [0x565fcee1]]
> > > >> 2:
> [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d)
> > > [0x565edc68]]
> > > >> 1:
> > [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33)
> > > >> [0x5675f333]]
> > > >> Aborted
> > > >>
> > > >> 2. With '--iova-mode pa', testpmd could startup.
> > > >> 3. With GCC64, there is no such issue.
> > > >> Thanks!
> > > >>
> > > >>
> > > > Does 32 bits support require IOMMU? It would be a
> surprise. If
> > > there is no
> > > > IOMMU hardware, no dma mask should be there at all.
> > >
> > > IOMMU is supported on 32-bits, however limited the address
> > space might
> > > be. Maybe limit IOMMU width to RTE_MIN(31, value) bits for
> > > everything on
> > > 32-bit?
> > >
> > >
> > > If IOMMU is supported in 32 bits, then the DMA mask check should
> > not be
> > > happening. AFAIK, the IOMMU hardware addressing limitations is a
> > problem
> > > only in 64 bits systems. The worst situation I have head of is 39
> > bits
> > > for virtualized IOMMU with QEMU.
> > >
> > > I would prefer not to invoke rte_mem_set_dma_mask for 32 bits
> > system for
> > > the Intel IOMMU case. The only other dma mask client is the NFP
> > PMD and
> > > we do not support 32 bits systems.
> > >
> >
> > I don't think not invoking DMA mask check is the right choice here.
> In
> > practice it may be, but i'd rather the behavior to be "correct", if
> at
> > all possible :) It is theoretically possible to have an IOMMU with an
> > addressing limitation of, say, 30 bits (even though they don't exist
> in
> > reality), so therefore our code should handle it, should it encounter
> > one, and it should also handle the "proper" ones correctly (as in,
> > treat
> > them as 32-bit-limited instead of 39- or 48-bit-limited).
> >
> >
> > Fine.
> >
> > The problem is the current sanity check about the dma mask width, what
> > is 31 for 32 bits systems.
> > Should we just leave a single max dma width to 63? This covers the
> > possibility of 32 bit systems integrating an IOMMU designed for 64
> > bits. I really doubt this is a real possibility in x86, although I can
> > see it more likely in embedded systems where this sort of hardware
> > components integration happens.
>
> Actually (and after a quick chat with Ferruh), is this even needed? IOVA
> addresses are independent from VA width, IOVA can happily be bigger than
> 32-bits if i understand things correctly. All of our IOVA addresses are
> always 64-bit throughout DPDK. I don't think this check is even valid.
>
>
Although iova_t is 64 bits, there should not be a IOVA higher than 32 bits,
although there could be exceptions like PAE extensions (I'm old enough for
remembering that option :-( ).
Anyway, the original idea of dma mask sanity check is 32 bits systems was
assuming there should not be a dma mask above 32 bits, but I'm happy with
removing that sanity check for 32 bits systems. So, do you agree to just
leave the sanity check for a max width of 63 bits?
> >
> > >
> > > --
> > > Thanks,
> > > Anatoly
> > >
> >
> >
> > --
> > Thanks,
> > Anatoly
> >
>
>
> --
> Thanks,
> Anatoly
>
On 06-Nov-18 10:37 AM, Alejandro Lucero wrote:
>
>
> On Tue, Nov 6, 2018 at 10:31 AM Burakov, Anatoly
> <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
>
> On 06-Nov-18 9:32 AM, Alejandro Lucero wrote:
> >
> >
> > On Mon, Nov 5, 2018 at 4:35 PM Burakov, Anatoly
> > <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>
> <mailto:anatoly.burakov@intel.com
> <mailto:anatoly.burakov@intel.com>>> wrote:
> >
> > On 05-Nov-18 3:33 PM, Alejandro Lucero wrote:
> > >
> > >
> > > On Mon, Nov 5, 2018 at 3:12 PM Burakov, Anatoly
> > > <anatoly.burakov@intel.com
> <mailto:anatoly.burakov@intel.com> <mailto:anatoly.burakov@intel.com
> <mailto:anatoly.burakov@intel.com>>
> > <mailto:anatoly.burakov@intel.com
> <mailto:anatoly.burakov@intel.com>
> > <mailto:anatoly.burakov@intel.com
> <mailto:anatoly.burakov@intel.com>>>> wrote:
> > >
> > > On 05-Nov-18 10:13 AM, Alejandro Lucero wrote:
> > > > On Mon, Nov 5, 2018 at 10:01 AM Li, WenjieX A
> > > <wenjiex.a.li@intel.com
> <mailto:wenjiex.a.li@intel.com> <mailto:wenjiex.a.li@intel.com
> <mailto:wenjiex.a.li@intel.com>>
> > <mailto:wenjiex.a.li@intel.com
> <mailto:wenjiex.a.li@intel.com> <mailto:wenjiex.a.li@intel.com
> <mailto:wenjiex.a.li@intel.com>>>>
> > > > wrote:
> > > >
> > > >> 1. With GCC32, testpmd could not startup without
> > '--iova-mode pa'.
> > > >> ./i686-native-linuxapp-gcc/app/testpmd -c f -n 4 -- -i
> > > >> The output is:
> > > >> EAL: Detected 16 lcore(s)
> > > >> EAL: Detected 1 NUMA nodes
> > > >> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > > >> EAL: Some devices want iova as va but pa will be used
> > because..
> > > EAL: few
> > > >> device bound to UIO
> > > >> EAL: No free hugepages reported in hugepages-1048576kB
> > > >> EAL: Probing VFIO support...
> > > >> EAL: VFIO support initialized
> > > >> EAL: wrong dma mask size 48 (Max: 31)
> > > >> EAL: alloc_pages_on_heap(): couldn't allocate
> memory due
> > to IOVA
> > > exceeding
> > > >> limits of current DMA mask
> > > >> error allocating rte services array
> > > >> EAL: FATAL: rte_service_init() failed
> > > >> EAL: rte_service_init() failed
> > > >> PANIC in main():
> > > >> Cannot init EAL
> > > >> 5: [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda)
> > [0x56606fda]]
> > > >> 4:
> [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6)
> > > [0xf74d1276]]
> > > >> 3: [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21)
> > [0x565fcee1]]
> > > >> 2:
> [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d)
> > > [0x565edc68]]
> > > >> 1:
> > [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33)
> > > >> [0x5675f333]]
> > > >> Aborted
> > > >>
> > > >> 2. With '--iova-mode pa', testpmd could startup.
> > > >> 3. With GCC64, there is no such issue.
> > > >> Thanks!
> > > >>
> > > >>
> > > > Does 32 bits support require IOMMU? It would be a
> surprise. If
> > > there is no
> > > > IOMMU hardware, no dma mask should be there at all.
> > >
> > > IOMMU is supported on 32-bits, however limited the address
> > space might
> > > be. Maybe limit IOMMU width to RTE_MIN(31, value) bits for
> > > everything on
> > > 32-bit?
> > >
> > >
> > > If IOMMU is supported in 32 bits, then the DMA mask check
> should
> > not be
> > > happening. AFAIK, the IOMMU hardware addressing
> limitations is a
> > problem
> > > only in 64 bits systems. The worst situation I have head
> of is 39
> > bits
> > > for virtualized IOMMU with QEMU.
> > >
> > > I would prefer not to invoke rte_mem_set_dma_mask for 32 bits
> > system for
> > > the Intel IOMMU case. The only other dma mask client is
> the NFP
> > PMD and
> > > we do not support 32 bits systems.
> > >
> >
> > I don't think not invoking DMA mask check is the right choice
> here. In
> > practice it may be, but i'd rather the behavior to be
> "correct", if at
> > all possible :) It is theoretically possible to have an IOMMU
> with an
> > addressing limitation of, say, 30 bits (even though they
> don't exist in
> > reality), so therefore our code should handle it, should it
> encounter
> > one, and it should also handle the "proper" ones correctly
> (as in,
> > treat
> > them as 32-bit-limited instead of 39- or 48-bit-limited).
> >
> >
> > Fine.
> >
> > The problem is the current sanity check about the dma mask width,
> what
> > is 31 for 32 bits systems.
> > Should we just leave a single max dma width to 63? This covers the
> > possibility of 32 bit systems integrating an IOMMU designed for 64
> > bits. I really doubt this is a real possibility in x86, although
> I can
> > see it more likely in embedded systems where this sort of hardware
> > components integration happens.
>
> Actually (and after a quick chat with Ferruh), is this even needed?
> IOVA
> addresses are independent from VA width, IOVA can happily be bigger
> than
> 32-bits if i understand things correctly. All of our IOVA addresses are
> always 64-bit throughout DPDK. I don't think this check is even valid.
>
>
> Although iova_t is 64 bits, there should not be a IOVA higher than 32
> bits, although there could be exceptions like PAE extensions (I'm old
> enough for remembering that option :-( ).
>
> Anyway, the original idea of dma mask sanity check is 32 bits systems
> was assuming there should not be a dma mask above 32 bits, but I'm happy
> with removing that sanity check for 32 bits systems. So, do you agree to
> just leave the sanity check for a max width of 63 bits?
>
So, the issue with 32-bit here is that for this check to make sense, the
*kernel* must be 32-bit - not just userspace. IOW, this check should
*not* be present in a 32-bit application running on a 64-bit kernel.
So IMO, unless you know of a way to easily check if 1) kernel is 32-bit,
and 2) PAE is enabled/disabled (and by easily i mean using something
other than reading sysfs etc.), i don't think this check should be in
there :)
> >
> > >
> > > --
> > > Thanks,
> > > Anatoly
> > >
> >
> >
> > --
> > Thanks,
> > Anatoly
> >
>
>
> --
> Thanks,
> Anatoly
>
On Tue, Nov 6, 2018 at 10:48 AM Burakov, Anatoly <anatoly.burakov@intel.com>
wrote:
> On 06-Nov-18 10:37 AM, Alejandro Lucero wrote:
> >
> >
> > On Tue, Nov 6, 2018 at 10:31 AM Burakov, Anatoly
> > <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
> >
> > On 06-Nov-18 9:32 AM, Alejandro Lucero wrote:
> > >
> > >
> > > On Mon, Nov 5, 2018 at 4:35 PM Burakov, Anatoly
> > > <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>
> > <mailto:anatoly.burakov@intel.com
> > <mailto:anatoly.burakov@intel.com>>> wrote:
> > >
> > > On 05-Nov-18 3:33 PM, Alejandro Lucero wrote:
> > > >
> > > >
> > > > On Mon, Nov 5, 2018 at 3:12 PM Burakov, Anatoly
> > > > <anatoly.burakov@intel.com
> > <mailto:anatoly.burakov@intel.com> <mailto:anatoly.burakov@intel.com
> > <mailto:anatoly.burakov@intel.com>>
> > > <mailto:anatoly.burakov@intel.com
> > <mailto:anatoly.burakov@intel.com>
> > > <mailto:anatoly.burakov@intel.com
> > <mailto:anatoly.burakov@intel.com>>>> wrote:
> > > >
> > > > On 05-Nov-18 10:13 AM, Alejandro Lucero wrote:
> > > > > On Mon, Nov 5, 2018 at 10:01 AM Li, WenjieX A
> > > > <wenjiex.a.li@intel.com
> > <mailto:wenjiex.a.li@intel.com> <mailto:wenjiex.a.li@intel.com
> > <mailto:wenjiex.a.li@intel.com>>
> > > <mailto:wenjiex.a.li@intel.com
> > <mailto:wenjiex.a.li@intel.com> <mailto:wenjiex.a.li@intel.com
> > <mailto:wenjiex.a.li@intel.com>>>>
> > > > > wrote:
> > > > >
> > > > >> 1. With GCC32, testpmd could not startup without
> > > '--iova-mode pa'.
> > > > >> ./i686-native-linuxapp-gcc/app/testpmd -c f -n 4
> -- -i
> > > > >> The output is:
> > > > >> EAL: Detected 16 lcore(s)
> > > > >> EAL: Detected 1 NUMA nodes
> > > > >> EAL: Multi-process socket
> /var/run/dpdk/rte/mp_socket
> > > > >> EAL: Some devices want iova as va but pa will be
> used
> > > because..
> > > > EAL: few
> > > > >> device bound to UIO
> > > > >> EAL: No free hugepages reported in
> hugepages-1048576kB
> > > > >> EAL: Probing VFIO support...
> > > > >> EAL: VFIO support initialized
> > > > >> EAL: wrong dma mask size 48 (Max: 31)
> > > > >> EAL: alloc_pages_on_heap(): couldn't allocate
> > memory due
> > > to IOVA
> > > > exceeding
> > > > >> limits of current DMA mask
> > > > >> error allocating rte services array
> > > > >> EAL: FATAL: rte_service_init() failed
> > > > >> EAL: rte_service_init() failed
> > > > >> PANIC in main():
> > > > >> Cannot init EAL
> > > > >> 5:
> [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda)
> > > [0x56606fda]]
> > > > >> 4:
> > [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6)
> > > > [0xf74d1276]]
> > > > >> 3:
> [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21)
> > > [0x565fcee1]]
> > > > >> 2:
> > [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d)
> > > > [0x565edc68]]
> > > > >> 1:
> > > [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33)
> > > > >> [0x5675f333]]
> > > > >> Aborted
> > > > >>
> > > > >> 2. With '--iova-mode pa', testpmd could startup.
> > > > >> 3. With GCC64, there is no such issue.
> > > > >> Thanks!
> > > > >>
> > > > >>
> > > > > Does 32 bits support require IOMMU? It would be a
> > surprise. If
> > > > there is no
> > > > > IOMMU hardware, no dma mask should be there at all.
> > > >
> > > > IOMMU is supported on 32-bits, however limited the
> address
> > > space might
> > > > be. Maybe limit IOMMU width to RTE_MIN(31, value) bits
> for
> > > > everything on
> > > > 32-bit?
> > > >
> > > >
> > > > If IOMMU is supported in 32 bits, then the DMA mask check
> > should
> > > not be
> > > > happening. AFAIK, the IOMMU hardware addressing
> > limitations is a
> > > problem
> > > > only in 64 bits systems. The worst situation I have head
> > of is 39
> > > bits
> > > > for virtualized IOMMU with QEMU.
> > > >
> > > > I would prefer not to invoke rte_mem_set_dma_mask for 32
> bits
> > > system for
> > > > the Intel IOMMU case. The only other dma mask client is
> > the NFP
> > > PMD and
> > > > we do not support 32 bits systems.
> > > >
> > >
> > > I don't think not invoking DMA mask check is the right choice
> > here. In
> > > practice it may be, but i'd rather the behavior to be
> > "correct", if at
> > > all possible :) It is theoretically possible to have an IOMMU
> > with an
> > > addressing limitation of, say, 30 bits (even though they
> > don't exist in
> > > reality), so therefore our code should handle it, should it
> > encounter
> > > one, and it should also handle the "proper" ones correctly
> > (as in,
> > > treat
> > > them as 32-bit-limited instead of 39- or 48-bit-limited).
> > >
> > >
> > > Fine.
> > >
> > > The problem is the current sanity check about the dma mask width,
> > what
> > > is 31 for 32 bits systems.
> > > Should we just leave a single max dma width to 63? This covers the
> > > possibility of 32 bit systems integrating an IOMMU designed for
> 64
> > > bits. I really doubt this is a real possibility in x86, although
> > I can
> > > see it more likely in embedded systems where this sort of hardware
> > > components integration happens.
> >
> > Actually (and after a quick chat with Ferruh), is this even needed?
> > IOVA
> > addresses are independent from VA width, IOVA can happily be bigger
> > than
> > 32-bits if i understand things correctly. All of our IOVA addresses
> are
> > always 64-bit throughout DPDK. I don't think this check is even
> valid.
> >
> >
> > Although iova_t is 64 bits, there should not be a IOVA higher than 32
> > bits, although there could be exceptions like PAE extensions (I'm old
> > enough for remembering that option :-( ).
> >
> > Anyway, the original idea of dma mask sanity check is 32 bits systems
> > was assuming there should not be a dma mask above 32 bits, but I'm happy
> > with removing that sanity check for 32 bits systems. So, do you agree to
> > just leave the sanity check for a max width of 63 bits?
> >
>
> So, the issue with 32-bit here is that for this check to make sense, the
> *kernel* must be 32-bit - not just userspace. IOW, this check should
> *not* be present in a 32-bit application running on a 64-bit kernel.
>
> So IMO, unless you know of a way to easily check if 1) kernel is 32-bit,
> and 2) PAE is enabled/disabled (and by easily i mean using something
> other than reading sysfs etc.), i don't think this check should be in
> there :)
>
Ok then. If there are no other opinions, I will remove the sanity check for
32 bits systems.
Thanks
>
> > >
> > > >
> > > > --
> > > > Thanks,
> > > > Anatoly
> > > >
> > >
> > >
> > > --
> > > Thanks,
> > > Anatoly
> > >
> >
> >
> > --
> > Thanks,
> > Anatoly
> >
>
>
> --
> Thanks,
> Anatoly
>
@@ -321,13 +321,44 @@ alloc_pages_on_heap(struct malloc_heap *heap, uint64_t pg_sz, size_t elt_size,
goto fail;
}
- if (mcfg->dma_maskbits) {
- if (rte_mem_check_dma_mask(mcfg->dma_maskbits)) {
+ /*
+ * Once we have all the memseg lists configured, if there is a dma mask
+ * set, check iova addresses are not out of range. Otherwise the device
+ * setting the dma mask could have problems with the mapped memory.
+ *
+ * There are two situations when this can happen:
+ * 1) memory initialization
+ * 2) dynamic memory allocation
+ *
+ * For 1), an error when checking dma mask implies app can not be
+ * executed. For 2) implies the new memory can not be added.
+ */
+ if (mcfg->dma_maskbits &&
+ rte_mem_check_dma_mask(mcfg->dma_maskbits)) {
+ /*
+ * Currently this can only happen if IOMMU is enabled
+ * and the address width supported by the IOMMU hw is
+ * not enough for using the memory mapped IOVAs.
+ *
+ * If IOVA is VA, advice to try with '--iova-mode pa'
+ * which could solve some situations when IOVA VA is not
+ * really needed.
+ */
+ RTE_LOG(ERR, EAL,
+ "%s(): couldn't allocate memory due to IOVA exceeding limits of current DMA mask\n",
+ __func__);
+
+ /*
+ * If IOVA is VA and it is possible to run with IOVA PA,
+ * because user is root, give and advice for solving the
+ * problem.
+ */
+ if ((rte_eal_iova_mode() == RTE_IOVA_VA) &&
+ rte_eal_using_phys_addrs())
RTE_LOG(ERR, EAL,
- "%s(): couldn't allocate memory due to DMA mask\n",
+ "%s(): Please try initializing EAL with --iova-mode=pa parameter\n",
__func__);
- goto fail;
- }
+ goto fail;
}
/* add newly minted memsegs to malloc heap */