mbox series

[v5,0/1] vfio: modify spapr iommu support to use static window sizing

Message ID 20201103220532.176225-1-drc@linux.vnet.ibm.com (mailing list archive)
Headers
Series vfio: modify spapr iommu support to use static window sizing |

Message

David Christensen Nov. 3, 2020, 10:05 p.m. UTC
  The SPAPR v2 IOMMU used on bare-metal PowerNV systems requires that a DMA
window be defined before mapping/unmapping memory.  The current VFIO code
dynamically resizes this DMA window every time a new memory request is
made, which requires that all existing memory be unmapped/remapped.
While this strategy worked in DPDK 17.11 and earlier where memory was
statically allocated during startup, it is potentially dangerous in DPDK
18.11 and later where memory can be allocated during runtime, temporarily
invalidating IOVA memory used by hardware.

This new code statically sizes the DMA window at startup, based on the
amount of memory installed in the system, avoiding the need to unmap
memory during runtime.
---
v5:
- Modify get_highest_mem_addr to return error, not address
- Add comment regarding sPAPR v1/v2 default window and why it
  needs to be removed
- Added indent to second line of vfio_spapr_dma_mem_map() definition

v4:
- Move file reading code out of vfio_spapr_window_size_walk()

v3:
- Rebase for 20.08

v2:
- Drop patch to wrap ppc64 code with ifdef's
- Add warning when external memory detected
- Change VA memory size detection to scan memseg list when setting DMA window
  for IOVA=VA
- Add explicit error message when attempting to map outside the DMA window

David Christensen (1):
  vfio: modify spapr iommu support to use static window sizing

 lib/librte_eal/linux/eal_vfio.c | 421 +++++++++++++++-----------------
 1 file changed, 198 insertions(+), 223 deletions(-)
  

Comments

David Christensen Nov. 9, 2020, 8:35 p.m. UTC | #1
The SPAPR v2 IOMMU used on bare-metal PowerNV systems requires that a DMA
window be defined before mapping/unmapping memory.  The current VFIO code
dynamically resizes this DMA window every time a new memory request is
made, which requires that all existing memory be unmapped/remapped.
While this strategy worked in DPDK 17.11 and earlier where memory was
statically allocated during startup, it is potentially dangerous in DPDK
18.11 and later where memory can be allocated during runtime, temporarily
invalidating IOVA memory used by hardware.

This new code statically sizes the DMA window at startup, based on the
amount of memory installed in the system, avoiding the need to unmap
memory during runtime.
---
v6:
- Fix build error on Linux kernels prior to 4.2.0
- Rebased on 20.11-rc3

v5:
- Modify get_highest_mem_addr to return error, not address
- Add comment regarding sPAPR v1/v2 default window and why it
  needs to be removed
- Added indent to second line of vfio_spapr_dma_mem_map() definition

v4:
- Move file reading code out of vfio_spapr_window_size_walk()

v3:
- Rebase for 20.08

v2:
- Drop patch to wrap ppc64 code with ifdef's
- Add warning when external memory detected
- Change VA memory size detection to scan memseg list when setting DMA window
  for IOVA=VA
- Add explicit error message when attempting to map outside the DMA window

David Christensen (1):
  vfio: modify spapr iommu support to use static window sizing

 lib/librte_eal/linux/eal_vfio.c | 430 +++++++++++++++-----------------
 1 file changed, 207 insertions(+), 223 deletions(-)
  
David Christensen Nov. 10, 2020, 5:41 p.m. UTC | #2
From: David Christensen <drc@linux.vnet.ibm.com>

The SPAPR v2 IOMMU used on bare-metal PowerNV systems requires that a DMA
window be defined before mapping/unmapping memory.  The current VFIO code
dynamically resizes this DMA window every time a new memory request is
made, which requires that all existing memory be unmapped/remapped.
While this strategy worked in DPDK 17.11 and earlier where memory was
statically allocated during startup, it is potentially dangerous in DPDK
18.11 and later where memory can be allocated during runtime, temporarily
invalidating IOVA memory used by hardware.

This new code statically sizes the DMA window at startup, based on the
amount of memory installed in the system, avoiding the need to unmap
memory during runtime.
---
v7:
- No patch changes, fixed email patch description

v6:
- Fix build error on Linux kernels prior to 4.2.0
- Rebased on 20.11-rc3

v5:
- Modify get_highest_mem_addr to return error, not address
- Add comment regarding sPAPR v1/v2 default window and why it
  needs to be removed
- Added indent to second line of vfio_spapr_dma_mem_map() definition

v4:
- Move file reading code out of vfio_spapr_window_size_walk()

v3:
- Rebase for 20.08

v2:
- Drop patch to wrap ppc64 code with ifdef's
- Add warning when external memory detected
- Change VA memory size detection to scan memseg list when setting DMA window
  for IOVA=VA
- Add explicit error message when attempting to map outside the DMA window

David Christensen (1):
  vfio: modify spapr iommu support to use static window sizing

 lib/librte_eal/linux/eal_vfio.c | 430 +++++++++++++++-----------------
 1 file changed, 207 insertions(+), 223 deletions(-)