diff mbox series

doc: Clarify IOMMU usage with "uio-pci" kernel module

Message ID 1536051547-8797-1-git-send-email-tone.zhang@arm.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series doc: Clarify IOMMU usage with "uio-pci" kernel module |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

tone.zhang Sept. 4, 2018, 8:59 a.m. UTC 
  When binding the devices used by DPDK to the "uio-pci" kernel module,
the IOMMU should be disabled in order not to break the IO transmission
because of the virtual / physical address mapping.

The patch clarifies the IOMMU configuration on both x86_64 and arm64
systems.

Signed-off-by: tone.zhang <tone.zhang@arm.com>
---
 doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
 1 file changed, 7 insertions(+)

Comments

Gavin Hu Sept. 4, 2018, 9:16 a.m. UTC | #1 
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of tone.zhang
> Sent: Tuesday, September 4, 2018 4:59 PM
> To: dev@dpdk.org
> Cc: nd <nd@arm.com>
> Subject: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel
> module
> 
> When binding the devices used by DPDK to the "uio-pci" kernel module, the
> IOMMU should be disabled in order not to break the IO transmission
> because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configuration on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
Acked-by: Gavin Hu <gavin.hu@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..8f9ec8f 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
>     For more details see :ref:`linux_gsg_binding_kernel` below.
> 
> +.. note::
> +
> +   If the devices for used DPDK bound to the ``uio-pci`` kernel module, please
> make
> +   sure that the IOMMU is disabled. We can add ``intel_iommu=off`` or
> ``amd_iommu=off``
> +   in ``GRUB_CMDLINE_LINUX`` in grub on x86_64 systems, or add
> ``iommu.passthrough=1``
> +   on arm64 system.
> +
>  Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
> for platforms that support using VFIO.
> 
> --
> 2.7.4
Bruce Richardson Sept. 4, 2018, 10:06 a.m. UTC | #2 
On Tue, Sep 04, 2018 at 04:59:07PM +0800, tone.zhang wrote:
> When binding the devices used by DPDK to the "uio-pci" kernel module,
> the IOMMU should be disabled in order not to break the IO transmission
> because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configuration on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..8f9ec8f 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
>     For more details see :ref:`linux_gsg_binding_kernel` below.
>  
> +.. note::
> +
> +   If the devices for used DPDK bound to the ``uio-pci`` kernel module, please make
> +   sure that the IOMMU is disabled. We can add ``intel_iommu=off`` or ``amd_iommu=off``
> +   in ``GRUB_CMDLINE_LINUX`` in grub on x86_64 systems, or add ``iommu.passthrough=1``
> +   on arm64 system.
> +

I think passthrough mode should work on x86 too. I remember running with
iommu=pt setting in the kernel in the past.

/Bruce
Luca Boccassi Sept. 4, 2018, 10:15 a.m. UTC | #3 
On Tue, 2018-09-04 at 11:06 +0100, Bruce Richardson wrote:
> On Tue, Sep 04, 2018 at 04:59:07PM +0800, tone.zhang wrote:
> > When binding the devices used by DPDK to the "uio-pci" kernel
> > module,
> > the IOMMU should be disabled in order not to break the IO
> > transmission
> > because of the virtual / physical address mapping.
> > 
> > The patch clarifies the IOMMU configuration on both x86_64 and
> > arm64
> > systems.
> > 
> > Signed-off-by: tone.zhang <tone.zhang@arm.com>
> > ---
> >  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> > b/doc/guides/linux_gsg/linux_drivers.rst
> > index 371a817..8f9ec8f 100644
> > --- a/doc/guides/linux_gsg/linux_drivers.rst
> > +++ b/doc/guides/linux_gsg/linux_drivers.rst
> > @@ -48,6 +48,13 @@ be loaded as shown below:
> >     ``vfio-pci`` kernel module rather than ``igb_uio`` or
> > ``uio_pci_generic``.
> >     For more details see :ref:`linux_gsg_binding_kernel` below.
> >  
> > +.. note::
> > +
> > +   If the devices for used DPDK bound to the ``uio-pci`` kernel
> > module, please make
> > +   sure that the IOMMU is disabled. We can add ``intel_iommu=off``
> > or ``amd_iommu=off``
> > +   in ``GRUB_CMDLINE_LINUX`` in grub on x86_64 systems, or add
> > ``iommu.passthrough=1``
> > +   on arm64 system.
> > +
> 
> I think passthrough mode should work on x86 too. I remember running
> with
> iommu=pt setting in the kernel in the past.
> 
> /Bruce

It does, can confirm.
tone.zhang Sept. 4, 2018, 10:25 a.m. UTC | #4 
-----Original Message-----
From: Luca Boccassi <bluca@debian.org> 
Sent: Tuesday, September 4, 2018 6:15 PM
To: Bruce Richardson <bruce.richardson@intel.com>; Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
Cc: dev@dpdk.org; nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

On Tue, 2018-09-04 at 11:06 +0100, Bruce Richardson wrote:
> On Tue, Sep 04, 2018 at 04:59:07PM +0800, tone.zhang wrote:
> > When binding the devices used by DPDK to the "uio-pci" kernel 
> > module, the IOMMU should be disabled in order not to break the IO 
> > transmission because of the virtual / physical address mapping.
> > 
> > The patch clarifies the IOMMU configuration on both x86_64 and
> > arm64
> > systems.
> > 
> > Signed-off-by: tone.zhang <tone.zhang@arm.com>
> > ---
> >  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> > b/doc/guides/linux_gsg/linux_drivers.rst
> > index 371a817..8f9ec8f 100644
> > --- a/doc/guides/linux_gsg/linux_drivers.rst
> > +++ b/doc/guides/linux_gsg/linux_drivers.rst
> > @@ -48,6 +48,13 @@ be loaded as shown below:
> >     ``vfio-pci`` kernel module rather than ``igb_uio`` or 
> > ``uio_pci_generic``.
> >     For more details see :ref:`linux_gsg_binding_kernel` below.
> >  
> > +.. note::
> > +
> > +   If the devices for used DPDK bound to the ``uio-pci`` kernel
> > module, please make
> > +   sure that the IOMMU is disabled. We can add ``intel_iommu=off``
> > or ``amd_iommu=off``
> > +   in ``GRUB_CMDLINE_LINUX`` in grub on x86_64 systems, or add
> > ``iommu.passthrough=1``
> > +   on arm64 system.
> > +
> 
> I think passthrough mode should work on x86 too. I remember running 
> with iommu=pt setting in the kernel in the past.
> 
> /Bruce

It does, can confirm.

--
Kind regards,
Luca Boccassi


@Luca, @Bruce, 
Thanks  for the comments. I will update the change and push V2. Thanks!

Br,
Tone
Rami Rosen Sept. 4, 2018, 7:55 p.m. UTC | #5 
Hi Tone,
>if the devices for used DPDK bound to the ``uio-pci`` kernel module,
please make

The three kernel modules which can be used for DPDK binding are vfio-pci,
uio_pci_generic and igb_uio. Don't you mean here uio_pci_generic ?

Regards,
Rami Rosen


בתאריך יום ג׳, 4 בספט׳ 2018, 11:59, מאת tone.z

> When binding the devices used by DPDK to the "uio-pci" kernel module,
> the IOMMU should be disabled in order not to break the
> because of the virtual / physical address mapping.
>
> The patch clarifies the IOMMU configuration on both x86_64 and arm64
> systems.
>
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..8f9ec8f 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci``
> 2.7.4
>
>
tone.zhang Sept. 5, 2018, 12:20 a.m. UTC | #6 
Hi Rami,

Yes, I mean “uio_pci_generic” kernel module.

I will update the change accordingly.

Thanks.

Br,
Tone

From: Rami Rosen <roszenrami@gmail.com>
Sent: Wednesday, September 5, 2018 3:55 AM
To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
Cc: dev@dpdk.org; nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

Hi Tone,
>if the devices for used DPDK bound to the ``uio-pci`` kernel module, please make

The three kernel modules which can be used for DPDK binding are vfio-pci, uio_pci_generic and igb_uio. Don't you mean here uio_pci_generic ?

Regards,
Rami Rosen


בתאריך יום ג׳, 4 בספט׳ 2018, 11:59, מאת tone.z
When binding the devices used by DPDK to the "uio-pci" kernel module,
the IOMMU should be disabled in order not to break the
because of the virtual / physical address mapping.

The patch clarifies the IOMMU configuration on both x86_64 and arm64
systems.

Signed-off-by: tone.zhang <tone.zhang@arm.com<mailto:tone.zhang@arm.com>>
---
 doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 371a817..8f9ec8f 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -48,6 +48,13 @@ be loaded as shown below:
    ``vfio-pci``
2.7.4
Stephen Hemminger Sept. 5, 2018, 9:22 a.m. UTC | #7 
On Wed, 5 Sep 2018 00:20:29 +0000
"Tone Zhang (Arm Technology China)" <Tone.Zhang@arm.com> wrote:

> Hi Rami,
> 
> Yes, I mean “uio_pci_generic” kernel module.
> 
> I will update the change accordingly.
> 
> Thanks.
> 
> Br,
> Tone
> 
> From: Rami Rosen <roszenrami@gmail.com>
> Sent: Wednesday, September 5, 2018 3:55 AM
> To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
> Cc: dev@dpdk.org; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module
> 
> Hi Tone,
> >if the devices for used DPDK bound to the ``uio-pci`` kernel module, please make  
> 
> The three kernel modules which can be used for DPDK binding are vfio-pci, uio_pci_generic and igb_uio. Don't you mean here uio_pci_generic ?
> 
> Regards,
> Rami Rosen
> 
> 
> בתאריך יום ג׳, 4 בספט׳ 2018, 11:59, מאת tone.z
> When binding the devices used by DPDK to the "uio-pci" kernel module,
> the IOMMU should be disabled in order not to break the
> because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configuration on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com<mailto:tone.zhang@arm.com>>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..8f9ec8f 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci``
> 2.7.4

Also, it would be good if documentation was clear in recommending using VFIO
instead if IOMMU is present.
tone.zhang Sept. 5, 2018, 9:49 a.m. UTC | #8 
-----Original Message-----
From: Stephen Hemminger <stephen@networkplumber.org> 
Sent: Wednesday, September 5, 2018 5:22 PM
To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
Cc: Rami Rosen <roszenrami@gmail.com>; dev@dpdk.org; nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

On Wed, 5 Sep 2018 00:20:29 +0000
"Tone Zhang (Arm Technology China)" <Tone.Zhang@arm.com> wrote:

> Hi Rami,
> 
> Yes, I mean “uio_pci_generic” kernel module.
> 
> I will update the change accordingly.
> 
> Thanks.
> 
> Br,
> Tone
> 
> From: Rami Rosen <roszenrami@gmail.com>
> Sent: Wednesday, September 5, 2018 3:55 AM
> To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
> Cc: dev@dpdk.org; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module
> 
> Hi Tone,
> >if the devices for used DPDK bound to the ``uio-pci`` kernel module, please make  
> 
> The three kernel modules which can be used for DPDK binding are vfio-pci, uio_pci_generic and igb_uio. Don't you mean here uio_pci_generic ?
> 
> Regards,
> Rami Rosen
> 
> 
> בתאריך יום ג׳, 4 בספט׳ 2018, 11:59, מאת tone.z
> When binding the devices used by DPDK to the "uio-pci" kernel module,
> the IOMMU should be disabled in order not to break the
> because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configuration on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com<mailto:tone.zhang@arm.com>>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..8f9ec8f 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci``
> 2.7.4

Also, it would be good if documentation was clear in recommending using VFIO
instead if IOMMU is present.

Hi Stephen,

Yes, according to the description of the "UIO" and "VFIO" in the document, I believe that VFIO is recommended.

BTW, I have update the change to V3 (https://patches.dpdk.org/patch/44295/). Please have a review.

Thanks a lot!

Br,
Tone
diff mbox series

Patch

diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 371a817..8f9ec8f 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -48,6 +48,13 @@  be loaded as shown below:
    ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
    For more details see :ref:`linux_gsg_binding_kernel` below.
 
+.. note::
+
+   If the devices for used DPDK bound to the ``uio-pci`` kernel module, please make
+   sure that the IOMMU is disabled. We can add ``intel_iommu=off`` or ``amd_iommu=off``
+   in ``GRUB_CMDLINE_LINUX`` in grub on x86_64 systems, or add ``iommu.passthrough=1``
+   on arm64 system.
+
 Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
 for platforms that support using VFIO.