[dpdk-dev,v3] doc: add driver limitation for vhost dequeue zero copy

  In vhost-switch example, when binding nic to vfio-pci with iommu enabled,
dequeue zero copy cannot work in VM2NIC mode due to no iommu dma mapping
is setup for guest memory currently.

Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
---
Changes in V3:
- update limitation to iommu
Changes in V2:
- add doc in vhost lib

 doc/guides/prog_guide/vhost_lib.rst | 5 +++++
 doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
 2 files changed, 9 insertions(+), 1 deletion(-)
  

On 3/9/2018 6:07 PM, Junjie Chen wrote:
> In vhost-switch example, when binding nic to vfio-pci with iommu enabled,
> dequeue zero copy cannot work in VM2NIC mode due to no iommu dma mapping
> is setup for guest memory currently.
>
> Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
> ---
> Changes in V3:
> - update limitation to iommu
> Changes in V2:
> - add doc in vhost lib
>
>   doc/guides/prog_guide/vhost_lib.rst | 5 +++++
>   doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
>   2 files changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
> index 18227b6..06d214f 100644
> --- a/doc/guides/prog_guide/vhost_lib.rst
> +++ b/doc/guides/prog_guide/vhost_lib.rst
> @@ -83,6 +83,11 @@ The following is an overview of some key Vhost API functions:
>         of those segments, thus the fewer the segments, the quicker we will get
>         the mapping. NOTE: we may speed it by using tree searching in future.
>   
> +    * zero copy does not work when using driver with iommu mode currently, this

Considering FreeBSD driver nic_uio does not support iommu, we can make 
"driver" more explicitly. For example,

"driver with iommu mode" -> "vfio-pci with iommu mode"

> +      is because we don't setup iommu dma mapping for guest memory.

> For example, when you bind device to vfio-pci driver, you need to set driver to work in noiommu mode.

Reword the above sentence a little bit:
"If vfio-pci is a must in your case, insert vfio-pci kernel module in 
noiommu mode."

> +
>     - ``RTE_VHOST_USER_IOMMU_SUPPORT``
>   
>       IOMMU support will be enabled when this flag is set. It is disabled by
> diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst
> index a4bdc6a..f0bb169 100644
> --- a/doc/guides/sample_app_ug/vhost.rst
> +++ b/doc/guides/sample_app_ug/vhost.rst
> @@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx retry is enabled. The
>   default value is 15.
>   
>   **--dequeue-zero-copy**
> -Dequeue zero copy will be enabled when this option is given.
> +Dequeue zero copy will be enabled when this option is given. it is worth to
> +note that if NIC is binded to driver with iommu enabled, dequeue zero copy
> +cannot work at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu
> +dma mapping for guest memory.
>   
>   **--vlan-strip 0|1**
>   VLAN strip option is removed, because different NICs have different behaviors
  

Hi Jianfeng.

> -----Original Message-----
> From: Tan, Jianfeng
> Sent: Monday, March 12, 2018 11:15 AM
> To: Chen, Junjie J <junjie.j.chen@intel.com>; yliu@fridaylinux.org;
> maxime.coquein@redhat.com
> Cc: dev@dpdk.org
> Subject: Re: [PATCH v3] doc: add driver limitation for vhost dequeue zero copy
> 
> 
> 
> On 3/9/2018 6:07 PM, Junjie Chen wrote:
> > In vhost-switch example, when binding nic to vfio-pci with iommu
> > enabled, dequeue zero copy cannot work in VM2NIC mode due to no iommu
> > dma mapping is setup for guest memory currently.
> >
> > Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
> > ---
> > Changes in V3:
> > - update limitation to iommu
> > Changes in V2:
> > - add doc in vhost lib
> >
> >   doc/guides/prog_guide/vhost_lib.rst | 5 +++++
> >   doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
> >   2 files changed, 9 insertions(+), 1 deletion(-)
> >
> > diff --git a/doc/guides/prog_guide/vhost_lib.rst
> > b/doc/guides/prog_guide/vhost_lib.rst
> > index 18227b6..06d214f 100644
> > --- a/doc/guides/prog_guide/vhost_lib.rst
> > +++ b/doc/guides/prog_guide/vhost_lib.rst
> > @@ -83,6 +83,11 @@ The following is an overview of some key Vhost API
> functions:
> >         of those segments, thus the fewer the segments, the quicker we will
> get
> >         the mapping. NOTE: we may speed it by using tree searching in
> future.
> >
> > +    * zero copy does not work when using driver with iommu mode
> > + currently, this
> 
> Considering FreeBSD driver nic_uio does not support iommu, we can make
> "driver" more explicitly. For example,
> 
> "driver with iommu mode" -> "vfio-pci with iommu mode"
Will update.
> 
> > +      is because we don't setup iommu dma mapping for guest memory.
> 
> > For example, when you bind device to vfio-pci driver, you need to set driver to
> work in noiommu mode.
> 
> Reword the above sentence a little bit:
> "If vfio-pci is a must in your case, insert vfio-pci kernel module in noiommu
> mode."
Will update, thanks.

> > +
> >     - ``RTE_VHOST_USER_IOMMU_SUPPORT``
> >
> >       IOMMU support will be enabled when this flag is set. It is
> > disabled by diff --git a/doc/guides/sample_app_ug/vhost.rst
> > b/doc/guides/sample_app_ug/vhost.rst
> > index a4bdc6a..f0bb169 100644
> > --- a/doc/guides/sample_app_ug/vhost.rst
> > +++ b/doc/guides/sample_app_ug/vhost.rst
> > @@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx
> retry is enabled. The
> >   default value is 15.
> >
> >   **--dequeue-zero-copy**
> > -Dequeue zero copy will be enabled when this option is given.
> > +Dequeue zero copy will be enabled when this option is given. it is
> > +worth to note that if NIC is binded to driver with iommu enabled,
> > +dequeue zero copy cannot work at VM2NIC mode (vm2vm=0) due to
> > +currently we don't setup iommu dma mapping for guest memory.
> >
> >   **--vlan-strip 0|1**
> >   VLAN strip option is removed, because different NICs have different
> > behaviors