[dpdk-dev,2/2] virtio: support IOMMU platform
Commit Message
Virtio pmd doesn't support VFIO in the past since devices bypass IOMMU
completely. But recently, the work of making virtio device work with
IOMMU is near to complete. So this patch make pmd support IOMMU by:
- Allow VFIO mapping by setting RTE_PCI_DRV_NEED_MAPPING flag
- Negotiate feature VIRTIO_F_IOMMU_PLATFORM
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
drivers/net/virtio/virtio_ethdev.c | 2 +-
drivers/net/virtio/virtio_ethdev.h | 3 ++-
drivers/net/virtio/virtio_pci.h | 3 ++-
3 files changed, 5 insertions(+), 3 deletions(-)
Comments
2016-09-02 14:37, Jason Wang:
> Virtio pmd doesn't support VFIO in the past since devices bypass IOMMU
> completely. But recently, the work of making virtio device work with
> IOMMU is near to complete.
Good news!
What are the requirements for Qemu and Linux version numbers please?
On Fri, Sep 02, 2016 at 03:04:56PM +0200, Thomas Monjalon wrote:
> 2016-09-02 14:37, Jason Wang:
> > Virtio pmd doesn't support VFIO in the past since devices bypass IOMMU
> > completely. But recently, the work of making virtio device work with
> > IOMMU is near to complete.
>
> Good news!
> What are the requirements for Qemu and Linux version numbers please?
I expect QEMU 2.8 and Linux 4.8 to have the support.
I know RedHat is working on a vIOMMU so I guess this work is related to
that effort, but it is a surprise virtio using IOMMU. I thought IOMMU just
made sense when using SRIOV. My second guess is using IOMMU with virtio is
a matter of security, but by other hand, virtio + IOMMU could imply serious
performance degradation when multiple VMs are in use. I'm talking about
IOMMU contention, exactly about IOTLB contention. This performance issue is
complex to describe or even analyze as there are several factors having an
impact on it. For example, 1GB hugepages can avoid most of it and the same
if TX & RX rings are not bigger than 256. So, my question: is RedHat aware
of this potential IOMMU contention which can limit scalability?
On Fri, Sep 2, 2016 at 6:26 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Fri, Sep 02, 2016 at 03:04:56PM +0200, Thomas Monjalon wrote:
> > 2016-09-02 14:37, Jason Wang:
> > > Virtio pmd doesn't support VFIO in the past since devices bypass IOMMU
> > > completely. But recently, the work of making virtio device work with
> > > IOMMU is near to complete.
> >
> > Good news!
> > What are the requirements for Qemu and Linux version numbers please?
>
> I expect QEMU 2.8 and Linux 4.8 to have the support.
>
On 2016年09月02日 21:04, Thomas Monjalon wrote:
> 2016-09-02 14:37, Jason Wang:
>> Virtio pmd doesn't support VFIO in the past since devices bypass IOMMU
>> completely. But recently, the work of making virtio device work with
>> IOMMU is near to complete.
> Good news!
> What are the requirements for Qemu and Linux version numbers please?
Linux(vhost) support was in 4.8. Qemu support will be ready at 2.8
Thanks
On 2016年09月02日 21:04, Thomas Monjalon wrote:
> 2016-09-02 14:37, Jason Wang:
>> Virtio pmd doesn't support VFIO in the past since devices bypass IOMMU
>> completely. But recently, the work of making virtio device work with
>> IOMMU is near to complete.
> Good news!
> What are the requirements for Qemu and Linux version numbers please?
Linux 4.8 has support for this. For qemu, it will support this for 2.8.
Thanks
On 2016年09月04日 16:08, Alejandro Lucero wrote:
> I know RedHat is working on a vIOMMU so I guess this work is related
> to that effort, but it is a surprise virtio using IOMMU. I thought
> IOMMU just made sense when using SRIOV. My second guess is using IOMMU
> with virtio is a matter of security, but by other hand, virtio + IOMMU
> could imply serious performance degradation when multiple VMs are in use.
We will use qemu vIOMMU for virito, so there's no such issue.
> I'm talking about IOMMU contention, exactly about IOTLB contention.
I thought device IOTLB (ATS) was just designed to solve this contention.
> This performance issue is complex to describe or even analyze as there
> are several factors having an impact on it. For example, 1GB hugepages
> can avoid most of it and the same if TX & RX rings are not bigger than
> 256. So, my question: is RedHat aware of this potential IOMMU
> contention which can limit scalability?
For virtio, we use vIOMMU per VM and implement a device IOTLB in vhost
side. Technically, it does not have such issue I think.
Thanks
>
> On Fri, Sep 2, 2016 at 6:26 PM, Michael S. Tsirkin <mst@redhat.com
> <mailto:mst@redhat.com>> wrote:
>
> On Fri, Sep 02, 2016 at 03:04:56PM +0200, Thomas Monjalon wrote:
> > 2016-09-02 14:37, Jason Wang:
> > > Virtio pmd doesn't support VFIO in the past since devices
> bypass IOMMU
> > > completely. But recently, the work of making virtio device
> work with
> > > IOMMU is near to complete.
> >
> > Good news!
> > What are the requirements for Qemu and Linux version numbers please?
>
> I expect QEMU 2.8 and Linux 4.8 to have the support.
>
>
Oops, seems I failed to send it out (I happened to know it when
I was checking the patchwork status: I didn't find my comment).
I also happened to find out that I failed to receive quite many
emails, including some patches. Seems something went wrong :(
--yliu
On Mon, Sep 05, 2016 at 03:16:26PM +0800, Yuanhan Liu wrote:
> On Fri, Sep 02, 2016 at 02:37:00PM +0800, Jason Wang wrote:
> > Virtio pmd doesn't support VFIO in the past since devices bypass IOMMU
> > completely. But recently, the work of making virtio device work with
> > IOMMU is near to complete. So this patch make pmd support IOMMU by:
> >
> > - Allow VFIO mapping by setting RTE_PCI_DRV_NEED_MAPPING flag
>
> I think that's not needed, since virtio_read_caps() invokes
> rte_eal_pci_map_device().
>
> --yliu
On 2016年09月06日 15:46, Yuanhan Liu wrote:
> Oops, seems I failed to send it out (I happened to know it when
> I was checking the patchwork status: I didn't find my comment).
>
> I also happened to find out that I failed to receive quite many
> emails, including some patches. Seems something went wrong :(
>
> --yliu
>
> On Mon, Sep 05, 2016 at 03:16:26PM +0800, Yuanhan Liu wrote:
>> On Fri, Sep 02, 2016 at 02:37:00PM +0800, Jason Wang wrote:
>>> Virtio pmd doesn't support VFIO in the past since devices bypass IOMMU
>>> completely. But recently, the work of making virtio device work with
>>> IOMMU is near to complete. So this patch make pmd support IOMMU by:
>>>
>>> - Allow VFIO mapping by setting RTE_PCI_DRV_NEED_MAPPING flag
>> I think that's not needed, since virtio_read_caps() invokes
>> rte_eal_pci_map_device().
>>
>> --yliu
Right.
Thanks.
@@ -1307,7 +1307,7 @@ static struct eth_driver rte_virtio_pmd = {
.pci_drv = {
.name = "rte_virtio_pmd",
.id_table = pci_id_virtio_map,
- .drv_flags = RTE_PCI_DRV_DETACHABLE,
+ .drv_flags = RTE_PCI_DRV_DETACHABLE | RTE_PCI_DRV_NEED_MAPPING,
},
.eth_dev_init = eth_virtio_dev_init,
.eth_dev_uninit = eth_virtio_dev_uninit,
@@ -63,7 +63,8 @@
1u << VIRTIO_NET_F_CTRL_RX | \
1u << VIRTIO_NET_F_CTRL_VLAN | \
1u << VIRTIO_NET_F_MRG_RXBUF | \
- 1ULL << VIRTIO_F_VERSION_1)
+ 1ULL << VIRTIO_F_VERSION_1 | \
+ 1ULL << VIRTIO_F_IOMMU_PLATFORM )
/*
* CQ function prototype
@@ -139,6 +139,7 @@ struct virtnet_ctl;
#define VIRTIO_RING_F_INDIRECT_DESC 28
#define VIRTIO_F_VERSION_1 32
+#define VIRTIO_F_IOMMU_PLATFORM 33
/*
* Some VirtIO feature bits (currently bits 28 through 31) are
@@ -146,7 +147,7 @@ struct virtnet_ctl;
* rest are per-device feature bits.
*/
#define VIRTIO_TRANSPORT_F_START 28
-#define VIRTIO_TRANSPORT_F_END 32
+#define VIRTIO_TRANSPORT_F_END 34
/* The Guest publishes the used index for which it expects an interrupt
* at the end of the avail ring. Host should ignore the avail->flags field. */