[dpdk-dev,v2,1/2] vfio: fix build if build envrionment is on old kernel
Commit Message
From: Stephen Hemminger <shemming@brocade.com>
The build of DPDK may be done on a system where Linux headers
in /usr/include (and therefore kernel version macro) are much
older than the target runtime system.
In order to work around this, one solution is to put in simplified
kernel header (this is what the compat stuff is already doing).
The application will still have to check at runtime that VFIO is
present, but that is manageable.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_eal/linuxapp/eal/compat_vfio.h | 181 ++++++++++++++++++++++++++++++
lib/librte_eal/linuxapp/eal/eal_vfio.h | 13 ++-
2 files changed, 188 insertions(+), 6 deletions(-)
create mode 100644 lib/librte_eal/linuxapp/eal/compat_vfio.h
Comments
2015-07-09 16:26, Stephen Hemminger:
> From: Stephen Hemminger <shemming@brocade.com>
>
> The build of DPDK may be done on a system where Linux headers
> in /usr/include (and therefore kernel version macro) are much
> older than the target runtime system.
It seems strange wanting to build a feature not present in the kernel
headers. Why not upgrading the build system kernel?
> In order to work around this, one solution is to put in simplified
> kernel header (this is what the compat stuff is already doing).
The other solution (as already suggested by Anatoly) is to have a configure
script (not an autotool one). It would make clear that VFIO support is not
built.
Ref: http://dpdk.org/ml/archives/dev/2015-April/016772.html
Hi Stephen,
> --- /dev/null
> +++ b/lib/librte_eal/linuxapp/eal/compat_vfio.h
> @@ -0,0 +1,181 @@
Wouldn't this need a GPL license header?
Thanks,Anatoly
On Fri, 10 Jul 2015 10:13:41 +0000
"Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:
> Hi Stephen,
>
> > --- /dev/null
> > +++ b/lib/librte_eal/linuxapp/eal/compat_vfio.h
> > @@ -0,0 +1,181 @@
>
> Wouldn't this need a GPL license header?
>
> Thanks,Anatoly
No. It is perfectly valid to create new headers with
different license (at least until SCOTUS changes their mind).
On Fri, 10 Jul 2015 01:43:17 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> 2015-07-09 16:26, Stephen Hemminger:
> > From: Stephen Hemminger <shemming@brocade.com>
> >
> > The build of DPDK may be done on a system where Linux headers
> > in /usr/include (and therefore kernel version macro) are much
> > older than the target runtime system.
>
> It seems strange wanting to build a feature not present in the kernel
> headers. Why not upgrading the build system kernel?
The build machine is running Debian with stock headers and kernel.
I can see many people having build environments controlled by other
parts of organization where you are not allowed to update packages.
> > In order to work around this, one solution is to put in simplified
> > kernel header (this is what the compat stuff is already doing).
>
> The other solution (as already suggested by Anatoly) is to have a configure
> script (not an autotool one). It would make clear that VFIO support is not
> built.
> Ref: http://dpdk.org/ml/archives/dev/2015-April/016772.html
That won't solve the issue.
The other possiblities are:
1. Fail the build if VFIO is configured but headers files are missing.
This is better than surprising user at runtime.
2. Don't use /usr/include/linux as path as expected path of kernel headers
instead use build path (this is what modules do).
2015-07-10 10:29, Stephen Hemminger:
> On Fri, 10 Jul 2015 01:43:17 +0200
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
>
> > 2015-07-09 16:26, Stephen Hemminger:
> > > From: Stephen Hemminger <shemming@brocade.com>
> > >
> > > The build of DPDK may be done on a system where Linux headers
> > > in /usr/include (and therefore kernel version macro) are much
> > > older than the target runtime system.
> >
> > It seems strange wanting to build a feature not present in the kernel
> > headers. Why not upgrading the build system kernel?
>
> The build machine is running Debian with stock headers and kernel.
> I can see many people having build environments controlled by other
> parts of organization where you are not allowed to update packages.
>
> > > In order to work around this, one solution is to put in simplified
> > > kernel header (this is what the compat stuff is already doing).
> >
> > The other solution (as already suggested by Anatoly) is to have a configure
> > script (not an autotool one). It would make clear that VFIO support is not
> > built.
> > Ref: http://dpdk.org/ml/archives/dev/2015-April/016772.html
>
> That won't solve the issue.
That allows to see the error earlier and more clearly.
> The other possiblities are:
> 1. Fail the build if VFIO is configured but headers files are missing.
> This is better than surprising user at runtime.
> 2. Don't use /usr/include/linux as path as expected path of kernel headers
> instead use build path (this is what modules do).
The latter one seems to be the right one. There is a name for that technique:
cross-compilation. If you want to build DPDK to run on another system with
different kernel and libraries, you should adapt the toolchain and libraries
headers. And guess what? include/linux/ is part of this toolchain.
Problem solved.
On Fri, 10 Jul 2015 22:19:58 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> 2015-07-10 10:29, Stephen Hemminger:
> > On Fri, 10 Jul 2015 01:43:17 +0200
> > Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> >
> > > 2015-07-09 16:26, Stephen Hemminger:
> > > > From: Stephen Hemminger <shemming@brocade.com>
> > > >
> > > > The build of DPDK may be done on a system where Linux headers
> > > > in /usr/include (and therefore kernel version macro) are much
> > > > older than the target runtime system.
> > >
> > > It seems strange wanting to build a feature not present in the kernel
> > > headers. Why not upgrading the build system kernel?
> >
> > The build machine is running Debian with stock headers and kernel.
> > I can see many people having build environments controlled by other
> > parts of organization where you are not allowed to update packages.
> >
> > > > In order to work around this, one solution is to put in simplified
> > > > kernel header (this is what the compat stuff is already doing).
> > >
> > > The other solution (as already suggested by Anatoly) is to have a configure
> > > script (not an autotool one). It would make clear that VFIO support is not
> > > built.
> > > Ref: http://dpdk.org/ml/archives/dev/2015-April/016772.html
> >
> > That won't solve the issue.
>
> That allows to see the error earlier and more clearly.
>
> > The other possiblities are:
> > 1. Fail the build if VFIO is configured but headers files are missing.
> > This is better than surprising user at runtime.
> > 2. Don't use /usr/include/linux as path as expected path of kernel headers
> > instead use build path (this is what modules do).
>
> The latter one seems to be the right one. There is a name for that technique:
> cross-compilation. If you want to build DPDK to run on another system with
> different kernel and libraries, you should adapt the toolchain and libraries
> headers. And guess what? include/linux/ is part of this toolchain.
> Problem solved.
The toolchain Just-works for build igb_uio, kni, etc. Why doesn't it work
for VFIO. I would view that as a of the mk scripts.
2015-07-10 13:44, Stephen Hemminger:
> On Fri, 10 Jul 2015 22:19:58 +0200
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
>
> > 2015-07-10 10:29, Stephen Hemminger:
> > > On Fri, 10 Jul 2015 01:43:17 +0200
> > > Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> > >
> > > > 2015-07-09 16:26, Stephen Hemminger:
> > > > > From: Stephen Hemminger <shemming@brocade.com>
> > > > >
> > > > > The build of DPDK may be done on a system where Linux headers
> > > > > in /usr/include (and therefore kernel version macro) are much
> > > > > older than the target runtime system.
> > > >
> > > > It seems strange wanting to build a feature not present in the kernel
> > > > headers. Why not upgrading the build system kernel?
> > >
> > > The build machine is running Debian with stock headers and kernel.
> > > I can see many people having build environments controlled by other
> > > parts of organization where you are not allowed to update packages.
> > >
> > > > > In order to work around this, one solution is to put in simplified
> > > > > kernel header (this is what the compat stuff is already doing).
> > > >
> > > > The other solution (as already suggested by Anatoly) is to have a configure
> > > > script (not an autotool one). It would make clear that VFIO support is not
> > > > built.
> > > > Ref: http://dpdk.org/ml/archives/dev/2015-April/016772.html
> > >
> > > That won't solve the issue.
> >
> > That allows to see the error earlier and more clearly.
> >
> > > The other possiblities are:
> > > 1. Fail the build if VFIO is configured but headers files are missing.
> > > This is better than surprising user at runtime.
> > > 2. Don't use /usr/include/linux as path as expected path of kernel headers
> > > instead use build path (this is what modules do).
> >
> > The latter one seems to be the right one. There is a name for that technique:
> > cross-compilation. If you want to build DPDK to run on another system with
> > different kernel and libraries, you should adapt the toolchain and libraries
> > headers. And guess what? include/linux/ is part of this toolchain.
> > Problem solved.
>
> The toolchain Just-works for build igb_uio, kni, etc. Why doesn't it work
> for VFIO. I would view that as a of the mk scripts.
I mean you have to use a cross toolchain including the kernel headers matching
the target kernel. A cross toolchain can be used thanks to the CROSS variable.
new file mode 100644
@@ -0,0 +1,181 @@
+#ifndef VFIO_H
+#define VFIO_H
+/* This is a santized version of the kernel header for vfio.
+ * It is included to allow build DPDK with VFIO when build system
+ * kernel version is older than runtime system kernel version.
+ */
+
+#include <linux/types.h>
+#include <linux/ioctl.h>
+#define VFIO_API_VERSION 0
+#define VFIO_TYPE1_IOMMU 1
+#define VFIO_SPAPR_TCE_IOMMU 2
+#define VFIO_TYPE1v2_IOMMU 3
+#define VFIO_DMA_CC_IOMMU 4
+#define VFIO_EEH 5
+#define VFIO_TYPE1_NESTING_IOMMU 6
+#define VFIO_TYPE (';')
+#define VFIO_BASE 100
+#define VFIO_GET_API_VERSION _IO(VFIO_TYPE, VFIO_BASE + 0)
+#define VFIO_CHECK_EXTENSION _IO(VFIO_TYPE, VFIO_BASE + 1)
+#define VFIO_SET_IOMMU _IO(VFIO_TYPE, VFIO_BASE + 2)
+struct vfio_group_status {
+ __u32 argsz;
+ __u32 flags;
+#define VFIO_GROUP_FLAGS_VIABLE (1 << 0)
+#define VFIO_GROUP_FLAGS_CONTAINER_SET (1 << 1)
+};
+#define VFIO_GROUP_GET_STATUS _IO(VFIO_TYPE, VFIO_BASE + 3)
+#define VFIO_GROUP_SET_CONTAINER _IO(VFIO_TYPE, VFIO_BASE + 4)
+#define VFIO_GROUP_UNSET_CONTAINER _IO(VFIO_TYPE, VFIO_BASE + 5)
+#define VFIO_GROUP_GET_DEVICE_FD _IO(VFIO_TYPE, VFIO_BASE + 6)
+struct vfio_device_info {
+ __u32 argsz;
+ __u32 flags;
+#define VFIO_DEVICE_FLAGS_RESET (1 << 0)
+#define VFIO_DEVICE_FLAGS_PCI (1 << 1)
+#define VFIO_DEVICE_FLAGS_PLATFORM (1 << 2)
+#define VFIO_DEVICE_FLAGS_AMBA (1 << 3)
+ __u32 num_regions;
+ __u32 num_irqs;
+};
+#define VFIO_DEVICE_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 7)
+struct vfio_region_info {
+ __u32 argsz;
+ __u32 flags;
+#define VFIO_REGION_INFO_FLAG_READ (1 << 0)
+#define VFIO_REGION_INFO_FLAG_WRITE (1 << 1)
+#define VFIO_REGION_INFO_FLAG_MMAP (1 << 2)
+ __u32 index;
+ __u32 resv;
+ __u64 size;
+ __u64 offset;
+};
+#define VFIO_DEVICE_GET_REGION_INFO _IO(VFIO_TYPE, VFIO_BASE + 8)
+struct vfio_irq_info {
+ __u32 argsz;
+ __u32 flags;
+#define VFIO_IRQ_INFO_EVENTFD (1 << 0)
+#define VFIO_IRQ_INFO_MASKABLE (1 << 1)
+#define VFIO_IRQ_INFO_AUTOMASKED (1 << 2)
+#define VFIO_IRQ_INFO_NORESIZE (1 << 3)
+ __u32 index;
+ __u32 count;
+};
+#define VFIO_DEVICE_GET_IRQ_INFO _IO(VFIO_TYPE, VFIO_BASE + 9)
+struct vfio_irq_set {
+ __u32 argsz;
+ __u32 flags;
+#define VFIO_IRQ_SET_DATA_NONE (1 << 0)
+#define VFIO_IRQ_SET_DATA_BOOL (1 << 1)
+#define VFIO_IRQ_SET_DATA_EVENTFD (1 << 2)
+#define VFIO_IRQ_SET_ACTION_MASK (1 << 3)
+#define VFIO_IRQ_SET_ACTION_UNMASK (1 << 4)
+#define VFIO_IRQ_SET_ACTION_TRIGGER (1 << 5)
+ __u32 index;
+ __u32 start;
+ __u32 count;
+ __u8 data[];
+};
+#define VFIO_DEVICE_SET_IRQS _IO(VFIO_TYPE, VFIO_BASE + 10)
+#define VFIO_IRQ_SET_DATA_TYPE_MASK (VFIO_IRQ_SET_DATA_NONE | \
+ VFIO_IRQ_SET_DATA_BOOL | \
+ VFIO_IRQ_SET_DATA_EVENTFD)
+#define VFIO_IRQ_SET_ACTION_TYPE_MASK (VFIO_IRQ_SET_ACTION_MASK | \
+ VFIO_IRQ_SET_ACTION_UNMASK | \
+ VFIO_IRQ_SET_ACTION_TRIGGER)
+#define VFIO_DEVICE_RESET _IO(VFIO_TYPE, VFIO_BASE + 11)
+enum {
+ VFIO_PCI_BAR0_REGION_INDEX,
+ VFIO_PCI_BAR1_REGION_INDEX,
+ VFIO_PCI_BAR2_REGION_INDEX,
+ VFIO_PCI_BAR3_REGION_INDEX,
+ VFIO_PCI_BAR4_REGION_INDEX,
+ VFIO_PCI_BAR5_REGION_INDEX,
+ VFIO_PCI_ROM_REGION_INDEX,
+ VFIO_PCI_CONFIG_REGION_INDEX,
+ VFIO_PCI_VGA_REGION_INDEX,
+ VFIO_PCI_NUM_REGIONS
+};
+enum {
+ VFIO_PCI_INTX_IRQ_INDEX,
+ VFIO_PCI_MSI_IRQ_INDEX,
+ VFIO_PCI_MSIX_IRQ_INDEX,
+ VFIO_PCI_ERR_IRQ_INDEX,
+ VFIO_PCI_REQ_IRQ_INDEX,
+ VFIO_PCI_NUM_IRQS
+};
+struct vfio_pci_dependent_device {
+ __u32 group_id;
+ __u16 segment;
+ __u8 bus;
+ __u8 devfn;
+};
+struct vfio_pci_hot_reset_info {
+ __u32 argsz;
+ __u32 flags;
+ __u32 count;
+ struct vfio_pci_dependent_device devices[];
+};
+#define VFIO_DEVICE_GET_PCI_HOT_RESET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
+struct vfio_pci_hot_reset {
+ __u32 argsz;
+ __u32 flags;
+ __u32 count;
+ __s32 group_fds[];
+};
+#define VFIO_DEVICE_PCI_HOT_RESET _IO(VFIO_TYPE, VFIO_BASE + 13)
+struct vfio_iommu_type1_info {
+ __u32 argsz;
+ __u32 flags;
+#define VFIO_IOMMU_INFO_PGSIZES (1 << 0)
+ __u64 iova_pgsizes;
+};
+#define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
+struct vfio_iommu_type1_dma_map {
+ __u32 argsz;
+ __u32 flags;
+#define VFIO_DMA_MAP_FLAG_READ (1 << 0)
+#define VFIO_DMA_MAP_FLAG_WRITE (1 << 1)
+ __u64 vaddr;
+ __u64 iova;
+ __u64 size;
+};
+#define VFIO_IOMMU_MAP_DMA _IO(VFIO_TYPE, VFIO_BASE + 13)
+struct vfio_iommu_type1_dma_unmap {
+ __u32 argsz;
+ __u32 flags;
+ __u64 iova;
+ __u64 size;
+};
+#define VFIO_IOMMU_UNMAP_DMA _IO(VFIO_TYPE, VFIO_BASE + 14)
+#define VFIO_IOMMU_ENABLE _IO(VFIO_TYPE, VFIO_BASE + 15)
+#define VFIO_IOMMU_DISABLE _IO(VFIO_TYPE, VFIO_BASE + 16)
+struct vfio_iommu_spapr_tce_info {
+ __u32 argsz;
+ __u32 flags;
+ __u32 dma32_window_start;
+ __u32 dma32_window_size;
+};
+#define VFIO_IOMMU_SPAPR_TCE_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
+struct vfio_eeh_pe_op {
+ __u32 argsz;
+ __u32 flags;
+ __u32 op;
+};
+#define VFIO_EEH_PE_DISABLE 0
+#define VFIO_EEH_PE_ENABLE 1
+#define VFIO_EEH_PE_UNFREEZE_IO 2
+#define VFIO_EEH_PE_UNFREEZE_DMA 3
+#define VFIO_EEH_PE_GET_STATE 4
+#define VFIO_EEH_PE_STATE_NORMAL 0
+#define VFIO_EEH_PE_STATE_RESET 1
+#define VFIO_EEH_PE_STATE_STOPPED 2
+#define VFIO_EEH_PE_STATE_STOPPED_DMA 4
+#define VFIO_EEH_PE_STATE_UNAVAIL 5
+#define VFIO_EEH_PE_RESET_DEACTIVATE 5
+#define VFIO_EEH_PE_RESET_HOT 6
+#define VFIO_EEH_PE_RESET_FUNDAMENTAL 7
+#define VFIO_EEH_PE_CONFIGURE 8
+#define VFIO_EEH_PE_OP _IO(VFIO_TYPE, VFIO_BASE + 21)
+#endif
@@ -34,15 +34,16 @@
#ifndef EAL_VFIO_H_
#define EAL_VFIO_H_
-/*
- * determine if VFIO is present on the system
- */
#ifdef RTE_EAL_VFIO
#include <linux/version.h>
-#if LINUX_VERSION_CODE >= KERNEL_VERSION(3, 6, 0)
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(3, 6, 0)
+#include "compat_vfio.h"
+#else
#include <linux/vfio.h>
+#endif /* kernel version */
-#if LINUX_VERSION_CODE < KERNEL_VERSION(3, 10, 0)
+#ifndef PCI_MSIX_TABLE_BIR
#define RTE_PCI_MSIX_TABLE_BIR 0x7
#define RTE_PCI_MSIX_TABLE_OFFSET 0xfffffff8
#define RTE_PCI_MSIX_FLAGS_QSIZE 0x07ff
@@ -53,7 +54,7 @@
#endif
#define VFIO_PRESENT
-#endif /* kernel version */
+
#endif /* RTE_EAL_VFIO */
#endif /* EAL_VFIO_H_ */