Message ID | 1436395131-11205-1-git-send-email-stephen@networkplumber.org (mailing list archive) |
---|---|
State | RFC, archived |
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 6461A5A57; Thu, 9 Jul 2015 00:38:45 +0200 (CEST) Received: from mail-pa0-f45.google.com (mail-pa0-f45.google.com [209.85.220.45]) by dpdk.org (Postfix) with ESMTP id 933C4952 for <dev@dpdk.org>; Thu, 9 Jul 2015 00:38:43 +0200 (CEST) Received: by pactm7 with SMTP id tm7so138822661pac.2 for <dev@dpdk.org>; Wed, 08 Jul 2015 15:38:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Rtbnp6DqDw6+2ouTo6o4LWa+l4EHDU65KLSWwhUjxgA=; b=hDVuG5bDCOKV7dvVhqTwY9h+I5m9EBpf0TSNigcgt5WY8QN0cfQuVddd5NXc4TQn5d ddB5pvlwnGNsLCfw58BQOstQ/vV5J9w/XOrM8uh+oc1rrcNrh03dfZw1rr2+R9hw0m3w WMCLjrEWWpVS7wfbTrpnFLdJtD4qlmg5u200WuIgembC9TzX3n8fxUe6dGJAmDFwk+5g ZpZAkSsC3i8KW8S6YQOjKKar7u34i5dilvLGRznyD/uzO29LTeSIcB0ygR0poq7hf7fs VgL1g5zyprV2MpL54qEmjO4/ghzy8lqgq7RrHBtwp/4M4EptOYhYnTihd6q3mM5WO93D jhZQ== X-Gm-Message-State: ALoCoQkMqR7ToV0jLsZhi0W2rXRjAinkA/NPAFCELGSOzPjpKxNXEB55bNyKfwYAd0ifs0FwvCLo X-Received: by 10.66.62.133 with SMTP id y5mr25058562par.8.1436395122872; Wed, 08 Jul 2015 15:38:42 -0700 (PDT) Received: from urahara.home.lan (static-50-53-82-155.bvtn.or.frontiernet.net. [50.53.82.155]) by smtp.gmail.com with ESMTPSA id pc5sm3619024pbc.15.2015.07.08.15.38.41 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 08 Jul 2015 15:38:42 -0700 (PDT) From: Stephen Hemminger <stephen@networkplumber.org> To: dev@dpdk.org Date: Wed, 8 Jul 2015 15:38:51 -0700 Message-Id: <1436395131-11205-1-git-send-email-stephen@networkplumber.org> X-Mailer: git-send-email 2.1.4 Subject: [dpdk-dev] [RFC] vfio: only map regions VFIO supports X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Commit Message
Stephen Hemminger
July 8, 2015, 10:38 p.m. UTC
The FM10K driver has 3 BARS numbered 0, 4, and 8. But the kernel
VFIO driver only allows mapping 0-5 anything bigger than that will
return -EINVAL (see kernel source vfio_pci.c:vfio_pci_mmap).
The workaround is to limit the DPDK EAL VFIO support only map
the regions that will work. The FM10K driver is not using BAR8
in current code anyway.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
Comments
Hi, Stephen This patch does not work for fm10k with vfio, see error below: EAL: PCI device 0000:84:00.0 on NUMA socket 1 EAL: probe driver: 8086:15a4 rte_pmd_fm10k EAL: PCI memory mapped at 0x7f1980000000 EAL: Trying to map BAR 2 that contains the MSI-X table. Trying offsets: 0000:0000, 1000:1000 EAL: PCI memory mapped at 0x7f1980401000 EAL: pci_map_resource(): cannot mmap(105, 0x7f1980402000, 0x4000000, 0x0): Invalid argument (0xffffffffffffffff) EAL: 0000:84:00.0 mapping BAR4 failed: Invalid argument EAL: Error - exiting with code: 1 Cause: Requested device 0000:84:00.0 cannot be used Thanks, Michael On 7/9/2015 6:39 AM, Stephen Hemminger wrote: > The FM10K driver has 3 BARS numbered 0, 4, and 8. But the kernel > VFIO driver only allows mapping 0-5 anything bigger than that will > return -EINVAL (see kernel source vfio_pci.c:vfio_pci_mmap). > > The workaround is to limit the DPDK EAL VFIO support only map > the regions that will work. The FM10K driver is not using BAR8 > in current code anyway. > > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> > --- > lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c > index 426953a..e269e75 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c > @@ -549,7 +549,7 @@ pci_vfio_map_resource(struct rte_pci_device *dev) > int iommu_group_no; > char pci_addr[PATH_MAX] = {0}; > struct rte_pci_addr *loc = &dev->addr; > - int i, ret, msix_bar; > + int i, ret, msix_bar, nmaps; > struct mapped_pci_resource *vfio_res = NULL; > struct mapped_pci_res_list *vfio_res_list = RTE_TAILQ_CAST(rte_vfio_tailq.head, mapped_pci_res_list); > > @@ -724,7 +724,9 @@ pci_vfio_map_resource(struct rte_pci_device *dev) > /* map BARs */ > maps = vfio_res->maps; > > - for (i = 0; i < (int) vfio_res->nb_maps; i++) { > + /* VFIO supports limited (0-5) maps */ > + nmaps = RTE_MIN(vfio_res->nb_maps, VFIO_PCI_ROM_REGION_INDEX - 1); > + for (i = 0; i < nmaps; i++) { > struct vfio_region_info reg = { .argsz = sizeof(reg) }; > void *bar_addr; > struct memreg {
On Fri, 10 Jul 2015 07:54:10 +0000 "Qiu, Michael" <michael.qiu@intel.com> wrote: > Hi, Stephen > > This patch does not work for fm10k with vfio, see error below: > > EAL: PCI device 0000:84:00.0 on NUMA socket 1 > EAL: probe driver: 8086:15a4 rte_pmd_fm10k > EAL: PCI memory mapped at 0x7f1980000000 > EAL: Trying to map BAR 2 that contains the MSI-X table. Trying offsets: > 0000:0000, 1000:1000 > EAL: PCI memory mapped at 0x7f1980401000 > EAL: pci_map_resource(): cannot mmap(105, 0x7f1980402000, 0x4000000, > 0x0): Invalid argument (0xffffffffffffffff) > EAL: 0000:84:00.0 mapping BAR4 failed: Invalid argument > EAL: Error - exiting with code: 1 > Cause: Requested device 0000:84:00.0 cannot be used Yes. The patch doesn't solve the problem (but might be needed in some future weird hw).
Hi, Stephen I have found out the root cause of this bug, and generate a patch. Will send out later after :) Thanks, Michael On 7/11/2015 1:24 AM, Stephen Hemminger wrote: > On Fri, 10 Jul 2015 07:54:10 +0000 > "Qiu, Michael" <michael.qiu@intel.com> wrote: > >> Hi, Stephen >> >> This patch does not work for fm10k with vfio, see error below: >> >> EAL: PCI device 0000:84:00.0 on NUMA socket 1 >> EAL: probe driver: 8086:15a4 rte_pmd_fm10k >> EAL: PCI memory mapped at 0x7f1980000000 >> EAL: Trying to map BAR 2 that contains the MSI-X table. Trying offsets: >> 0000:0000, 1000:1000 >> EAL: PCI memory mapped at 0x7f1980401000 >> EAL: pci_map_resource(): cannot mmap(105, 0x7f1980402000, 0x4000000, >> 0x0): Invalid argument (0xffffffffffffffff) >> EAL: 0000:84:00.0 mapping BAR4 failed: Invalid argument >> EAL: Error - exiting with code: 1 >> Cause: Requested device 0000:84:00.0 cannot be used > Yes. The patch doesn't solve the problem (but might be needed in some future weird hw). > >
Hi, all While I check the maillist, I found a patch: [dpdk-dev] [PATCH v3] vfio: Fix overflow while assigning vfio BAR region offset and size This patch has fixed this issue :), and will be merged. This patch is same with mine. So I will not post the patch again. Thanks, Michael On 7/11/2015 1:24 AM, Stephen Hemminger wrote: > On Fri, 10 Jul 2015 07:54:10 +0000 > "Qiu, Michael" <michael.qiu@intel.com> wrote: > >> Hi, Stephen >> >> This patch does not work for fm10k with vfio, see error below: >> >> EAL: PCI device 0000:84:00.0 on NUMA socket 1 >> EAL: probe driver: 8086:15a4 rte_pmd_fm10k >> EAL: PCI memory mapped at 0x7f1980000000 >> EAL: Trying to map BAR 2 that contains the MSI-X table. Trying offsets: >> 0000:0000, 1000:1000 >> EAL: PCI memory mapped at 0x7f1980401000 >> EAL: pci_map_resource(): cannot mmap(105, 0x7f1980402000, 0x4000000, >> 0x0): Invalid argument (0xffffffffffffffff) >> EAL: 0000:84:00.0 mapping BAR4 failed: Invalid argument >> EAL: Error - exiting with code: 1 >> Cause: Requested device 0000:84:00.0 cannot be used > Yes. The patch doesn't solve the problem (but might be needed in some future weird hw). > >
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c index 426953a..e269e75 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c @@ -549,7 +549,7 @@ pci_vfio_map_resource(struct rte_pci_device *dev) int iommu_group_no; char pci_addr[PATH_MAX] = {0}; struct rte_pci_addr *loc = &dev->addr; - int i, ret, msix_bar; + int i, ret, msix_bar, nmaps; struct mapped_pci_resource *vfio_res = NULL; struct mapped_pci_res_list *vfio_res_list = RTE_TAILQ_CAST(rte_vfio_tailq.head, mapped_pci_res_list); @@ -724,7 +724,9 @@ pci_vfio_map_resource(struct rte_pci_device *dev) /* map BARs */ maps = vfio_res->maps; - for (i = 0; i < (int) vfio_res->nb_maps; i++) { + /* VFIO supports limited (0-5) maps */ + nmaps = RTE_MIN(vfio_res->nb_maps, VFIO_PCI_ROM_REGION_INDEX - 1); + for (i = 0; i < nmaps; i++) { struct vfio_region_info reg = { .argsz = sizeof(reg) }; void *bar_addr; struct memreg {