Message ID | 1615397790-16169-1-git-send-email-huawei.xhw@alibaba-inc.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BAAA1A056A; Wed, 10 Mar 2021 18:36:51 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A31DF22A38C; Wed, 10 Mar 2021 18:36:51 +0100 (CET) Received: from out0-134.mail.aliyun.com (out0-134.mail.aliyun.com [140.205.0.134]) by mails.dpdk.org (Postfix) with ESMTP id 67CB522A38A for <dev@dpdk.org>; Wed, 10 Mar 2021 18:36:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alibaba-inc.com; s=default; t=1615397807; h=From:To:Subject:Date:Message-Id; bh=Y3l1uevJKdQF1nach0wXJOqEZ+sgz9Xmk/Acv1XDHaY=; b=J/QaSegK/cIyUf9fX6fPs+kB+zRYWgWacnreNaMTtmMW5sYWfvz/PWttjz/SEUAadBVY9cJ4c5VMC4SmjOKTQnfKiCxp12T7ZFf51nSJsaMvZKFPkry4Z3ribuSGVLOk4sltEfXTkkDxJiqMaWAtj4xRSnXBw/pRPrwajI6B6dI= X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R191e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=ay29a033018047192; MF=huawei.xhw@alibaba-inc.com; NM=1; PH=DS; RN=8; SR=0; TI=SMTPD_---.Jj2m8YS_1615397793; Received: from rs3a10040.et2sqa.z1.et2sqa.tbsite.net(mailfrom:huawei.xhw@alibaba-inc.com fp:SMTPD_---.Jj2m8YS_1615397793) by smtp.aliyun-inc.com(127.0.0.1); Thu, 11 Mar 2021 01:36:36 +0800 From: " =?utf-8?b?6LCi5Y2O5LyfKOatpOaXtuatpOWIu++8iQ==?= " <huawei.xhw@alibaba-inc.com> To: david.marchand@redhat.com, maxime.coquelin@redhat.com, ferruh.yigit@intel.com Cc: <dev@dpdk.org>, <anatoly.burakov@intel.com>, <xuemingl@nvidia.com>, <grive@u256.net>, " =?utf-8?b?6LCi5Y2O5LyfKOatpOaXtuatpOWIu++8iQ==?= " <huawei.xhw@alibaba-inc.com> Date: Thu, 11 Mar 2021 01:36:28 +0800 Message-Id: <1615397790-16169-1-git-send-email-huawei.xhw@alibaba-inc.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1614797225-114594-1-git-send-email-huawei.xhw@alibaba-inc.com> References: <1614797225-114594-1-git-send-email-huawei.xhw@alibaba-inc.com> Subject: [dpdk-dev] [PATCH v11 0/2] support both PIO and MMIO BAR for legacy virito device X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series |
support both PIO and MMIO BAR for legacy virito device
|
|
Message
谢华伟(此时此刻)
March 10, 2021, 5:36 p.m. UTC
virtio PMD assumes legacy device only supports PIO(port-mapped) BAR resource. This is wrong. As we need to create lots of devices, adn PIO resource on x86 is very limited, we expose MMIO(memory-mapped I/O) BAR. Kernel supports both PIO and MMIO BAR for legacy virtio-pci device, and for all other pci devices. This patchset handles different type of BAR in the similar way. In previous implementation, under igb_uio driver we get PIO address from igb_uio sysfs entry; with uio_pci_generic, we get PIO address from /proc/ioports for x86, and for other ARCHs, we get PIO address from standard PCI sysfs entry. For PIO/MMIO RW, there is different path for different drivers and arch. All of the above is too much twisted. This patchset unifies the way to get both PIO and MMIO address for different driver and ARCHs, all from standard resource attr under pci sysfs. This is most generic. We distinguish PIO and MMIO by their address range like how kernel does. It is ugly but works. v2 changes: - add more explanation in the commit message v3 changes: - fix patch format issues v4 changes: - fixes for RTE_KDRV_UIO_GENERIC -> RTE_PCI_KDRV_UIO_GENERIC v5 changes: - split into three seperate patches v6 changes: - change to DEBUG level for IO bar detection in pci_uio_ioport_map - rework the code in iobar branch - fixes commit message format issue - temporarily remove the 3rd patch for vfio path, leave it for future discusssion - rework against virtio_pmd_rework_v2 v7 changes: - fix compilation issues of in/out instruction on non X86 archs v8 changes: - change the word fix to refactor in patch 1's commit message v9 changes: - keep pause version in in/out instructions v10 changes: - trival fixes in commit message, like > 75 chars v11 changes: - commit message fix and change huawei xie (2): bus/pci: use PCI standard sysfs entry to get PIO address bus/pci: support MMIO in PCI ioport accessors drivers/bus/pci/linux/pci.c | 81 ---------------- drivers/bus/pci/linux/pci_uio.c | 202 +++++++++++++++++++++++++++++----------- 2 files changed, 150 insertions(+), 133 deletions(-)
Comments
Tested-by: Wang, Yinan <yinan.wang@intel.com> Tested PVP case with virtio PMD assumes legacy device, VM with below kernel LTS versions, all pass except vfio-pci test blocked with kernel v4.4. 5.10.0-051000-generic virtio-pmd test with vfio-pci/ igb_uio/uio_pci_generic all pass 5.4.0-050400-generic virtio-pmd test with vfio-pci/ igb_uio/uio_pci_generic all pass 4.19.179-0419179-generic virtio-pmd test with vfio-pci/ igb_uio/uio_pci_generic all pass 4.9.260-0409260-generic virtio-pmd test with vfio-pci/ igb_uio/uio_pci_generic all pass 4.4.260-0404260-generic virtio-pmd test with igb_uio/uio_pci_generic can pass ; vfio-pci blocked as fail to bind vfio-pci to virtio-pmd Error info: root@vmubuntu2004:~/dpdk/usertools# ./dpdk-devbind.py -b vfio-pci 00:04.0 lspci: Unable to load libkmod resources: error -12 root@vmubuntu2004:~/dpdk/usertools# dmesg [ 161.553493] VFIO - User Level meta-driver version: 0.3 [ 179.430529] vfio-pci: probe of 0000:00:04.0 failed with error -22 BR, Yinan > -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of ???(????) > Sent: 2021年3月11日 1:36 > To: david.marchand@redhat.com; maxime.coquelin@redhat.com; Yigit, > Ferruh <ferruh.yigit@intel.com> > Cc: dev@dpdk.org; Burakov, Anatoly <anatoly.burakov@intel.com>; > xuemingl@nvidia.com; grive@u256.net; 谢华伟(此时此刻) > <huawei.xhw@alibaba-inc.com> > Subject: [dpdk-dev] [PATCH v11 0/2] support both PIO and MMIO BAR for > legacy virito device > > virtio PMD assumes legacy device only supports PIO(port-mapped) BAR > resource. This is wrong. As we need to create lots of devices, adn PIO > resource on x86 is very limited, we expose MMIO(memory-mapped I/O) BAR. > > Kernel supports both PIO and MMIO BAR for legacy virtio-pci device, and > for all other pci devices. This patchset handles different type of BAR in > the similar way. > > In previous implementation, under igb_uio driver we get PIO address from > igb_uio sysfs entry; with uio_pci_generic, we get PIO address from > /proc/ioports for x86, and for other ARCHs, we get PIO address from > standard PCI sysfs entry. For PIO/MMIO RW, there is different path for > different drivers and arch. > > All of the above is too much twisted. This patchset unifies the way to get > both PIO and MMIO address for different driver and ARCHs, all from standard > resource attr under pci sysfs. This is most generic. > > We distinguish PIO and MMIO by their address range like how kernel does. > It is ugly but works. > > v2 changes: > - add more explanation in the commit message > > v3 changes: > - fix patch format issues > > v4 changes: > - fixes for RTE_KDRV_UIO_GENERIC -> RTE_PCI_KDRV_UIO_GENERIC > > v5 changes: > - split into three seperate patches > > v6 changes: > - change to DEBUG level for IO bar detection in pci_uio_ioport_map > - rework the code in iobar branch > - fixes commit message format issue > - temporarily remove the 3rd patch for vfio path, leave it for future > discusssion > - rework against virtio_pmd_rework_v2 > > v7 changes: > - fix compilation issues of in/out instruction on non X86 archs > > v8 changes: > - change the word fix to refactor in patch 1's commit message > > v9 changes: > - keep pause version in in/out instructions > > v10 changes: > - trival fixes in commit message, like > 75 chars > > v11 changes: > - commit message fix and change > > huawei xie (2): > bus/pci: use PCI standard sysfs entry to get PIO address > bus/pci: support MMIO in PCI ioport accessors > > drivers/bus/pci/linux/pci.c | 81 ---------------- > drivers/bus/pci/linux/pci_uio.c | 202 +++++++++++++++++++++++++++++-- > --------- > 2 files changed, 150 insertions(+), 133 deletions(-) > > -- > 1.8.3.1
Hello, On Thu, Mar 11, 2021 at 12:55 PM Wang, Yinan <yinan.wang@intel.com> wrote: > > Tested-by: Wang, Yinan <yinan.wang@intel.com> > > Tested PVP case with virtio PMD assumes legacy device, VM with below kernel LTS versions, all pass except vfio-pci test blocked with kernel v4.4. > 5.10.0-051000-generic virtio-pmd test with vfio-pci/ igb_uio/uio_pci_generic all pass > 5.4.0-050400-generic virtio-pmd test with vfio-pci/ igb_uio/uio_pci_generic all pass > 4.19.179-0419179-generic virtio-pmd test with vfio-pci/ igb_uio/uio_pci_generic all pass > 4.9.260-0409260-generic virtio-pmd test with vfio-pci/ igb_uio/uio_pci_generic all pass > 4.4.260-0404260-generic virtio-pmd test with igb_uio/uio_pci_generic can pass ; vfio-pci blocked as fail to bind vfio-pci to virtio-pmd > > Error info: > root@vmubuntu2004:~/dpdk/usertools# ./dpdk-devbind.py -b vfio-pci 00:04.0 > lspci: Unable to load libkmod resources: error -12 > > root@vmubuntu2004:~/dpdk/usertools# dmesg > [ 161.553493] VFIO - User Level meta-driver version: 0.3 > [ 179.430529] vfio-pci: probe of 0000:00:04.0 failed with error -22 The vanilla stable 4.4 kernel does not support "No-IOMMU mode" for vfio. ae5515d66362 - Revert: "vfio: Include No-IOMMU mode" (5 years ago) <Alex Williamson> So this behavior is expected if you were testing with no iommu in your virtual machine. Thanks for the tests!
On Wed, Mar 10, 2021 at 6:37 PM 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com> wrote: > > virtio PMD assumes legacy device only supports PIO(port-mapped) BAR > resource. This is wrong. As we need to create lots of devices, adn PIO > resource on x86 is very limited, we expose MMIO(memory-mapped I/O) BAR. > > Kernel supports both PIO and MMIO BAR for legacy virtio-pci device, and > for all other pci devices. This patchset handles different type of BAR in > the similar way. > > In previous implementation, under igb_uio driver we get PIO address from > igb_uio sysfs entry; with uio_pci_generic, we get PIO address from > /proc/ioports for x86, and for other ARCHs, we get PIO address from > standard PCI sysfs entry. For PIO/MMIO RW, there is different path for > different drivers and arch. > > All of the above is too much twisted. This patchset unifies the way to get > both PIO and MMIO address for different driver and ARCHs, all from standard > resource attr under pci sysfs. This is most generic. > > We distinguish PIO and MMIO by their address range like how kernel does. > It is ugly but works. > > v2 changes: > - add more explanation in the commit message > > v3 changes: > - fix patch format issues > > v4 changes: > - fixes for RTE_KDRV_UIO_GENERIC -> RTE_PCI_KDRV_UIO_GENERIC > > v5 changes: > - split into three seperate patches > > v6 changes: > - change to DEBUG level for IO bar detection in pci_uio_ioport_map > - rework the code in iobar branch > - fixes commit message format issue > - temporarily remove the 3rd patch for vfio path, leave it for future discusssion > - rework against virtio_pmd_rework_v2 > > v7 changes: > - fix compilation issues of in/out instruction on non X86 archs > > v8 changes: > - change the word fix to refactor in patch 1's commit message > > v9 changes: > - keep pause version in in/out instructions > > v10 changes: > - trival fixes in commit message, like > 75 chars > > v11 changes: > - commit message fix and change > Aligned Sob and Author to fix the last checkpatch warning. Series applied to the main branch. Thanks Huawei and thanks too to reviewers/testers.
On 2021/3/15 22:16, David Marchand wrote: >> v10 changes: >> - trival fixes in commit message, like > 75 chars >> >> v11 changes: >> - commit message fix and change >> > Aligned Sob and Author to fix the last checkpatch warning. > > Series applied to the main branch. > Thanks Huawei and thanks too to reviewers/testers. Thanks!