Cover Detail
Show a cover letter.
GET /api/covers/126213/?format=api
{ "id": 126213, "url": "http://patches.dpdk.org/api/covers/126213/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/cover/20230418053012.10667-1-chenbo.xia@intel.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20230418053012.10667-1-chenbo.xia@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20230418053012.10667-1-chenbo.xia@intel.com", "date": "2023-04-18T05:30:08", "name": "[RFC,0/4] Support VFIO sparse mmap in PCI bus", "submitter": { "id": 1276, "url": "http://patches.dpdk.org/api/people/1276/?format=api", "name": "Chenbo Xia", "email": "chenbo.xia@intel.com" }, "mbox": "http://patches.dpdk.org/project/dpdk/cover/20230418053012.10667-1-chenbo.xia@intel.com/mbox/", "series": [ { "id": 27749, "url": "http://patches.dpdk.org/api/series/27749/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=27749", "date": "2023-04-18T05:30:08", "name": "Support VFIO sparse mmap in PCI bus", "version": 1, "mbox": "http://patches.dpdk.org/series/27749/mbox/" } ], "comments": "http://patches.dpdk.org/api/covers/126213/comments/", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 1564442977;\n\tTue, 18 Apr 2023 07:49:21 +0200 (CEST)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 04AED410EA;\n\tTue, 18 Apr 2023 07:49:21 +0200 (CEST)", "from mga06.intel.com (mga06b.intel.com [134.134.136.31])\n by mails.dpdk.org (Postfix) with ESMTP id 9A5E640EDF\n for <dev@dpdk.org>; Tue, 18 Apr 2023 07:49:19 +0200 (CEST)", "from orsmga006.jf.intel.com ([10.7.209.51])\n by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 17 Apr 2023 22:49:18 -0700", "from npg-dpdk-virtio-xiachenbo-nw.sh.intel.com ([10.67.119.79])\n by orsmga006.jf.intel.com with ESMTP; 17 Apr 2023 22:49:17 -0700" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple;\n d=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n t=1681796959; x=1713332959;\n h=from:to:cc:subject:date:message-id;\n bh=lUJqsMViEBycgWQ0JYEZygBmWwKCozcIZPfVLT17r8Y=;\n b=E1yXsW2El0/HungtLBopic3LQ3Td4ogSUia+5xK7sPnBSSe+kJqMhm+W\n 2BGpdyawxyCrKBN+8zLtAfPM71/KN5XOOh99IPXk9bHwTztO8cN/DL0E+\n FX/ZdxOqxU9mQwdOwAxMi6VJsFoUUnkB9lifnlicjPyX/IsmhoyaKXxle\n ScLP1osSc2RkBQGSz+DtU6V9qti40OVWKt5RWH0ij9BRhOCZ+AQ3noARg\n zkl2aipkPrc78oeBQvYN0Bc4RCVCERZhz6WYufao9tsBRfvSisUeYakgh\n cZ8LshtdHp1PiDn7LoChZHRpQ12yHHl4+9JRalD7dbqHwMo3hUbTfw6us A==;", "X-IronPort-AV": [ "E=McAfee;i=\"6600,9927,10683\"; a=\"407978627\"", "E=Sophos;i=\"5.99,206,1677571200\"; d=\"scan'208\";a=\"407978627\"", "E=McAfee;i=\"6600,9927,10683\"; a=\"668373188\"", "E=Sophos;i=\"5.99,206,1677571200\"; d=\"scan'208\";a=\"668373188\"" ], "X-ExtLoop1": "1", "From": "Chenbo Xia <chenbo.xia@intel.com>", "To": "dev@dpdk.org", "Cc": "skori@marvell.com", "Subject": "[RFC 0/4] Support VFIO sparse mmap in PCI bus", "Date": "Tue, 18 Apr 2023 13:30:08 +0800", "Message-Id": "<20230418053012.10667-1-chenbo.xia@intel.com>", "X-Mailer": "git-send-email 2.17.1", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "This series introduces a VFIO standard capability, called sparse\nmmap to PCI bus. In linux kernel, it's defined as\nVFIO_REGION_INFO_CAP_SPARSE_MMAP. Sparse mmap means instead of\nmmap whole BAR region into DPDK process, only mmap part of the\nBAR region after getting sparse mmap information from kernel.\nFor the rest of BAR region that is not mmap-ed, DPDK process\ncan use pread/pwrite system calls to access. Sparse mmap is\nuseful when kernel does not want userspace to mmap whole BAR\nregion, or kernel wants to control over access to specific BAR\nregion. Vendors can choose to enable this feature or not for\ntheir devices in their specific kernel modules. \n\nIn this patchset:\n\nPatch 1-3 is mainly for introducing BAR access APIs so that\ndriver could use them to access specific BAR using pread/pwrite\nsystem calls when part of the BAR is not mmap-able.\n\nPatch 4 adds the VFIO sparse mmap support finally. A question\nis for all sparse mmap regions, should they be mapped to a\ncontinuous virtual address region that follows device-specific\nBAR layout or not. In theory, there could be three options to\nsupport this feature.\n\nOption 1: Map sparse mmap regions independently\n======================================================\nIn this approach, we mmap each sparse mmap region one by one\nand each region could be located anywhere in process address\nspace. But accessing the mmaped BAR will not be as easy as\n'bar_base_address + bar_offset', driver needs to check the\nsparse mmap information to access specific BAR register. \n\nPatch 4 in this patchset adopts this option. Driver API change\nis introduced in bus_pci_driver.h. Corresponding changes in\nall drivers are also done and currently I am assuming drivers\ndo not support this feature so they will not check the\n'is_sparse' flag but assumes it to be false. Note that it will\nnot break any driver and each vendor can add related logic when\nthey start to support this feature. This is only because I don't\nwant to introduce complexity to drivers that do not want to\nsupport this feature.\n\nOption 2: Map sparse mmap regions based on device-specific BAR layout \n======================================================================\nIn this approach, the sparse mmap regions are mapped to continuous\nvirtual address region that follows device-specific BAR layout.\nFor example, the BAR size is 0x4000 and only 0-0x1000 (sparse mmap\nregion #1) and 0x3000-0x4000 (sparse mmap region #2) could be\nmmaped. Region #1 will be mapped at 'base_addr' and region #2\nwill be mapped at 'base_addr + 0x3000'. The good thing is if\nwe implement like this, driver can still access all BAR registers\nusing 'bar_base_address + bar_offset' way and we don't need\nto introduce any driver API change. But the address space\nrange 'base_addr + 0x1000' to 'base_addr + 0x3000' may need to\nbe reserved so it could result in waste of address space or memory\n(when we use MAP_ANONYMOUS and MAP_PRIVATE flag to reserve this\nrange). Meanwhile, driver needs to know which part of BAR is\nmmaped (this is possible since the range is defined by vendor's\nspecific kernel module).\n\nOption 3: Support both option 1 & 2 \n===================================\nWe could define a driver flag to let driver choose which way it\nperfers since either option has its own Pros & Cons.\n\nPlease share your comments, Thanks!\n\n\nChenbo Xia (4):\n bus/pci: introduce an internal representation of PCI device\n bus/pci: avoid depending on private value in kernel source\n bus/pci: introduce helper for MMIO read and write\n bus/pci: add VFIO sparse mmap support\n\n drivers/baseband/acc/rte_acc100_pmd.c | 6 +-\n drivers/baseband/acc/rte_vrb_pmd.c | 6 +-\n .../fpga_5gnr_fec/rte_fpga_5gnr_fec.c | 6 +-\n drivers/baseband/fpga_lte_fec/fpga_lte_fec.c | 6 +-\n drivers/bus/pci/bsd/pci.c | 43 +-\n drivers/bus/pci/bus_pci_driver.h | 24 +-\n drivers/bus/pci/linux/pci.c | 91 +++-\n drivers/bus/pci/linux/pci_init.h | 14 +-\n drivers/bus/pci/linux/pci_uio.c | 34 +-\n drivers/bus/pci/linux/pci_vfio.c | 445 ++++++++++++++----\n drivers/bus/pci/pci_common.c | 57 ++-\n drivers/bus/pci/pci_common_uio.c | 12 +-\n drivers/bus/pci/private.h | 25 +-\n drivers/bus/pci/rte_bus_pci.h | 48 ++\n drivers/bus/pci/version.map | 3 +\n drivers/common/cnxk/roc_dev.c | 4 +-\n drivers/common/cnxk/roc_dpi.c | 2 +-\n drivers/common/cnxk/roc_ml.c | 22 +-\n drivers/common/qat/dev/qat_dev_gen1.c | 2 +-\n drivers/common/qat/dev/qat_dev_gen4.c | 4 +-\n drivers/common/sfc_efx/sfc_efx.c | 2 +-\n drivers/compress/octeontx/otx_zip.c | 4 +-\n drivers/crypto/ccp/ccp_dev.c | 4 +-\n drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 2 +-\n drivers/crypto/nitrox/nitrox_device.c | 4 +-\n drivers/crypto/octeontx/otx_cryptodev_ops.c | 6 +-\n drivers/crypto/virtio/virtio_pci.c | 6 +-\n drivers/dma/cnxk/cnxk_dmadev.c | 2 +-\n drivers/dma/hisilicon/hisi_dmadev.c | 6 +-\n drivers/dma/idxd/idxd_pci.c | 4 +-\n drivers/dma/ioat/ioat_dmadev.c | 2 +-\n drivers/event/dlb2/pf/dlb2_main.c | 16 +-\n drivers/event/octeontx/ssovf_probe.c | 38 +-\n drivers/event/octeontx/timvf_probe.c | 18 +-\n drivers/event/skeleton/skeleton_eventdev.c | 2 +-\n drivers/mempool/octeontx/octeontx_fpavf.c | 6 +-\n drivers/net/ark/ark_ethdev.c | 4 +-\n drivers/net/atlantic/atl_ethdev.c | 2 +-\n drivers/net/avp/avp_ethdev.c | 20 +-\n drivers/net/axgbe/axgbe_ethdev.c | 4 +-\n drivers/net/bnx2x/bnx2x_ethdev.c | 6 +-\n drivers/net/bnxt/bnxt_ethdev.c | 8 +-\n drivers/net/cpfl/cpfl_ethdev.c | 4 +-\n drivers/net/cxgbe/cxgbe_ethdev.c | 2 +-\n drivers/net/cxgbe/cxgbe_main.c | 2 +-\n drivers/net/cxgbe/cxgbevf_ethdev.c | 2 +-\n drivers/net/cxgbe/cxgbevf_main.c | 2 +-\n drivers/net/e1000/em_ethdev.c | 4 +-\n drivers/net/e1000/igb_ethdev.c | 4 +-\n drivers/net/ena/ena_ethdev.c | 4 +-\n drivers/net/enetc/enetc_ethdev.c | 2 +-\n drivers/net/enic/enic_main.c | 4 +-\n drivers/net/fm10k/fm10k_ethdev.c | 2 +-\n drivers/net/gve/gve_ethdev.c | 4 +-\n drivers/net/hinic/base/hinic_pmd_hwif.c | 14 +-\n drivers/net/hns3/hns3_ethdev.c | 2 +-\n drivers/net/hns3/hns3_ethdev_vf.c | 2 +-\n drivers/net/hns3/hns3_rxtx.c | 4 +-\n drivers/net/i40e/i40e_ethdev.c | 2 +-\n drivers/net/iavf/iavf_ethdev.c | 2 +-\n drivers/net/ice/ice_dcf.c | 2 +-\n drivers/net/ice/ice_ethdev.c | 2 +-\n drivers/net/idpf/idpf_ethdev.c | 4 +-\n drivers/net/igc/igc_ethdev.c | 2 +-\n drivers/net/ionic/ionic_dev_pci.c | 2 +-\n drivers/net/ixgbe/ixgbe_ethdev.c | 4 +-\n drivers/net/liquidio/lio_ethdev.c | 4 +-\n drivers/net/nfp/nfp_ethdev.c | 2 +-\n drivers/net/nfp/nfp_ethdev_vf.c | 6 +-\n drivers/net/nfp/nfpcore/nfp_cpp_pcie_ops.c | 4 +-\n drivers/net/ngbe/ngbe_ethdev.c | 2 +-\n drivers/net/octeon_ep/otx_ep_ethdev.c | 2 +-\n drivers/net/octeontx/base/octeontx_pkivf.c | 6 +-\n drivers/net/octeontx/base/octeontx_pkovf.c | 12 +-\n drivers/net/qede/qede_main.c | 6 +-\n drivers/net/sfc/sfc.c | 2 +-\n drivers/net/thunderx/nicvf_ethdev.c | 2 +-\n drivers/net/txgbe/txgbe_ethdev.c | 2 +-\n drivers/net/txgbe/txgbe_ethdev_vf.c | 2 +-\n drivers/net/virtio/virtio_pci.c | 6 +-\n drivers/net/vmxnet3/vmxnet3_ethdev.c | 4 +-\n drivers/raw/cnxk_bphy/cnxk_bphy.c | 10 +-\n drivers/raw/cnxk_bphy/cnxk_bphy_cgx.c | 6 +-\n drivers/raw/ifpga/afu_pmd_n3000.c | 4 +-\n drivers/raw/ifpga/ifpga_rawdev.c | 6 +-\n drivers/raw/ntb/ntb_hw_intel.c | 8 +-\n drivers/vdpa/ifc/ifcvf_vdpa.c | 6 +-\n drivers/vdpa/sfc/sfc_vdpa_hw.c | 2 +-\n drivers/vdpa/sfc/sfc_vdpa_ops.c | 2 +-\n lib/eal/include/rte_vfio.h | 1 -\n 90 files changed, 853 insertions(+), 352 deletions(-)" }