Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/66553/?format=api
{ "id": 66553, "url": "http://patches.dpdk.org/api/patches/66553/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/patch/158396395214.5601.11207416598267070486.stgit@gimli.home/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<158396395214.5601.11207416598267070486.stgit@gimli.home>", "list_archive_url": "https://inbox.dpdk.org/dev/158396395214.5601.11207416598267070486.stgit@gimli.home", "date": "2020-03-11T21:59:12", "name": "[v3,5/7] vfio/pci: Add sriov_configure support", "commit_ref": null, "pull_url": null, "state": "not-applicable", "archived": true, "hash": "312fca7992ad3a6b60c24f6b5b32f9922d252a05", "submitter": { "id": 391, "url": "http://patches.dpdk.org/api/people/391/?format=api", "name": "Alex Williamson", "email": "alex.williamson@redhat.com" }, "delegate": null, "mbox": "http://patches.dpdk.org/project/dpdk/patch/158396395214.5601.11207416598267070486.stgit@gimli.home/mbox/", "series": [ { "id": 8887, "url": "http://patches.dpdk.org/api/series/8887/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=8887", "date": "2020-03-11T21:58:25", "name": "vfio/pci: SR-IOV support", "version": 3, "mbox": "http://patches.dpdk.org/series/8887/mbox/" } ], "comments": "http://patches.dpdk.org/api/patches/66553/comments/", "check": "fail", "checks": "http://patches.dpdk.org/api/patches/66553/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from dpdk.org (dpdk.org [92.243.14.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id D7543A056D;\n\tWed, 11 Mar 2020 22:59:27 +0100 (CET)", "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id A84A61C037;\n\tWed, 11 Mar 2020 22:59:22 +0100 (CET)", "from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com\n [205.139.110.120]) by dpdk.org (Postfix) with ESMTP id C7B481BF7F\n for <dev@dpdk.org>; Wed, 11 Mar 2020 22:59:21 +0100 (CET)", "from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com\n [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id\n us-mta-152-1uQDeSaMNOOKR958PrCnNg-1; Wed, 11 Mar 2020 17:59:17 -0400", "from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com\n [10.5.11.23])\n (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n (No client certificate requested)\n by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C6BEF18A5500;\n Wed, 11 Mar 2020 21:59:15 +0000 (UTC)", "from gimli.home (ovpn-116-28.phx2.redhat.com [10.3.116.28])\n by smtp.corp.redhat.com (Postfix) with ESMTP id 8750827180;\n Wed, 11 Mar 2020 21:59:12 +0000 (UTC)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1583963961;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n content-transfer-encoding:content-transfer-encoding:\n in-reply-to:in-reply-to:references:references;\n bh=Ge3zFYqD4WTeTZkA9NRw4yoofJjMSVffWhtRm9bmEFs=;\n b=G/5ipPqvzEXgbQXubjNJtqlU+kY2wMuAW8zEiM2Z6PjxQk9wjQTeZjIeXJNWB6gh3gz8VG\n 1A+TGQsxnW2qFa17ZHyNnQat4WnM0xYxqbO5m1HH0DiOnrYD8MXWrwX1NyPi5EgF0JKglc\n eHS5t/guAg13R5o6rSuvsUtT/OqlqXg=", "X-MC-Unique": "1uQDeSaMNOOKR958PrCnNg-1", "From": "Alex Williamson <alex.williamson@redhat.com>", "To": "kvm@vger.kernel.org", "Cc": "linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, dev@dpdk.org,\n mtosatti@redhat.com, thomas@monjalon.net, bluca@debian.org,\n jerinjacobk@gmail.com, bruce.richardson@intel.com, cohuck@redhat.com,\n kevin.tian@intel.com", "Date": "Wed, 11 Mar 2020 15:59:12 -0600", "Message-ID": "<158396395214.5601.11207416598267070486.stgit@gimli.home>", "In-Reply-To": "<158396044753.5601.14804870681174789709.stgit@gimli.home>", "References": "<158396044753.5601.14804870681174789709.stgit@gimli.home>", "User-Agent": "StGit/0.19-dirty", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=\"utf-8\"", "Content-Transfer-Encoding": "7bit", "X-Scanned-By": "MIMEDefang 2.84 on 10.5.11.23", "Subject": "[dpdk-dev] [PATCH v3 5/7] vfio/pci: Add sriov_configure support", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "With the VF Token interface we can now expect that a vfio userspace\ndriver must be in collaboration with the PF driver, an unwitting\nuserspace driver will not be able to get past the GET_DEVICE_FD step\nin accessing the device. We can now move on to actually allowing\nSR-IOV to be enabled by vfio-pci on the PF. Support for this is not\nenabled by default in this commit, but it does provide a module option\nfor this to be enabled (enable_sriov=1). Enabling VFs is rather\nstraightforward, except we don't want to risk that a VF might get\nautoprobed and bound to other drivers, so a bus notifier is used to\n\"capture\" VFs to vfio-pci using the driver_override support. We\nassume any later action to bind the device to other drivers is\ncondoned by the system admin and allow it with a log warning.\n\nvfio-pci will disable SR-IOV on a PF before releasing the device,\nallowing a VF driver to be assured other drivers cannot take over the\nPF and that any other userspace driver must know the shared VF token.\nThis support also does not provide a mechanism for the PF userspace\ndriver itself to manipulate SR-IOV through the vfio API. With this\npatch SR-IOV can only be enabled via the host sysfs interface and the\nPF driver user cannot create or remove VFs.\n\nSigned-off-by: Alex Williamson <alex.williamson@redhat.com>\n---\n drivers/vfio/pci/vfio_pci.c | 106 +++++++++++++++++++++++++++++++----\n drivers/vfio/pci/vfio_pci_private.h | 2 +\n 2 files changed, 97 insertions(+), 11 deletions(-)", "diff": "diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c\nindex 6c28860f768a..df6bae75c8dd 100644\n--- a/drivers/vfio/pci/vfio_pci.c\n+++ b/drivers/vfio/pci/vfio_pci.c\n@@ -54,6 +54,12 @@ module_param(disable_idle_d3, bool, S_IRUGO | S_IWUSR);\n MODULE_PARM_DESC(disable_idle_d3,\n \t\t \"Disable using the PCI D3 low power state for idle, unused devices\");\n \n+static bool enable_sriov;\n+#ifdef CONFIG_PCI_IOV\n+module_param(enable_sriov, bool, 0644);\n+MODULE_PARM_DESC(enable_sriov, \"Enable support for SR-IOV configuration. Enabling SR-IOV on a PF typically requires support of the userspace PF driver, enabling VFs without such support may result in non-functional VFs or PF.\");\n+#endif\n+\n static inline bool vfio_vga_disabled(void)\n {\n #ifdef CONFIG_VFIO_PCI_VGA\n@@ -1535,6 +1541,35 @@ static const struct vfio_device_ops vfio_pci_ops = {\n \n static int vfio_pci_reflck_attach(struct vfio_pci_device *vdev);\n static void vfio_pci_reflck_put(struct vfio_pci_reflck *reflck);\n+static struct pci_driver vfio_pci_driver;\n+\n+static int vfio_pci_bus_notifier(struct notifier_block *nb,\n+\t\t\t\t unsigned long action, void *data)\n+{\n+\tstruct vfio_pci_device *vdev = container_of(nb,\n+\t\t\t\t\t\t struct vfio_pci_device, nb);\n+\tstruct device *dev = data;\n+\tstruct pci_dev *pdev = to_pci_dev(dev);\n+\tstruct pci_dev *physfn = pci_physfn(pdev);\n+\n+\tif (action == BUS_NOTIFY_ADD_DEVICE &&\n+\t pdev->is_virtfn && physfn == vdev->pdev) {\n+\t\tpci_info(vdev->pdev, \"Captured SR-IOV VF %s driver_override\\n\",\n+\t\t\t pci_name(pdev));\n+\t\tpdev->driver_override = kasprintf(GFP_KERNEL, \"%s\",\n+\t\t\t\t\t\t vfio_pci_ops.name);\n+\t} else if (action == BUS_NOTIFY_BOUND_DRIVER &&\n+\t\t pdev->is_virtfn && physfn == vdev->pdev) {\n+\t\tstruct pci_driver *drv = pci_dev_driver(pdev);\n+\n+\t\tif (drv && drv != &vfio_pci_driver)\n+\t\t\tpci_warn(vdev->pdev,\n+\t\t\t\t \"VF %s bound to driver %s while PF bound to vfio-pci\\n\",\n+\t\t\t\t pci_name(pdev), drv->name);\n+\t}\n+\n+\treturn 0;\n+}\n \n static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)\n {\n@@ -1546,12 +1581,12 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)\n \t\treturn -EINVAL;\n \n \t/*\n-\t * Prevent binding to PFs with VFs enabled, this too easily allows\n-\t * userspace instance with VFs and PFs from the same device, which\n-\t * cannot work. Disabling SR-IOV here would initiate removing the\n-\t * VFs, which would unbind the driver, which is prone to blocking\n-\t * if that VF is also in use by vfio-pci. Just reject these PFs\n-\t * and let the user sort it out.\n+\t * Prevent binding to PFs with VFs enabled, the VFs might be in use\n+\t * by the host or other users. We cannot capture the VFs if they\n+\t * already exist, nor can we track VF users. Disabling SR-IOV here\n+\t * would initiate removing the VFs, which would unbind the driver,\n+\t * which is prone to blocking if that VF is also in use by vfio-pci.\n+\t * Just reject these PFs and let the user sort it out.\n \t */\n \tif (pci_num_vf(pdev)) {\n \t\tpci_warn(pdev, \"Cannot bind to PF with SR-IOV enabled\\n\");\n@@ -1599,6 +1634,18 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)\n \t\t\tkfree(vdev);\n \t\t\treturn -ENOMEM;\n \t\t}\n+\n+\t\tvdev->nb.notifier_call = vfio_pci_bus_notifier;\n+\t\tret = bus_register_notifier(&pci_bus_type, &vdev->nb);\n+\t\tif (ret) {\n+\t\t\tkfree(vdev->vf_token);\n+\t\t\tvfio_pci_reflck_put(vdev->reflck);\n+\t\t\tvfio_del_group_dev(&pdev->dev);\n+\t\t\tvfio_iommu_group_put(group, &pdev->dev);\n+\t\t\tkfree(vdev);\n+\t\t\treturn ret;\n+\t\t}\n+\n \t\tmutex_init(&vdev->vf_token->lock);\n \t\tuuid_gen(&vdev->vf_token->uuid);\n \t}\n@@ -1632,6 +1679,8 @@ static void vfio_pci_remove(struct pci_dev *pdev)\n {\n \tstruct vfio_pci_device *vdev;\n \n+\tpci_disable_sriov(pdev);\n+\n \tvdev = vfio_del_group_dev(&pdev->dev);\n \tif (!vdev)\n \t\treturn;\n@@ -1642,6 +1691,9 @@ static void vfio_pci_remove(struct pci_dev *pdev)\n \t\tkfree(vdev->vf_token);\n \t}\n \n+\tif (vdev->nb.notifier_call)\n+\t\tbus_unregister_notifier(&pci_bus_type, &vdev->nb);\n+\n \tvfio_pci_reflck_put(vdev->reflck);\n \n \tvfio_iommu_group_put(pdev->dev.iommu_group, &pdev->dev);\n@@ -1690,16 +1742,48 @@ static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev,\n \treturn PCI_ERS_RESULT_CAN_RECOVER;\n }\n \n+static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn)\n+{\n+\tstruct vfio_pci_device *vdev;\n+\tstruct vfio_device *device;\n+\tint ret = 0;\n+\n+\tmight_sleep();\n+\n+\tif (!enable_sriov)\n+\t\treturn -ENOENT;\n+\n+\tdevice = vfio_device_get_from_dev(&pdev->dev);\n+\tif (!device)\n+\t\treturn -ENODEV;\n+\n+\tvdev = vfio_device_data(device);\n+\tif (!vdev) {\n+\t\tvfio_device_put(device);\n+\t\treturn -ENODEV;\n+\t}\n+\n+\tif (nr_virtfn == 0)\n+\t\tpci_disable_sriov(pdev);\n+\telse\n+\t\tret = pci_enable_sriov(pdev, nr_virtfn);\n+\n+\tvfio_device_put(device);\n+\n+\treturn ret < 0 ? ret : nr_virtfn;\n+}\n+\n static const struct pci_error_handlers vfio_err_handlers = {\n \t.error_detected = vfio_pci_aer_err_detected,\n };\n \n static struct pci_driver vfio_pci_driver = {\n-\t.name\t\t= \"vfio-pci\",\n-\t.id_table\t= NULL, /* only dynamic ids */\n-\t.probe\t\t= vfio_pci_probe,\n-\t.remove\t\t= vfio_pci_remove,\n-\t.err_handler\t= &vfio_err_handlers,\n+\t.name\t\t\t= \"vfio-pci\",\n+\t.id_table\t\t= NULL, /* only dynamic ids */\n+\t.probe\t\t\t= vfio_pci_probe,\n+\t.remove\t\t\t= vfio_pci_remove,\n+\t.sriov_configure\t= vfio_pci_sriov_configure,\n+\t.err_handler\t\t= &vfio_err_handlers,\n };\n \n static DEFINE_MUTEX(reflck_lock);\ndiff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pci_private.h\nindex 76c11c915949..36ec69081ecd 100644\n--- a/drivers/vfio/pci/vfio_pci_private.h\n+++ b/drivers/vfio/pci/vfio_pci_private.h\n@@ -13,6 +13,7 @@\n #include <linux/irqbypass.h>\n #include <linux/types.h>\n #include <linux/uuid.h>\n+#include <linux/notifier.h>\n \n #ifndef VFIO_PCI_PRIVATE_H\n #define VFIO_PCI_PRIVATE_H\n@@ -130,6 +131,7 @@ struct vfio_pci_device {\n \tstruct mutex\t\tioeventfds_lock;\n \tstruct list_head\tioeventfds_list;\n \tstruct vfio_pci_vf_token\t*vf_token;\n+\tstruct notifier_block\tnb;\n };\n \n #define is_intx(vdev) (vdev->irq_type == VFIO_PCI_INTX_IRQ_INDEX)\n", "prefixes": [ "v3", "5/7" ] }