Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/37941/?format=api
{ "id": 37941, "url": "http://patches.dpdk.org/api/patches/37941/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/patch/ffc78a8531c05e19e8b0e0a222abbb3a3c2c852c.1523448978.git.anatoly.burakov@intel.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<ffc78a8531c05e19e8b0e0a222abbb3a3c2c852c.1523448978.git.anatoly.burakov@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/ffc78a8531c05e19e8b0e0a222abbb3a3c2c852c.1523448978.git.anatoly.burakov@intel.com", "date": "2018-04-11T12:30:41", "name": "[dpdk-dev,v6,66/70] bus/fslmc: enable support for mem event callbacks for vfio", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": true, "hash": "c9d57b6d3a68fbfd75f85ab7ce89610f1219c78a", "submitter": { "id": 4, "url": "http://patches.dpdk.org/api/people/4/?format=api", "name": "Anatoly Burakov", "email": "anatoly.burakov@intel.com" }, "delegate": null, "mbox": "http://patches.dpdk.org/project/dpdk/patch/ffc78a8531c05e19e8b0e0a222abbb3a3c2c852c.1523448978.git.anatoly.burakov@intel.com/mbox/", "series": [], "comments": "http://patches.dpdk.org/api/patches/37941/comments/", "check": "success", "checks": "http://patches.dpdk.org/api/patches/37941/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 1B1261C14E;\n\tWed, 11 Apr 2018 14:33:26 +0200 (CEST)", "from mga01.intel.com (mga01.intel.com [192.55.52.88])\n\tby dpdk.org (Postfix) with ESMTP id 4D3161BBB0\n\tfor <dev@dpdk.org>; Wed, 11 Apr 2018 14:31:09 +0200 (CEST)", "from orsmga001.jf.intel.com ([10.7.209.18])\n\tby fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t11 Apr 2018 05:31:07 -0700", "from irvmail001.ir.intel.com ([163.33.26.43])\n\tby orsmga001.jf.intel.com with ESMTP; 11 Apr 2018 05:31:03 -0700", "from sivswdev01.ir.intel.com (sivswdev01.ir.intel.com\n\t[10.237.217.45])\n\tby irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id\n\tw3BCV2Aa012595; Wed, 11 Apr 2018 13:31:02 +0100", "from sivswdev01.ir.intel.com (localhost [127.0.0.1])\n\tby sivswdev01.ir.intel.com with ESMTP id w3BCV2w6014035;\n\tWed, 11 Apr 2018 13:31:02 +0100", "(from aburakov@localhost)\n\tby sivswdev01.ir.intel.com with LOCAL id w3BCV2OU014031;\n\tWed, 11 Apr 2018 13:31:02 +0100" ], "X-Amp-Result": "SKIPPED(no attachment in message)", "X-Amp-File-Uploaded": "False", "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.48,436,1517904000\"; d=\"scan'208\";a=\"46992940\"", "From": "Anatoly Burakov <anatoly.burakov@intel.com>", "To": "dev@dpdk.org", "Cc": "Hemant Agrawal <hemant.agrawal@nxp.com>,\n\tShreyansh Jain <shreyansh.jain@nxp.com>, keith.wiles@intel.com,\n\tjianfeng.tan@intel.com, andras.kovacs@ericsson.com,\n\tlaszlo.vadkeri@ericsson.com, benjamin.walker@intel.com,\n\tbruce.richardson@intel.com, thomas@monjalon.net,\n\tkonstantin.ananyev@intel.com, kuralamudhan.ramakrishnan@intel.com,\n\tlouise.m.daly@intel.com, nelio.laranjeiro@6wind.com,\n\tyskoh@mellanox.com, pepperjo@japf.ch, jerin.jacob@caviumnetworks.com, \n\tolivier.matz@6wind.com, gowrishankar.m@linux.vnet.ibm.com", "Date": "Wed, 11 Apr 2018 13:30:41 +0100", "Message-Id": "<ffc78a8531c05e19e8b0e0a222abbb3a3c2c852c.1523448978.git.anatoly.burakov@intel.com>", "X-Mailer": "git-send-email 1.7.0.7", "In-Reply-To": [ "<cover.1523448978.git.anatoly.burakov@intel.com>", "<cover.1523448978.git.anatoly.burakov@intel.com>" ], "References": [ "<cover.1523448978.git.anatoly.burakov@intel.com>", "<cover.1523296700.git.anatoly.burakov@intel.com>\n\t<cover.1523448978.git.anatoly.burakov@intel.com>" ], "Subject": "[dpdk-dev] [PATCH v6 66/70] bus/fslmc: enable support for mem event\n\tcallbacks for vfio", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "VFIO needs to map and unmap segments for DMA whenever they\nbecome available or unavailable, so register a callback for\nmemory events, and provide map/unmap functions.\n\nRemove unneeded check for number of segments, as in non-legacy\nmode this now becomes a valid scenario.\n\nSigned-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>\nSigned-off-by: Anatoly Burakov <anatoly.burakov@intel.com>\nTested-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>\nTested-by: Hemant Agrawal <hemant.agrawal@nxp.com>\nTested-by: Gowrishankar Muthukrishnan <gowrishankar.m@linux.vnet.ibm.com>\n---\n drivers/bus/fslmc/fslmc_vfio.c | 153 ++++++++++++++++++++++++++++++++++++-----\n 1 file changed, 135 insertions(+), 18 deletions(-)", "diff": "diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c\nindex db3eb61..625fa7c 100644\n--- a/drivers/bus/fslmc/fslmc_vfio.c\n+++ b/drivers/bus/fslmc/fslmc_vfio.c\n@@ -30,6 +30,7 @@\n #include <rte_kvargs.h>\n #include <rte_dev.h>\n #include <rte_bus.h>\n+#include <rte_eal_memconfig.h>\n \n #include \"rte_fslmc.h\"\n #include \"fslmc_vfio.h\"\n@@ -188,11 +189,62 @@ static int vfio_map_irq_region(struct fslmc_vfio_group *group)\n \treturn -errno;\n }\n \n+static int fslmc_map_dma(uint64_t vaddr, rte_iova_t iovaddr, size_t len);\n+static int fslmc_unmap_dma(uint64_t vaddr, rte_iova_t iovaddr, size_t len);\n+\n+static void\n+fslmc_memevent_cb(enum rte_mem_event type, const void *addr, size_t len)\n+{\n+\tstruct rte_memseg_list *msl;\n+\tstruct rte_memseg *ms;\n+\tsize_t cur_len = 0, map_len = 0;\n+\tuint64_t virt_addr;\n+\trte_iova_t iova_addr;\n+\tint ret;\n+\n+\tmsl = rte_mem_virt2memseg_list(addr);\n+\n+\twhile (cur_len < len) {\n+\t\tconst void *va = RTE_PTR_ADD(addr, cur_len);\n+\n+\t\tms = rte_mem_virt2memseg(va, msl);\n+\t\tiova_addr = ms->iova;\n+\t\tvirt_addr = ms->addr_64;\n+\t\tmap_len = ms->len;\n+\n+\t\tDPAA2_BUS_DEBUG(\"Request for %s, va=%p, \"\n+\t\t\t\t\"virt_addr=0x%\" PRIx64 \", \"\n+\t\t\t\t\"iova=0x%\" PRIx64 \", map_len=%zu\",\n+\t\t\t\ttype == RTE_MEM_EVENT_ALLOC ?\n+\t\t\t\t\t\"alloc\" : \"dealloc\",\n+\t\t\t\tva, virt_addr, iova_addr, map_len);\n+\n+\t\tif (type == RTE_MEM_EVENT_ALLOC)\n+\t\t\tret = fslmc_map_dma(virt_addr, iova_addr, map_len);\n+\t\telse\n+\t\t\tret = fslmc_unmap_dma(virt_addr, iova_addr, map_len);\n+\n+\t\tif (ret != 0) {\n+\t\t\tDPAA2_BUS_ERR(\"DMA Mapping/Unmapping failed. \"\n+\t\t\t\t\t\"Map=%d, addr=%p, len=%zu, err:(%d)\",\n+\t\t\t\t\ttype, va, map_len, ret);\n+\t\t\treturn;\n+\t\t}\n+\n+\t\tcur_len += map_len;\n+\t}\n+\n+\tif (type == RTE_MEM_EVENT_ALLOC)\n+\t\tDPAA2_BUS_DEBUG(\"Total Mapped: addr=%p, len=%zu\",\n+\t\t\t\taddr, len);\n+\telse\n+\t\tDPAA2_BUS_DEBUG(\"Total Unmapped: addr=%p, len=%zu\",\n+\t\t\t\taddr, len);\n+}\n+\n static int\n-fslmc_vfio_map(const struct rte_memseg_list *msl __rte_unused,\n-\t\tconst struct rte_memseg *ms, void *arg)\n+fslmc_map_dma(uint64_t vaddr, rte_iova_t iovaddr __rte_unused, size_t len)\n {\n-\tint *n_segs = arg;\n \tstruct fslmc_vfio_group *group;\n \tstruct vfio_iommu_type1_dma_map dma_map = {\n \t\t.argsz = sizeof(struct vfio_iommu_type1_dma_map),\n@@ -200,10 +252,11 @@ fslmc_vfio_map(const struct rte_memseg_list *msl __rte_unused,\n \t};\n \tint ret;\n \n-\tdma_map.size = ms->len;\n-\tdma_map.vaddr = ms->addr_64;\n+\tdma_map.size = len;\n+\tdma_map.vaddr = vaddr;\n+\n #ifdef RTE_LIBRTE_DPAA2_USE_PHYS_IOVA\n-\tdma_map.iova = ms->iova;\n+\tdma_map.iova = iovaddr;\n #else\n \tdma_map.iova = dma_map.vaddr;\n #endif\n@@ -216,32 +269,91 @@ fslmc_vfio_map(const struct rte_memseg_list *msl __rte_unused,\n \t\treturn -1;\n \t}\n \n-\tDPAA2_BUS_DEBUG(\"-->Initial SHM Virtual ADDR %llX\",\n-\t\t\tdma_map.vaddr);\n-\tDPAA2_BUS_DEBUG(\"-----> DMA size 0x%llX\", dma_map.size);\n-\tret = ioctl(group->container->fd, VFIO_IOMMU_MAP_DMA,\n-\t\t\t&dma_map);\n+\tDPAA2_BUS_DEBUG(\"--> Map address: %llX, size: 0x%llX\",\n+\t\t\tdma_map.vaddr, dma_map.size);\n+\tret = ioctl(group->container->fd, VFIO_IOMMU_MAP_DMA, &dma_map);\n \tif (ret) {\n \t\tDPAA2_BUS_ERR(\"VFIO_IOMMU_MAP_DMA API(errno = %d)\",\n \t\t\t\terrno);\n \t\treturn -1;\n \t}\n-\t(*n_segs)++;\n+\n \treturn 0;\n }\n \n-int rte_fslmc_vfio_dmamap(void)\n+static int\n+fslmc_unmap_dma(uint64_t vaddr, uint64_t iovaddr __rte_unused, size_t len)\n {\n-\tint i = 0;\n+\tstruct fslmc_vfio_group *group;\n+\tstruct vfio_iommu_type1_dma_unmap dma_unmap = {\n+\t\t.argsz = sizeof(struct vfio_iommu_type1_dma_unmap),\n+\t\t.flags = 0,\n+\t};\n+\tint ret;\n+\n+\tdma_unmap.size = len;\n+\tdma_unmap.iova = vaddr;\n \n-\tif (rte_memseg_walk(fslmc_vfio_map, &i) < 0)\n+\t/* SET DMA MAP for IOMMU */\n+\tgroup = &vfio_group;\n+\n+\tif (!group->container) {\n+\t\tDPAA2_BUS_ERR(\"Container is not connected \");\n \t\treturn -1;\n+\t}\n \n-\t/* Verifying that at least single segment is available */\n-\tif (i <= 0) {\n-\t\tDPAA2_BUS_ERR(\"No Segments found for VFIO Mapping\");\n+\tDPAA2_BUS_DEBUG(\"--> Unmap address: %llX, size: 0x%llX\",\n+\t\t\tdma_unmap.iova, dma_unmap.size);\n+\tret = ioctl(group->container->fd, VFIO_IOMMU_UNMAP_DMA, &dma_unmap);\n+\tif (ret) {\n+\t\tDPAA2_BUS_ERR(\"VFIO_IOMMU_UNMAP_DMA API(errno = %d)\",\n+\t\t\t\terrno);\n \t\treturn -1;\n \t}\n+\n+\treturn 0;\n+}\n+\n+static int\n+fslmc_dmamap_seg(const struct rte_memseg_list *msl __rte_unused,\n+\t\t const struct rte_memseg *ms, void *arg)\n+{\n+\tint *n_segs = arg;\n+\tint ret;\n+\n+\tret = fslmc_map_dma(ms->addr_64, ms->iova, ms->len);\n+\tif (ret)\n+\t\tDPAA2_BUS_ERR(\"Unable to VFIO map (addr=%p, len=%zu)\",\n+\t\t\t\tms->addr, ms->len);\n+\telse\n+\t\t(*n_segs)++;\n+\n+\treturn ret;\n+}\n+\n+int rte_fslmc_vfio_dmamap(void)\n+{\n+\tint i = 0, ret;\n+\tstruct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;\n+\trte_rwlock_t *mem_lock = &mcfg->memory_hotplug_lock;\n+\n+\t/* Lock before parsing and registering callback to memory subsystem */\n+\trte_rwlock_read_lock(mem_lock);\n+\n+\tif (rte_memseg_walk(fslmc_dmamap_seg, &i) < 0) {\n+\t\trte_rwlock_read_unlock(mem_lock);\n+\t\treturn -1;\n+\t}\n+\n+\tret = rte_mem_event_callback_register(\"fslmc_memevent_clb\",\n+\t\t\t\t\t fslmc_memevent_cb);\n+\tif (ret && rte_errno == ENOTSUP)\n+\t\tDPAA2_BUS_DEBUG(\"Memory event callbacks not supported\");\n+\telse if (ret)\n+\t\tDPAA2_BUS_DEBUG(\"Unable to install memory handler\");\n+\telse\n+\t\tDPAA2_BUS_DEBUG(\"Installed memory callback handler\");\n+\n \tDPAA2_BUS_DEBUG(\"Total %d segments found.\", i);\n \n \t/* TODO - This is a W.A. as VFIO currently does not add the mapping of\n@@ -250,6 +362,11 @@ int rte_fslmc_vfio_dmamap(void)\n \t */\n \tvfio_map_irq_region(&vfio_group);\n \n+\t/* Existing segments have been mapped and memory callback for hotplug\n+\t * has been installed.\n+\t */\n+\trte_rwlock_read_unlock(mem_lock);\n+\n \treturn 0;\n }\n \n", "prefixes": [ "dpdk-dev", "v6", "66/70" ] }