[20/20] bus/fslmc: add function to map any addr via VFIO
Checks
Commit Message
From: Pankaj Chauhan <pankaj.chauhan@nxp.com>
This is required to map any accelerator memory
and PCI address to VFIO using QDMA.
Signed-off-by: Minghuan Lian <Minghuan.Lian@nxp.com>
Signed-off-by: Pankaj Chauhan <pankaj.chauhan@nxp.com>
---
drivers/bus/fslmc/fslmc_vfio.c | 42 +++++++++++++++++++++
drivers/bus/fslmc/fslmc_vfio.h | 1 +
drivers/bus/fslmc/rte_bus_fslmc_version.map | 7 ++++
3 files changed, 50 insertions(+)
Comments
On 12/27/2018 6:23 AM, Hemant Agrawal wrote:
> From: Pankaj Chauhan <pankaj.chauhan@nxp.com>
>
> This is required to map any accelerator memory
> and PCI address to VFIO using QDMA.
>
> Signed-off-by: Minghuan Lian <Minghuan.Lian@nxp.com>
> Signed-off-by: Pankaj Chauhan <pankaj.chauhan@nxp.com>
This requires either sign-off or ack from 'bus/fslmc' maintainers, which are:
M: Hemant Agrawal <hemant.agrawal@nxp.com>
M: Shreyansh Jain <shreyansh.jain@nxp.com>
I think Hemant sending the patchset implies that this has been reviewed by him
but this information get lost in git, so better to explicitly provide review/ack
tags whenever appropriate.
<...>
> + printf("PCIe vfio map 0x%llx:0x%llx, size 0x%llx\n", dma_map.vaddr,
> + dma_map.iova, dma_map.size);
This is causing build error [1], but why at first place using 'printf()' instead
of logging macros?
[1]
.../drivers/bus/fslmc/fslmc_vfio.c: In function ‘rte_fslmc_vfio_mem_dmamap’:
.../drivers/bus/fslmc/fslmc_vfio.c:376:29: error: format ‘%llx’ expects argument
of type ‘long long unsigned int’, but argument 2 has type ‘__u64’ {aka ‘long
unsigned int’} [-Werror=format=]
printf("PCIe vfio map 0x%llx:0x%llx, size 0x%llx\n", dma_map.vaddr,
~~~^ ~~~~~~~~~~~~~
%lx
<...>
> +DPDK_19.02 {
> + global:
> +
> + rte_fslmc_vfio_mem_dmamap;
Is this need to be an API? Who is the consumer of this API, I don't see anyone
calls this function?
Hello Ferruh,
Replying on behalf of Hemant...
On 08/01/19 7:40 PM, Ferruh Yigit wrote:
> On 12/27/2018 6:23 AM, Hemant Agrawal wrote:
>> From: Pankaj Chauhan <pankaj.chauhan@nxp.com>
>>
>> This is required to map any accelerator memory
>> and PCI address to VFIO using QDMA.
>>
>> Signed-off-by: Minghuan Lian <Minghuan.Lian@nxp.com>
>> Signed-off-by: Pankaj Chauhan <pankaj.chauhan@nxp.com>
>
> This requires either sign-off or ack from 'bus/fslmc' maintainers, which are:
> M: Hemant Agrawal <hemant.agrawal@nxp.com>
>
>
>
> M: Shreyansh Jain <shreyansh.jain@nxp.com>
>
> I think Hemant sending the patchset implies that this has been reviewed by him
> but this information get lost in git, so better to explicitly provide review/ack
> tags whenever appropriate.
What I will do is, re-spin the complete series (with error below
fixed...) and send again. All the patches are on FSLMC/DPAA2 drivers
only so probably Hemant's or my Acks would work directly.
>
> <...>
>
>> + printf("PCIe vfio map 0x%llx:0x%llx, size 0x%llx\n", dma_map.vaddr,
>> + dma_map.iova, dma_map.size);
>
> This is causing build error [1], but why at first place using 'printf()' instead
> of logging macros?
I will fix this.
>
> [1]
> .../drivers/bus/fslmc/fslmc_vfio.c: In function ‘rte_fslmc_vfio_mem_dmamap’:
> .../drivers/bus/fslmc/fslmc_vfio.c:376:29: error: format ‘%llx’ expects argument
> of type ‘long long unsigned int’, but argument 2 has type ‘__u64’ {aka ‘long
> unsigned int’} [-Werror=format=]
> printf("PCIe vfio map 0x%llx:0x%llx, size 0x%llx\n", dma_map.vaddr,
> ~~~^ ~~~~~~~~~~~~~
> %lx
>
> <...>
>
>> +DPDK_19.02 {
>> + global:
>> +
>> + rte_fslmc_vfio_mem_dmamap;
>
> Is this need to be an API? Who is the consumer of this API, I don't see anyone
> calls this function?
>
This API (internal to FSLMC) was added for one of NXP's internal
software stack over DPDK. As this is an internal API, I don't think it
would pollute the outer namespace - isn't it? And, I think its consumers
won't necessarily be within DPDK stack.
Or, if this is conflicting case, I will remove this patch (it is
independent) and send again. Let me know your reservation.
-
Shreyansh
On 1/10/2019 9:58 AM, Shreyansh Jain wrote:
> Hello Ferruh,
>
> Replying on behalf of Hemant...
>
> On 08/01/19 7:40 PM, Ferruh Yigit wrote:
>> On 12/27/2018 6:23 AM, Hemant Agrawal wrote:
>>> From: Pankaj Chauhan <pankaj.chauhan@nxp.com>
>>>
>>> This is required to map any accelerator memory
>>> and PCI address to VFIO using QDMA.
>>>
>>> Signed-off-by: Minghuan Lian <Minghuan.Lian@nxp.com>
>>> Signed-off-by: Pankaj Chauhan <pankaj.chauhan@nxp.com>
>>
>> This requires either sign-off or ack from 'bus/fslmc' maintainers, which are:
>> M: Hemant Agrawal <hemant.agrawal@nxp.com>
>>
>>
>>
>> M: Shreyansh Jain <shreyansh.jain@nxp.com>
>>
>> I think Hemant sending the patchset implies that this has been reviewed by him
>> but this information get lost in git, so better to explicitly provide review/ack
>> tags whenever appropriate.
>
> What I will do is, re-spin the complete series (with error below
> fixed...) and send again. All the patches are on FSLMC/DPAA2 drivers
> only so probably Hemant's or my Acks would work directly.
>
>>
>> <...>
>>
>>> + printf("PCIe vfio map 0x%llx:0x%llx, size 0x%llx\n", dma_map.vaddr,
>>> + dma_map.iova, dma_map.size);
>>
>> This is causing build error [1], but why at first place using 'printf()' instead
>> of logging macros?
>
> I will fix this.
>
>>
>> [1]
>> .../drivers/bus/fslmc/fslmc_vfio.c: In function ‘rte_fslmc_vfio_mem_dmamap’:
>> .../drivers/bus/fslmc/fslmc_vfio.c:376:29: error: format ‘%llx’ expects argument
>> of type ‘long long unsigned int’, but argument 2 has type ‘__u64’ {aka ‘long
>> unsigned int’} [-Werror=format=]
>> printf("PCIe vfio map 0x%llx:0x%llx, size 0x%llx\n", dma_map.vaddr,
>> ~~~^ ~~~~~~~~~~~~~
>> %lx
>>
>> <...>
>>
>>> +DPDK_19.02 {
>>> + global:
>>> +
>>> + rte_fslmc_vfio_mem_dmamap;
>>
>> Is this need to be an API? Who is the consumer of this API, I don't see anyone
>> calls this function?
>>
>
> This API (internal to FSLMC) was added for one of NXP's internal
> software stack over DPDK. As this is an internal API, I don't think it
> would pollute the outer namespace - isn't it? And, I think its consumers
> won't necessarily be within DPDK stack.
>
> Or, if this is conflicting case, I will remove this patch (it is
> independent) and send again. Let me know your reservation.
API consumers doesn't have to be in DPDK but than it is not an internal API.
And for API we recently decided to have an implementation for new APIs, it can
be unit test, sample application, or app like testpmd, so that we can detect
when it is broken.
Perhaps it can be good idea to separate patch, as you suggested, to not block
the rest of the patchset.
On 11/01/19 5:28 PM, Ferruh Yigit wrote:
> On 1/10/2019 9:58 AM, Shreyansh Jain wrote:
>> Hello Ferruh,
>>
>> Replying on behalf of Hemant...
>>
>> On 08/01/19 7:40 PM, Ferruh Yigit wrote:
>>> On 12/27/2018 6:23 AM, Hemant Agrawal wrote:
>>>> From: Pankaj Chauhan <pankaj.chauhan@nxp.com>
[...]
>>>> +DPDK_19.02 {
>>>> + global:
>>>> +
>>>> + rte_fslmc_vfio_mem_dmamap;
>>>
>>> Is this need to be an API? Who is the consumer of this API, I don't see anyone
>>> calls this function?
>>>
>>
>> This API (internal to FSLMC) was added for one of NXP's internal
>> software stack over DPDK. As this is an internal API, I don't think it
>> would pollute the outer namespace - isn't it? And, I think its consumers
>> won't necessarily be within DPDK stack.
>>
>> Or, if this is conflicting case, I will remove this patch (it is
>> independent) and send again. Let me know your reservation.
>
> API consumers doesn't have to be in DPDK but than it is not an internal API.
>
> And for API we recently decided to have an implementation for new APIs, it can
> be unit test, sample application, or app like testpmd, so that we can detect
> when it is broken.
>
> Perhaps it can be good idea to separate patch, as you suggested, to not block
> the rest of the patchset.
>
Oops, I think I sent my series just about the time you wrote this.
I will send it again (v3) without the last patch. I am OK with that API
not making it in right now.
@@ -50,6 +50,7 @@ static struct fslmc_vfio_group vfio_group;
static struct fslmc_vfio_container vfio_container;
static int container_device_fd;
static char *g_container;
+static int fslmc_iommu_type;
static uint32_t *msi_intr_vaddr;
void *(*rte_mcp_ptr_list);
@@ -90,6 +91,9 @@ fslmc_get_container_group(int *groupid)
}
}
+ fslmc_iommu_type = (rte_vfio_noiommu_is_enabled() == 1) ?
+ RTE_VFIO_NOIOMMU : VFIO_TYPE1_IOMMU;
+
/* get group number */
ret = rte_vfio_get_group_num(SYSFS_FSL_MC_DEVICES,
g_container, groupid);
@@ -344,6 +348,44 @@ fslmc_dmamap_seg(const struct rte_memseg_list *msl __rte_unused,
return ret;
}
+int rte_fslmc_vfio_mem_dmamap(uint64_t vaddr, uint64_t iova, uint64_t size)
+{
+ int ret;
+ struct fslmc_vfio_group *group;
+ struct vfio_iommu_type1_dma_map dma_map = {
+ .argsz = sizeof(struct vfio_iommu_type1_dma_map),
+ .flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE,
+ };
+
+ if (fslmc_iommu_type == RTE_VFIO_NOIOMMU) {
+ DPAA2_BUS_DEBUG("Running in NOIOMMU mode");
+ return 0;
+ }
+
+ /* SET DMA MAP for IOMMU */
+ group = &vfio_group;
+ if (!group->container) {
+ DPAA2_BUS_ERR("Container is not connected");
+ return -1;
+ }
+
+ dma_map.size = size;
+ dma_map.vaddr = vaddr;
+ dma_map.iova = iova;
+
+ printf("PCIe vfio map 0x%llx:0x%llx, size 0x%llx\n", dma_map.vaddr,
+ dma_map.iova, dma_map.size);
+ ret = ioctl(group->container->fd, VFIO_IOMMU_MAP_DMA,
+ &dma_map);
+ if (ret) {
+ printf("Unable to map DMA address (errno = %d)\n",
+ errno);
+ return ret;
+ }
+
+ return 0;
+}
+
int rte_fslmc_vfio_dmamap(void)
{
int i = 0, ret;
@@ -50,5 +50,6 @@ int fslmc_vfio_process_group(void);
char *fslmc_get_container(void);
int fslmc_get_container_group(int *gropuid);
int rte_fslmc_vfio_dmamap(void);
+int rte_fslmc_vfio_mem_dmamap(uint64_t vaddr, uint64_t iova, uint64_t size);
#endif /* _FSLMC_VFIO_H_ */
@@ -129,3 +129,10 @@ DPDK_18.11 {
dpci_set_opr;
} DPDK_18.05;
+
+DPDK_19.02 {
+ global:
+
+ rte_fslmc_vfio_mem_dmamap;
+
+} DPDK_18.11;