[v11,0/4] PCI Dev and SG copy support

  Improve dma-perf application to support PCI dev and SG copy,
along with additional supports as below:
 - validate copied memory
 - skip tests if not opted.

v11:
- Review suggestions.

Gowrishankar Muthukrishnan (4):
  app/dma-perf: add skip support
  app/dma-perf: add PCI device support
  app/dma-perf: validate copied memory
  app/dma-perf: add SG copy support

 app/test-dma-perf/benchmark.c | 413 ++++++++++++++++++++++++++++++----
 app/test-dma-perf/config.ini  |  56 +++++
 app/test-dma-perf/main.c      | 178 ++++++++++++---
 app/test-dma-perf/main.h      |  13 +-
 4 files changed, 595 insertions(+), 65 deletions(-)
  

29/02/2024 14:48, Gowrishankar Muthukrishnan:
> Improve dma-perf application to support PCI dev and SG copy,
> along with additional supports as below:
>  - validate copied memory
>  - skip tests if not opted.
> 
> v11:
> - Review suggestions.
> 
> Gowrishankar Muthukrishnan (4):
>   app/dma-perf: add skip support
>   app/dma-perf: add PCI device support
>   app/dma-perf: validate copied memory
>   app/dma-perf: add SG copy support

Waiting for a confirmation that this series is good to go.

Recheck-request: iol-unit-amd64-testing
  

Hi Thomas and Gowrishankar,

On 2024/3/7 3:50, Thomas Monjalon wrote:
> 29/02/2024 14:48, Gowrishankar Muthukrishnan:
>> Improve dma-perf application to support PCI dev and SG copy,
>> along with additional supports as below:
>>  - validate copied memory
>>  - skip tests if not opted.
>>
>> v11:
>> - Review suggestions.
>>
>> Gowrishankar Muthukrishnan (4):
>>   app/dma-perf: add skip support
>>   app/dma-perf: add PCI device support
>>   app/dma-perf: validate copied memory
>>   app/dma-perf: add SG copy support
> 
> Waiting for a confirmation that this series is good to go.

In the discuss of thread [1], I hope this patchset continue take a step
forward (means new version) to support bi-direction test just by modify
config.ini file.

[1] [PATCH v2] app/dma-perf: support bi-directional transfer

Thanks

> 
> Recheck-request: iol-unit-amd64-testing
> 
> 
> .
>
  

Hi Thomas,

> 29/02/2024 14:48, Gowrishankar Muthukrishnan:
> > Improve dma-perf application to support PCI dev and SG copy, along
> > with additional supports as below:
> >  - validate copied memory
> >  - skip tests if not opted.
> >
> > v11:
> > - Review suggestions.
> >
> > Gowrishankar Muthukrishnan (4):
> >   app/dma-perf: add skip support
> >   app/dma-perf: add PCI device support
> >   app/dma-perf: validate copied memory
> >   app/dma-perf: add SG copy support
> 
> Waiting for a confirmation that this series is good to go.
> 

Patches are already acked by maintainer and is ready for merge.

Thanks,
Gowrishankar
> Recheck-request: iol-unit-amd64-testing
>
  

Hi Fengchengwen,

> > Waiting for a confirmation that this series is good to go.
> 
> In the discuss of thread [1], I hope this patchset continue take a step forward
> (means new version) to support bi-direction test just by modify config.ini file.
> 

This patch set already exposes all configuration via config.ini. I didn't follow what is missing. For bi-direction, we can better continue discussing on that patch.

Thanks,
Gowrishankar

> [1] [PATCH v2] app/dma-perf: support bi-directional transfer
> 
> Thanks
> 
> >
> > Recheck-request: iol-unit-amd64-testing
> >
> >
> > .
> >
  

07/03/2024 14:55, Gowrishankar Muthukrishnan:
> Hi Fengchengwen,
> 
> > > Waiting for a confirmation that this series is good to go.
> > 
> > In the discuss of thread [1], I hope this patchset continue take a step forward
> > (means new version) to support bi-direction test just by modify config.ini file.
> > 
> 
> This patch set already exposes all configuration via config.ini. I didn't follow what is missing. For bi-direction, we can better continue discussing on that patch.

Chengwen, please can you confirm whether you require a new version?
Which change exactly is missing?
  

Hi Thomas,

On 2024/3/12 17:15, Thomas Monjalon wrote:
> 07/03/2024 14:55, Gowrishankar Muthukrishnan:
>> Hi Fengchengwen,
>>
>>>> Waiting for a confirmation that this series is good to go.
>>>
>>> In the discuss of thread [1], I hope this patchset continue take a step forward
>>> (means new version) to support bi-direction test just by modify config.ini file.
>>>
>>
>> This patch set already exposes all configuration via config.ini. I didn't follow what is missing. For bi-direction, we can better continue discussing on that patch.
> 
> Chengwen, please can you confirm whether you require a new version?
> Which change exactly is missing?

This patchset is OK with one sub-test only tackle one DMA direction.

But there is a later patch [1] which will support multiple DMA directions within one sub-test.
it will add a entry "xfer_mode", but I think it complicate the test, I prefer we do more in
this patchset to support some like bi-direction just by modify config.ini, some like this:

1. extend lcore_dma:
   current lcore_dma is: lcore10@0000:00:04.2
   extend it support: lcore10@0000:00:04.2,dir=m2d,coreid=1,pfid=2,vfid=3,raddr=0xXXXX
2. to fix one entry can't hold too many dma device, support entrys: lcore_dma_1, lcore_dma_2
   which value is same with lcore_dma.

So for bi-direction, we just define config.ini as:
lcore_dma=lcore10@0000:00:04.2,dir=m2d,coreid=1,pfid=2,vfid=3,raddr=0xXXXX, lcore10@0000:00:04.2,dir=d2m,coreid=1,pfid=2,vfid=3,raddr=0xXXXX

[1] https://patches.dpdk.org/project/dpdk/patch/20240229141426.4188428-1-amitprakashs@marvell.com/

>
  

Hi Fengchengwen

> 
> Hi Thomas,
> 
> On 2024/3/12 17:15, Thomas Monjalon wrote:
> > 07/03/2024 14:55, Gowrishankar Muthukrishnan:
> >> Hi Fengchengwen,
> >>
> >>>> Waiting for a confirmation that this series is good to go.
> >>>
> >>> In the discuss of thread [1], I hope this patchset continue take a
> >>> step forward (means new version) to support bi-direction test just by
> modify config.ini file.
> >>>
> >>
> >> This patch set already exposes all configuration via config.ini. I didn't follow
> what is missing. For bi-direction, we can better continue discussing on that
> patch.
> >
> > Chengwen, please can you confirm whether you require a new version?
> > Which change exactly is missing?
> 
> This patchset is OK with one sub-test only tackle one DMA direction.
> 
Thanks for the confirmation.

> But there is a later patch [1] which will support multiple DMA directions within
> one sub-test.
> it will add a entry "xfer_mode", but I think it complicate the test, I prefer we do
> more in this patchset to support some like bi-direction just by modify
> config.ini, some like this:
> 
I think we should discuss about that in bi-directional patch series. This series is self-contained and there is no need to add bi-directional as part of this series. As far as this patch set is concerned, all the options are exposed via config.ini. Can you comment if there is anything missing, assuming that we are taking bi-directional support as a separate feature addition.

Thanks,
Gowrishankar

> 1. extend lcore_dma:
>    current lcore_dma is: lcore10@0000:00:04.2
>    extend it support:
> lcore10@0000:00:04.2,dir=m2d,coreid=1,pfid=2,vfid=3,raddr=0xXXXX
> 2. to fix one entry can't hold too many dma device, support entrys:
> lcore_dma_1, lcore_dma_2
>    which value is same with lcore_dma.
> 
> So for bi-direction, we just define config.ini as:
> lcore_dma=lcore10@0000:00:04.2,dir=m2d,coreid=1,pfid=2,vfid=3,raddr=0x
> XXXX, lcore10@0000:00:04.2,dir=d2m,coreid=1,pfid=2,vfid=3,raddr=0xXXXX
> 
> [1] https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__patches.dpdk.org_project_dpdk_patch_20240229141426.4188428-
> 2D1-2Damitprakashs-
> 40marvell.com_&d=DwICaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=EAtr-
> g7yUFhtOio8r2Rtm13Aqe4WVp_S_gHpcu6KFVo&m=DUaL_AJR1zqM0T2yw3
> aV44EObB90uqw5weFzSm-
> w39citSeGozNdEe4kzicss_KG&s=UTAcoZx5DjSJHyzxyLMxXz1bPqfPXQM7feDx
> ZdC6Jgk&e=
> 
> >
  

Hi Gowrishankar,

On 2024/3/12 20:24, Gowrishankar Muthukrishnan wrote:
> Hi Fengchengwen
> 
>>
>> Hi Thomas,
>>
>> On 2024/3/12 17:15, Thomas Monjalon wrote:
>>> 07/03/2024 14:55, Gowrishankar Muthukrishnan:
>>>> Hi Fengchengwen,
>>>>
>>>>>> Waiting for a confirmation that this series is good to go.
>>>>>
>>>>> In the discuss of thread [1], I hope this patchset continue take a
>>>>> step forward (means new version) to support bi-direction test just by
>> modify config.ini file.
>>>>>
>>>>
>>>> This patch set already exposes all configuration via config.ini. I didn't follow
>> what is missing. For bi-direction, we can better continue discussing on that
>> patch.
>>>
>>> Chengwen, please can you confirm whether you require a new version?
>>> Which change exactly is missing?
>>
>> This patchset is OK with one sub-test only tackle one DMA direction.
>>
> Thanks for the confirmation.
> 
>> But there is a later patch [1] which will support multiple DMA directions within
>> one sub-test.
>> it will add a entry "xfer_mode", but I think it complicate the test, I prefer we do
>> more in this patchset to support some like bi-direction just by modify
>> config.ini, some like this:
>>
> I think we should discuss about that in bi-directional patch series. This series is self-contained and there is no need to add bi-directional as part of this series. As far as this patch set is concerned, all the options are exposed via config.ini. Can you comment if there is anything missing, assuming that we are taking bi-directional support as a separate feature addition.

I have identified some improvements to the dma-perf app, and I plan to do it in
24.07, so if you don't mind, I will incorporate your commits (keeping your signed-off-by) and
modify to the one that I described above, and then send to community (also with my improvements
commits).

Thanks

> 
> Thanks,
> Gowrishankar
> 
>> 1. extend lcore_dma:
>>    current lcore_dma is: lcore10@0000:00:04.2
>>    extend it support:
>> lcore10@0000:00:04.2,dir=m2d,coreid=1,pfid=2,vfid=3,raddr=0xXXXX
>> 2. to fix one entry can't hold too many dma device, support entrys:
>> lcore_dma_1, lcore_dma_2
>>    which value is same with lcore_dma.
>>
>> So for bi-direction, we just define config.ini as:
>> lcore_dma=lcore10@0000:00:04.2,dir=m2d,coreid=1,pfid=2,vfid=3,raddr=0x
>> XXXX, lcore10@0000:00:04.2,dir=d2m,coreid=1,pfid=2,vfid=3,raddr=0xXXXX
>>
>> [1] https://urldefense.proofpoint.com/v2/url?u=https-
>> 3A__patches.dpdk.org_project_dpdk_patch_20240229141426.4188428-
>> 2D1-2Damitprakashs-
>> 40marvell.com_&d=DwICaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=EAtr-
>> g7yUFhtOio8r2Rtm13Aqe4WVp_S_gHpcu6KFVo&m=DUaL_AJR1zqM0T2yw3
>> aV44EObB90uqw5weFzSm-
>> w39citSeGozNdEe4kzicss_KG&s=UTAcoZx5DjSJHyzxyLMxXz1bPqfPXQM7feDx
>> ZdC6Jgk&e=
>>
>>>
  

Hi Fengchengwen

> Hi Gowrishankar,
> 
> On 2024/3/12 20:24, Gowrishankar Muthukrishnan wrote:
> > Hi Fengchengwen
> >
> >>
> >> Hi Thomas,
> >>
> >> On 2024/3/12 17:15, Thomas Monjalon wrote:
> >>> 07/03/2024 14:55, Gowrishankar Muthukrishnan:
> >>>> Hi Fengchengwen,
> >>>>
> >>>>>> Waiting for a confirmation that this series is good to go.
> >>>>>
> >>>>> In the discuss of thread [1], I hope this patchset continue take a
> >>>>> step forward (means new version) to support bi-direction test just
> >>>>> by
> >> modify config.ini file.
> >>>>>
> >>>>
> >>>> This patch set already exposes all configuration via config.ini. I
> >>>> didn't follow
> >> what is missing. For bi-direction, we can better continue discussing
> >> on that patch.
> >>>
> >>> Chengwen, please can you confirm whether you require a new version?
> >>> Which change exactly is missing?
> >>
> >> This patchset is OK with one sub-test only tackle one DMA direction.
> >>
> > Thanks for the confirmation.
> >
> >> But there is a later patch [1] which will support multiple DMA
> >> directions within one sub-test.
> >> it will add a entry "xfer_mode", but I think it complicate the test,
> >> I prefer we do more in this patchset to support some like
> >> bi-direction just by modify config.ini, some like this:
> >>
> > I think we should discuss about that in bi-directional patch series. This series
> is self-contained and there is no need to add bi-directional as part of this
> series. As far as this patch set is concerned, all the options are exposed via
> config.ini. Can you comment if there is anything missing, assuming that we are
> taking bi-directional support as a separate feature addition.
> 
> I have identified some improvements to the dma-perf app, and I plan to do it

It is unclear at this point what is the issue that you have with the app or this patch set. This series was first submitted on Aug 10 2023. You had acked v8 on Jan 25 2024. After the patches were acked, there were still review comments on variable renames etc, which were all addressed. The patches had been under review for more than 8 months with very slow progress.

> in 24.07, so if you don't mind, I will incorporate your commits (keeping your
> signed-off-by) and modify to the one that I described above, and then send to
> community (also with my improvements commits).

I would like to have this series merged first and not pulled into another series. We do have few other features that we would like to add on top. I would assume that you can also add your changes on top. To make contribution easier, isn't it better to accept at least this patch set (as you acked earlier) and then you can continue working on the improvements?

Thanks,
Gowrishankar

> 
> Thanks
> 
> >
> > Thanks,
> > Gowrishankar
> >
> >> 1. extend lcore_dma:
> >>    current lcore_dma is: lcore10@0000:00:04.2
> >>    extend it support:
> >> lcore10@0000:00:04.2,dir=m2d,coreid=1,pfid=2,vfid=3,raddr=0xXXXX
> >> 2. to fix one entry can't hold too many dma device, support entrys:
> >> lcore_dma_1, lcore_dma_2
> >>    which value is same with lcore_dma.
> >>
> >> So for bi-direction, we just define config.ini as:
> >>
> lcore_dma=lcore10@0000:00:04.2,dir=m2d,coreid=1,pfid=2,vfid=3,raddr=0
> >> x XXXX,
> >> lcore10@0000:00:04.2,dir=d2m,coreid=1,pfid=2,vfid=3,raddr=0xXXXX
> >>
> >> [1] https://urldefense.proofpoint.com/v2/url?u=https-
> >> 3A__patches.dpdk.org_project_dpdk_patch_20240229141426.4188428-
> >> 2D1-2Damitprakashs-
> >> 40marvell.com_&d=DwICaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=EAtr-
> >>
> g7yUFhtOio8r2Rtm13Aqe4WVp_S_gHpcu6KFVo&m=DUaL_AJR1zqM0T2yw3
> >> aV44EObB90uqw5weFzSm-
> >>
> w39citSeGozNdEe4kzicss_KG&s=UTAcoZx5DjSJHyzxyLMxXz1bPqfPXQM7feDx
> >> ZdC6Jgk&e=
> >>
> >>>
  

Hi Thomas,

> 
> Hi Fengchengwen
> 
> > Hi Gowrishankar,
> >
> > On 2024/3/12 20:24, Gowrishankar Muthukrishnan wrote:
> > > Hi Fengchengwen
> > >
> > >>
> > >> Hi Thomas,
> > >>
> > >> On 2024/3/12 17:15, Thomas Monjalon wrote:
> > >>> 07/03/2024 14:55, Gowrishankar Muthukrishnan:
> > >>>> Hi Fengchengwen,
> > >>>>
> > >>>>>> Waiting for a confirmation that this series is good to go.
> > >>>>>
> > >>>>> In the discuss of thread [1], I hope this patchset continue take
> > >>>>> a step forward (means new version) to support bi-direction test
> > >>>>> just by
> > >> modify config.ini file.
> > >>>>>
> > >>>>
> > >>>> This patch set already exposes all configuration via config.ini.
> > >>>> I didn't follow
> > >> what is missing. For bi-direction, we can better continue
> > >> discussing on that patch.
> > >>>
> > >>> Chengwen, please can you confirm whether you require a new version?
> > >>> Which change exactly is missing?
> > >>
> > >> This patchset is OK with one sub-test only tackle one DMA direction.
> > >>
> > > Thanks for the confirmation.
> > >
> > >> But there is a later patch [1] which will support multiple DMA
> > >> directions within one sub-test.
> > >> it will add a entry "xfer_mode", but I think it complicate the
> > >> test, I prefer we do more in this patchset to support some like
> > >> bi-direction just by modify config.ini, some like this:
> > >>
> > > I think we should discuss about that in bi-directional patch series.
> > > This series
> > is self-contained and there is no need to add bi-directional as part
> > of this series. As far as this patch set is concerned, all the options
> > are exposed via config.ini. Can you comment if there is anything
> > missing, assuming that we are taking bi-directional support as a separate
> feature addition.
> >
> > I have identified some improvements to the dma-perf app, and I plan to
> > do it
> 
> It is unclear at this point what is the issue that you have with the app or this
> patch set. This series was first submitted on Aug 10 2023. You had acked v8 on
> Jan 25 2024. After the patches were acked, there were still review comments
> on variable renames etc, which were all addressed. The patches had been
> under review for more than 8 months with very slow progress.
> 
> > in 24.07, so if you don't mind, I will incorporate your commits
> > (keeping your
> > signed-off-by) and modify to the one that I described above, and then
> > send to community (also with my improvements commits).
> 
> I would like to have this series merged first and not pulled into another series.
> We do have few other features that we would like to add on top. I would
> assume that you can also add your changes on top. To make contribution
> easier, isn't it better to accept at least this patch set (as you acked earlier) and
> then you can continue working on the improvements?
> 

Can this series be merged ?

Thanks,
Gowrishankar
> Thanks,
> Gowrishankar
>
  

13/03/2024 09:22, Gowrishankar Muthukrishnan:
> Hi Fengchengwen
> 
> > Hi Gowrishankar,
> > 
> > On 2024/3/12 20:24, Gowrishankar Muthukrishnan wrote:
> > > Hi Fengchengwen
> > >
> > >>
> > >> Hi Thomas,
> > >>
> > >> On 2024/3/12 17:15, Thomas Monjalon wrote:
> > >>> 07/03/2024 14:55, Gowrishankar Muthukrishnan:
> > >>>> Hi Fengchengwen,
> > >>>>
> > >>>>>> Waiting for a confirmation that this series is good to go.
> > >>>>>
> > >>>>> In the discuss of thread [1], I hope this patchset continue take a
> > >>>>> step forward (means new version) to support bi-direction test just
> > >>>>> by
> > >> modify config.ini file.
> > >>>>>
> > >>>>
> > >>>> This patch set already exposes all configuration via config.ini. I
> > >>>> didn't follow
> > >> what is missing. For bi-direction, we can better continue discussing
> > >> on that patch.
> > >>>
> > >>> Chengwen, please can you confirm whether you require a new version?
> > >>> Which change exactly is missing?
> > >>
> > >> This patchset is OK with one sub-test only tackle one DMA direction.
> > >>
> > > Thanks for the confirmation.
> > >
> > >> But there is a later patch [1] which will support multiple DMA
> > >> directions within one sub-test.
> > >> it will add a entry "xfer_mode", but I think it complicate the test,
> > >> I prefer we do more in this patchset to support some like
> > >> bi-direction just by modify config.ini, some like this:
> > >>
> > > I think we should discuss about that in bi-directional patch series. This series
> > is self-contained and there is no need to add bi-directional as part of this
> > series. As far as this patch set is concerned, all the options are exposed via
> > config.ini. Can you comment if there is anything missing, assuming that we are
> > taking bi-directional support as a separate feature addition.
> > 
> > I have identified some improvements to the dma-perf app, and I plan to do it
> 
> It is unclear at this point what is the issue that you have with the app or this patch set. This series was first submitted on Aug 10 2023. You had acked v8 on Jan 25 2024. After the patches were acked, there were still review comments on variable renames etc, which were all addressed. The patches had been under review for more than 8 months with very slow progress.
> 
> > in 24.07, so if you don't mind, I will incorporate your commits (keeping your
> > signed-off-by) and modify to the one that I described above, and then send to
> > community (also with my improvements commits).
> 
> I would like to have this series merged first and not pulled into another series. We do have few other features that we would like to add on top. I would assume that you can also add your changes on top. To make contribution easier, isn't it better to accept at least this patch set (as you acked earlier) and then you can continue working on the improvements?

OK, one feature at a time.
Let's work on top of this patchset applied.
  

Hi Thomas and Fengchengwen,

> 
> 13/03/2024 09:22, Gowrishankar Muthukrishnan:
> > Hi Fengchengwen
> >
> > > Hi Gowrishankar,
> > >
> > > On 2024/3/12 20:24, Gowrishankar Muthukrishnan wrote:
> > > > Hi Fengchengwen
> > > >
> > > >>
> > > >> Hi Thomas,
> > > >>
> > > >> On 2024/3/12 17:15, Thomas Monjalon wrote:
> > > >>> 07/03/2024 14:55, Gowrishankar Muthukrishnan:
> > > >>>> Hi Fengchengwen,
> > > >>>>
> > > >>>>>> Waiting for a confirmation that this series is good to go.
> > > >>>>>
> > > >>>>> In the discuss of thread [1], I hope this patchset continue
> > > >>>>> take a step forward (means new version) to support
> > > >>>>> bi-direction test just by
> > > >> modify config.ini file.
> > > >>>>>
> > > >>>>
> > > >>>> This patch set already exposes all configuration via
> > > >>>> config.ini. I didn't follow
> > > >> what is missing. For bi-direction, we can better continue
> > > >> discussing on that patch.
> > > >>>
> > > >>> Chengwen, please can you confirm whether you require a new
> version?
> > > >>> Which change exactly is missing?
> > > >>
> > > >> This patchset is OK with one sub-test only tackle one DMA direction.
> > > >>
> > > > Thanks for the confirmation.
> > > >
> > > >> But there is a later patch [1] which will support multiple DMA
> > > >> directions within one sub-test.
> > > >> it will add a entry "xfer_mode", but I think it complicate the
> > > >> test, I prefer we do more in this patchset to support some like
> > > >> bi-direction just by modify config.ini, some like this:
> > > >>
> > > > I think we should discuss about that in bi-directional patch
> > > > series. This series
> > > is self-contained and there is no need to add bi-directional as part
> > > of this series. As far as this patch set is concerned, all the
> > > options are exposed via config.ini. Can you comment if there is
> > > anything missing, assuming that we are taking bi-directional support as a
> separate feature addition.
> > >
> > > I have identified some improvements to the dma-perf app, and I plan
> > > to do it
> >
> > It is unclear at this point what is the issue that you have with the app or this
> patch set. This series was first submitted on Aug 10 2023. You had acked v8 on
> Jan 25 2024. After the patches were acked, there were still review comments
> on variable renames etc, which were all addressed. The patches had been
> under review for more than 8 months with very slow progress.
> >
> > > in 24.07, so if you don't mind, I will incorporate your commits
> > > (keeping your
> > > signed-off-by) and modify to the one that I described above, and
> > > then send to community (also with my improvements commits).
> >
> > I would like to have this series merged first and not pulled into another
> series. We do have few other features that we would like to add on top. I
> would assume that you can also add your changes on top. To make
> contribution easier, isn't it better to accept at least this patch set (as you acked
> earlier) and then you can continue working on the improvements?
> 
> OK, one feature at a time.
> Let's work on top of this patchset applied.
> 

Thank you both for reviewing this series and accepting it in RC3.

Regards,
Gowrishankar