[v2,0/4] add crypto perf test graphing script

  This patchset introduces a python script to run various crypto performance
test cases, and graph the results in a consumable manner. The test suites
are configured via JSON file. Some config files are provided,
or the user may create one. Currently throughput and latency ptests for
devices crypto_qat, crypto_aesni_mb and crypto_aesni_gcm are supported.

The final collection of graphs are output in PDF format, with multiple PDFs
per test suite, one for each graph type.

Some fixes are included for the throughput performance test and latency
performance test csv outputs also.

v2:
  - Reduced changes to only fix csv format for all perf test types.
  - Added functionality for additional args such as config file,
    output directory and verbose.
  - Improved help text for script.
  - Improved script console output.
  - Added support for latency test cases with burst or buffer size lists.
  - Split config file into smaller config files, one for each device.
  - Split output PDFs into smaller files, based on test suite graph types.
  - Modified output directory naming and structure.
  - Made some general improvements to script.
  - Updated and improved documentation.

Ciara Power (4):
  test/cryptodev: fix latency test csv output
  test/cryptodev: fix csv output format
  usertools: add script to graph crypto perf results
  maintainers: update crypto perf app maintainers

 MAINTAINERS                                   |   3 +
 app/test-crypto-perf/cperf_test_latency.c     |   4 +-
 .../cperf_test_pmd_cyclecount.c               |   2 +-
 app/test-crypto-perf/cperf_test_throughput.c  |   4 +-
 app/test-crypto-perf/cperf_test_verify.c      |   2 +-
 doc/guides/tools/cryptoperf.rst               | 142 ++++++++
 usertools/configs/crypto-perf-aesni-gcm.json  |  99 ++++++
 usertools/configs/crypto-perf-aesni-mb.json   | 108 ++++++
 usertools/configs/crypto-perf-qat.json        |  94 ++++++
 usertools/dpdk-graph-crypto-perf.py           | 309 ++++++++++++++++++
 10 files changed, 761 insertions(+), 6 deletions(-)
 create mode 100644 usertools/configs/crypto-perf-aesni-gcm.json
 create mode 100644 usertools/configs/crypto-perf-aesni-mb.json
 create mode 100644 usertools/configs/crypto-perf-qat.json
 create mode 100755 usertools/dpdk-graph-crypto-perf.py
  

On 14/01/2021 10:41 AM, Ciara Power wrote:
> This patchset introduces a python script to run various crypto performance
> test cases, and graph the results in a consumable manner. The test suites
> are configured via JSON file. Some config files are provided,
> or the user may create one. Currently throughput and latency ptests for
> devices crypto_qat, crypto_aesni_mb and crypto_aesni_gcm are supported.
> 
> The final collection of graphs are output in PDF format, with multiple PDFs
> per test suite, one for each graph type.
> 
> Some fixes are included for the throughput performance test and latency
> performance test csv outputs also.
> 
> v2:
>    - Reduced changes to only fix csv format for all perf test types.
>    - Added functionality for additional args such as config file,
>      output directory and verbose.
>    - Improved help text for script.
>    - Improved script console output.
>    - Added support for latency test cases with burst or buffer size lists.
>    - Split config file into smaller config files, one for each device.
>    - Split output PDFs into smaller files, based on test suite graph types.
>    - Modified output directory naming and structure.
>    - Made some general improvements to script.
>    - Updated and improved documentation.
> 
> Ciara Power (4):
>    test/cryptodev: fix latency test csv output
>    test/cryptodev: fix csv output format
>    usertools: add script to graph crypto perf results
>    maintainers: update crypto perf app maintainers
> 
>   MAINTAINERS                                   |   3 +
>   app/test-crypto-perf/cperf_test_latency.c     |   4 +-
>   .../cperf_test_pmd_cyclecount.c               |   2 +-
>   app/test-crypto-perf/cperf_test_throughput.c  |   4 +-
>   app/test-crypto-perf/cperf_test_verify.c      |   2 +-
>   doc/guides/tools/cryptoperf.rst               | 142 ++++++++
>   usertools/configs/crypto-perf-aesni-gcm.json  |  99 ++++++
>   usertools/configs/crypto-perf-aesni-mb.json   | 108 ++++++
>   usertools/configs/crypto-perf-qat.json        |  94 ++++++
>   usertools/dpdk-graph-crypto-perf.py           | 309 ++++++++++++++++++
>   10 files changed, 761 insertions(+), 6 deletions(-)
>   create mode 100644 usertools/configs/crypto-perf-aesni-gcm.json
>   create mode 100644 usertools/configs/crypto-perf-aesni-mb.json
>   create mode 100644 usertools/configs/crypto-perf-qat.json
>   create mode 100755 usertools/dpdk-graph-crypto-perf.py
>

Series Acked-by: Declan Doherty <declan.doherty@intel.com>
  

> On 14/01/2021 10:41 AM, Ciara Power wrote:
> > This patchset introduces a python script to run various crypto performance
> > test cases, and graph the results in a consumable manner. The test suites
> > are configured via JSON file. Some config files are provided,
> > or the user may create one. Currently throughput and latency ptests for
> > devices crypto_qat, crypto_aesni_mb and crypto_aesni_gcm are supported.
> >
> > The final collection of graphs are output in PDF format, with multiple PDFs
> > per test suite, one for each graph type.
> >
> > Some fixes are included for the throughput performance test and latency
> > performance test csv outputs also.
> >
> > v2:
> >    - Reduced changes to only fix csv format for all perf test types.
> >    - Added functionality for additional args such as config file,
> >      output directory and verbose.
> >    - Improved help text for script.
> >    - Improved script console output.
> >    - Added support for latency test cases with burst or buffer size lists.
> >    - Split config file into smaller config files, one for each device.
> >    - Split output PDFs into smaller files, based on test suite graph types.
> >    - Modified output directory naming and structure.
> >    - Made some general improvements to script.
> >    - Updated and improved documentation.
> >
> > Ciara Power (4):
> >    test/cryptodev: fix latency test csv output
> >    test/cryptodev: fix csv output format
> >    usertools: add script to graph crypto perf results
> >    maintainers: update crypto perf app maintainers
> >
> >   MAINTAINERS                                   |   3 +
> >   app/test-crypto-perf/cperf_test_latency.c     |   4 +-
> >   .../cperf_test_pmd_cyclecount.c               |   2 +-
> >   app/test-crypto-perf/cperf_test_throughput.c  |   4 +-
> >   app/test-crypto-perf/cperf_test_verify.c      |   2 +-
> >   doc/guides/tools/cryptoperf.rst               | 142 ++++++++
> >   usertools/configs/crypto-perf-aesni-gcm.json  |  99 ++++++
> >   usertools/configs/crypto-perf-aesni-mb.json   | 108 ++++++
> >   usertools/configs/crypto-perf-qat.json        |  94 ++++++
> >   usertools/dpdk-graph-crypto-perf.py           | 309 ++++++++++++++++++
> >   10 files changed, 761 insertions(+), 6 deletions(-)
> >   create mode 100644 usertools/configs/crypto-perf-aesni-gcm.json
> >   create mode 100644 usertools/configs/crypto-perf-aesni-mb.json
> >   create mode 100644 usertools/configs/crypto-perf-qat.json
> >   create mode 100755 usertools/dpdk-graph-crypto-perf.py
> >
> 
> Series Acked-by: Declan Doherty <declan.doherty@intel.com>

Applied to dpdk-next-crypto

Thanks.
  

15/01/2021 16:54, Akhil Goyal:
> > On 14/01/2021 10:41 AM, Ciara Power wrote:
> > > Ciara Power (4):
> > >    test/cryptodev: fix latency test csv output
> > >    test/cryptodev: fix csv output format
> > >    usertools: add script to graph crypto perf results
> > >    maintainers: update crypto perf app maintainers
> > >
> > >   MAINTAINERS                                   |   3 +
> > >   app/test-crypto-perf/cperf_test_latency.c     |   4 +-
> > >   .../cperf_test_pmd_cyclecount.c               |   2 +-
> > >   app/test-crypto-perf/cperf_test_throughput.c  |   4 +-
> > >   app/test-crypto-perf/cperf_test_verify.c      |   2 +-
> > >   doc/guides/tools/cryptoperf.rst               | 142 ++++++++
> > >   usertools/configs/crypto-perf-aesni-gcm.json  |  99 ++++++
> > >   usertools/configs/crypto-perf-aesni-mb.json   | 108 ++++++
> > >   usertools/configs/crypto-perf-qat.json        |  94 ++++++
> > >   usertools/dpdk-graph-crypto-perf.py           | 309 ++++++++++++++++++
> > >   10 files changed, 761 insertions(+), 6 deletions(-)
> > >   create mode 100644 usertools/configs/crypto-perf-aesni-gcm.json
> > >   create mode 100644 usertools/configs/crypto-perf-aesni-mb.json
> > >   create mode 100644 usertools/configs/crypto-perf-qat.json
> > >   create mode 100755 usertools/dpdk-graph-crypto-perf.py
> > >
> > 
> > Series Acked-by: Declan Doherty <declan.doherty@intel.com>
> 
> Applied to dpdk-next-crypto

Sorry I missed this series and I discover it when looking at the crypto tree.
I see that the crypto perf script and configs are located in usertools.
I think it should be with the app in app/test-crypto-perf/
The usertools directory is for tools used in production by end users.

Please consider changing the directory for the -rc2.
Thanks
  

> 15/01/2021 16:54, Akhil Goyal:
> > > On 14/01/2021 10:41 AM, Ciara Power wrote:
> > > > Ciara Power (4):
> > > >    test/cryptodev: fix latency test csv output
> > > >    test/cryptodev: fix csv output format
> > > >    usertools: add script to graph crypto perf results
> > > >    maintainers: update crypto perf app maintainers
> > > >
> > > >   MAINTAINERS                                   |   3 +
> > > >   app/test-crypto-perf/cperf_test_latency.c     |   4 +-
> > > >   .../cperf_test_pmd_cyclecount.c               |   2 +-
> > > >   app/test-crypto-perf/cperf_test_throughput.c  |   4 +-
> > > >   app/test-crypto-perf/cperf_test_verify.c      |   2 +-
> > > >   doc/guides/tools/cryptoperf.rst               | 142 ++++++++
> > > >   usertools/configs/crypto-perf-aesni-gcm.json  |  99 ++++++
> > > >   usertools/configs/crypto-perf-aesni-mb.json   | 108 ++++++
> > > >   usertools/configs/crypto-perf-qat.json        |  94 ++++++
> > > >   usertools/dpdk-graph-crypto-perf.py           | 309 ++++++++++++++++++
> > > >   10 files changed, 761 insertions(+), 6 deletions(-)
> > > >   create mode 100644 usertools/configs/crypto-perf-aesni-gcm.json
> > > >   create mode 100644 usertools/configs/crypto-perf-aesni-mb.json
> > > >   create mode 100644 usertools/configs/crypto-perf-qat.json
> > > >   create mode 100755 usertools/dpdk-graph-crypto-perf.py
> > > >
> > >
> > > Series Acked-by: Declan Doherty <declan.doherty@intel.com>
> >
> > Applied to dpdk-next-crypto
> 
> Sorry I missed this series and I discover it when looking at the crypto tree.
> I see that the crypto perf script and configs are located in usertools.
> I think it should be with the app in app/test-crypto-perf/
> The usertools directory is for tools used in production by end users.
> 
> Please consider changing the directory for the -rc2.
> Thanks
> 
Removed from next-crypto for now.