| Message ID | 20210316143253.3849182-1-ciara.power@intel.com (mailing list archive) |
|---|---|
| Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CB355A054F; Tue, 16 Mar 2021 15:33:02 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 87E0D26005B; Tue, 16 Mar 2021 15:33:02 +0100 (CET) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id D89824069F for <dev@dpdk.org>; Tue, 16 Mar 2021 15:33:00 +0100 (CET) IronPort-SDR: lYhqcopjIXg8owFW9v/KPGWbpB2qM8tC7PNgegT4tuPdHenm/4dQQvbgwKUnsdaY8wfB5a0PeN 0ruvVZeuqnIg== X-IronPort-AV: E=McAfee;i="6000,8403,9924"; a="185906428" X-IronPort-AV: E=Sophos;i="5.81,251,1610438400"; d="scan'208";a="185906428" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Mar 2021 07:32:59 -0700 IronPort-SDR: 1aWNfoKumoSWwEB5FlxOARzT1EGe4INMNxa/6coSGyJMcXyjhaZ0g+2omswoKrtL8Eo5TN0Cyz jKKBmyyb5rjw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,251,1610438400"; d="scan'208";a="373805401" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.223.148]) by orsmga006.jf.intel.com with ESMTP; 16 Mar 2021 07:32:58 -0700 From: Ciara Power <ciara.power@intel.com> To: dev@dpdk.org Cc: declan.doherty@intel.com, Ciara Power <ciara.power@intel.com> Date: Tue, 16 Mar 2021 14:32:47 +0000 Message-Id: <20210316143253.3849182-1-ciara.power@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
| Series |
test: refactor crypto unit test framework
|
|
Message
Power, Ciara
March 16, 2021, 2:32 p.m. UTC
The current crypto unit test framework is not granular enough to accurately track unit test results. This is caused by one testcase in a suite actually running multiple testcases, but only returning one result. The approach taken in this patchset allows a test suite have either a list of sub-testsuites, or a list of testcases as previously used. The unit test suite runner can then recursively iterate and run the sub-testsuites, until it reaches a suite with testcases, and it then runs each testcase as it had done previously. By allowing this further breakdown into sub-testsuites, a refactor of the crypto unit tests solves the issue of inaccurate reporting, as sub-testsuites can be used in place of the testcases that had multiple testcases hidden on a sub level. The blockcipher tests previously had these hidden testcases, but are now sub-testsuites that are dynamically created and added to a parent test suite, allowing for each testcase status to be reported directly to the runner. The cryptodev test suite is broken down into smaller suites that are used as sub-testsuites, which allows for more flexibility choosing which sub-testsuites should run for the current device autotest. The introduction of sub-testsuites also allows for more precise setup/teardown functions, rather than general ones loaded with conditions as was seen with the initial setup function used for all crypto testsuites. For example, when running the cryptodev_aesni_mb_autotest, the AESNI MB parent test suite has its own setup function to initialise the AESNI MB device. Various sub-testsuites are added to the parent test suite, such as some of the static suites that were once in the cryptodev_testsuite, and blockcipher suites. The unit test runner can then run the AESNI MB parent test suite, which in turn will run the sub-testsuites. Documentation will be added in a later version of the patchset, adding to the test document that isn't yet merged. [1] --- [1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/ Ciara Power (6): app/test: refactor of unit test suite runner test: introduce parent testsuite format test/crypto: refactor to use sub-testsuites test/crypto: move testsuite params to header file test/crypto: dynamically build blockcipher suite doc: add unit test suite change to release notes app/test/test.c | 168 +- app/test/test.h | 22 +- app/test/test_bitratestats.c | 4 +- app/test/test_compressdev.c | 4 +- app/test/test_cryptodev.c | 2020 ++++++++++++++++++------ app/test/test_cryptodev.h | 20 + app/test/test_cryptodev_asym.c | 105 +- app/test/test_cryptodev_blockcipher.c | 121 +- app/test/test_cryptodev_blockcipher.h | 12 +- app/test/test_ethdev_link.c | 4 +- app/test/test_event_crypto_adapter.c | 4 +- app/test/test_event_eth_rx_adapter.c | 8 +- app/test/test_event_eth_tx_adapter.c | 4 +- app/test/test_event_timer_adapter.c | 4 +- app/test/test_eventdev.c | 4 +- app/test/test_fbarray.c | 4 +- app/test/test_fib.c | 8 +- app/test/test_fib6.c | 8 +- app/test/test_graph.c | 4 +- app/test/test_graph_perf.c | 4 +- app/test/test_ipfrag.c | 4 +- app/test/test_ipsec.c | 36 +- app/test/test_ipsec_sad.c | 4 +- app/test/test_latencystats.c | 4 +- app/test/test_link_bonding.c | 4 +- app/test/test_link_bonding_mode4.c | 4 +- app/test/test_link_bonding_rssconf.c | 4 +- app/test/test_metrics.c | 4 +- app/test/test_pmd_ring.c | 4 +- app/test/test_reorder.c | 4 +- app/test/test_rib.c | 8 +- app/test/test_rib6.c | 8 +- app/test/test_security.c | 4 +- app/test/test_service_cores.c | 4 +- app/test/test_trace.c | 4 +- doc/guides/rel_notes/release_21_05.rst | 5 + 36 files changed, 1898 insertions(+), 739 deletions(-)
Comments
Hey Aaron, based on the work you've been doing on the unit test documentation we would appreciate if you could take a look over this patchset and get your thoughts. The primary drive here is to make it easier to get a clear picture of what is being executed in the cryptodev testsuite, as at the moment there are test suites masquerading as unit tests and the output doesn't reflect the actual number of unit tests being executed. Thanks Declan On 16/03/2021 2:32 PM, Ciara Power wrote: > The current crypto unit test framework is not granular enough to > accurately track unit test results. This is caused by one testcase > in a suite actually running multiple testcases, but only returning > one result. > > The approach taken in this patchset allows a test suite have either a > list of sub-testsuites, or a list of testcases as previously used. > The unit test suite runner can then recursively iterate and run the > sub-testsuites, until it reaches a suite with testcases, > and it then runs each testcase as it had done previously. > > By allowing this further breakdown into sub-testsuites, > a refactor of the crypto unit tests solves the issue of inaccurate > reporting, as sub-testsuites can be used in place of the testcases > that had multiple testcases hidden on a sub level. > The blockcipher tests previously had these hidden testcases, > but are now sub-testsuites that are dynamically created and added to a > parent test suite, allowing for each testcase status to be reported > directly to the runner. > The cryptodev test suite is broken down into smaller suites that are > used as sub-testsuites, which allows for more flexibility choosing which > sub-testsuites should run for the current device autotest. > The introduction of sub-testsuites also allows for more precise > setup/teardown functions, rather than general ones loaded with > conditions as was seen with the initial setup function used for all > crypto testsuites. > > For example, when running the cryptodev_aesni_mb_autotest, > the AESNI MB parent test suite has its own setup function to initialise > the AESNI MB device. > Various sub-testsuites are added to the parent test suite, such as some > of the static suites that were once in the cryptodev_testsuite, > and blockcipher suites. > The unit test runner can then run the AESNI MB parent test suite, > which in turn will run the sub-testsuites. > > Documentation will be added in a later version of the patchset, > adding to the test document that isn't yet merged. [1] > > --- > [1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/ > > Ciara Power (6): > app/test: refactor of unit test suite runner > test: introduce parent testsuite format > test/crypto: refactor to use sub-testsuites > test/crypto: move testsuite params to header file > test/crypto: dynamically build blockcipher suite > doc: add unit test suite change to release notes > > app/test/test.c | 168 +- > app/test/test.h | 22 +- > app/test/test_bitratestats.c | 4 +- > app/test/test_compressdev.c | 4 +- > app/test/test_cryptodev.c | 2020 ++++++++++++++++++------ > app/test/test_cryptodev.h | 20 + > app/test/test_cryptodev_asym.c | 105 +- > app/test/test_cryptodev_blockcipher.c | 121 +- > app/test/test_cryptodev_blockcipher.h | 12 +- > app/test/test_ethdev_link.c | 4 +- > app/test/test_event_crypto_adapter.c | 4 +- > app/test/test_event_eth_rx_adapter.c | 8 +- > app/test/test_event_eth_tx_adapter.c | 4 +- > app/test/test_event_timer_adapter.c | 4 +- > app/test/test_eventdev.c | 4 +- > app/test/test_fbarray.c | 4 +- > app/test/test_fib.c | 8 +- > app/test/test_fib6.c | 8 +- > app/test/test_graph.c | 4 +- > app/test/test_graph_perf.c | 4 +- > app/test/test_ipfrag.c | 4 +- > app/test/test_ipsec.c | 36 +- > app/test/test_ipsec_sad.c | 4 +- > app/test/test_latencystats.c | 4 +- > app/test/test_link_bonding.c | 4 +- > app/test/test_link_bonding_mode4.c | 4 +- > app/test/test_link_bonding_rssconf.c | 4 +- > app/test/test_metrics.c | 4 +- > app/test/test_pmd_ring.c | 4 +- > app/test/test_reorder.c | 4 +- > app/test/test_rib.c | 8 +- > app/test/test_rib6.c | 8 +- > app/test/test_security.c | 4 +- > app/test/test_service_cores.c | 4 +- > app/test/test_trace.c | 4 +- > doc/guides/rel_notes/release_21_05.rst | 5 + > 36 files changed, 1898 insertions(+), 739 deletions(-) >
"Doherty, Declan" <declan.doherty@intel.com> writes: > Hey Aaron, > > based on the work you've been doing on the unit test documentation we > would appreciate if you could take a look over this patchset and get > your thoughts. The primary drive here is to make it easier to get a > clear picture of what is being executed in the cryptodev testsuite, as > at the moment there are test suites masquerading as unit tests and the > output doesn't reflect the actual number of unit tests being executed. I sent some comments - thanks :) > Thanks > Declan > > On 16/03/2021 2:32 PM, Ciara Power wrote: >> The current crypto unit test framework is not granular enough to >> accurately track unit test results. This is caused by one testcase >> in a suite actually running multiple testcases, but only returning >> one result. >> The approach taken in this patchset allows a test suite have >> either a >> list of sub-testsuites, or a list of testcases as previously used. >> The unit test suite runner can then recursively iterate and run the >> sub-testsuites, until it reaches a suite with testcases, >> and it then runs each testcase as it had done previously. >> By allowing this further breakdown into sub-testsuites, >> a refactor of the crypto unit tests solves the issue of inaccurate >> reporting, as sub-testsuites can be used in place of the testcases >> that had multiple testcases hidden on a sub level. >> The blockcipher tests previously had these hidden testcases, >> but are now sub-testsuites that are dynamically created and added to a >> parent test suite, allowing for each testcase status to be reported >> directly to the runner. >> The cryptodev test suite is broken down into smaller suites that are >> used as sub-testsuites, which allows for more flexibility choosing which >> sub-testsuites should run for the current device autotest. >> The introduction of sub-testsuites also allows for more precise >> setup/teardown functions, rather than general ones loaded with >> conditions as was seen with the initial setup function used for all >> crypto testsuites. >> For example, when running the cryptodev_aesni_mb_autotest, >> the AESNI MB parent test suite has its own setup function to initialise >> the AESNI MB device. >> Various sub-testsuites are added to the parent test suite, such as some >> of the static suites that were once in the cryptodev_testsuite, >> and blockcipher suites. >> The unit test runner can then run the AESNI MB parent test suite, >> which in turn will run the sub-testsuites. >> Documentation will be added in a later version of the patchset, >> adding to the test document that isn't yet merged. [1] >> >> --- >> [1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/ >> >> Ciara Power (6): >> app/test: refactor of unit test suite runner >> test: introduce parent testsuite format >> test/crypto: refactor to use sub-testsuites >> test/crypto: move testsuite params to header file >> test/crypto: dynamically build blockcipher suite >> doc: add unit test suite change to release notes >> >> app/test/test.c | 168 +- >> app/test/test.h | 22 +- >> app/test/test_bitratestats.c | 4 +- >> app/test/test_compressdev.c | 4 +- >> app/test/test_cryptodev.c | 2020 ++++++++++++++++++------ >> app/test/test_cryptodev.h | 20 + >> app/test/test_cryptodev_asym.c | 105 +- >> app/test/test_cryptodev_blockcipher.c | 121 +- >> app/test/test_cryptodev_blockcipher.h | 12 +- >> app/test/test_ethdev_link.c | 4 +- >> app/test/test_event_crypto_adapter.c | 4 +- >> app/test/test_event_eth_rx_adapter.c | 8 +- >> app/test/test_event_eth_tx_adapter.c | 4 +- >> app/test/test_event_timer_adapter.c | 4 +- >> app/test/test_eventdev.c | 4 +- >> app/test/test_fbarray.c | 4 +- >> app/test/test_fib.c | 8 +- >> app/test/test_fib6.c | 8 +- >> app/test/test_graph.c | 4 +- >> app/test/test_graph_perf.c | 4 +- >> app/test/test_ipfrag.c | 4 +- >> app/test/test_ipsec.c | 36 +- >> app/test/test_ipsec_sad.c | 4 +- >> app/test/test_latencystats.c | 4 +- >> app/test/test_link_bonding.c | 4 +- >> app/test/test_link_bonding_mode4.c | 4 +- >> app/test/test_link_bonding_rssconf.c | 4 +- >> app/test/test_metrics.c | 4 +- >> app/test/test_pmd_ring.c | 4 +- >> app/test/test_reorder.c | 4 +- >> app/test/test_rib.c | 8 +- >> app/test/test_rib6.c | 8 +- >> app/test/test_security.c | 4 +- >> app/test/test_service_cores.c | 4 +- >> app/test/test_trace.c | 4 +- >> doc/guides/rel_notes/release_21_05.rst | 5 + >> 36 files changed, 1898 insertions(+), 739 deletions(-) >>
> -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of Ciara Power > Sent: Tuesday, March 16, 2021 10:33 PM > To: dev@dpdk.org > Cc: declan.doherty@intel.com; Ciara Power <ciara.power@intel.com> > Subject: [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test framework > > The current crypto unit test framework is not granular enough to accurately > track unit test results. This is caused by one testcase in a suite actually > running multiple testcases, but only returning one result. > > The approach taken in this patchset allows a test suite have either a list of > sub-testsuites, or a list of testcases as previously used. > The unit test suite runner can then recursively iterate and run the sub- > testsuites, until it reaches a suite with testcases, and it then runs each > testcase as it had done previously. > > By allowing this further breakdown into sub-testsuites, a refactor of the > crypto unit tests solves the issue of inaccurate reporting, as sub-testsuites > can be used in place of the testcases that had multiple testcases hidden on a > sub level. > The blockcipher tests previously had these hidden testcases, but are now > sub-testsuites that are dynamically created and added to a parent test suite, > allowing for each testcase status to be reported directly to the runner. > The cryptodev test suite is broken down into smaller suites that are used as > sub-testsuites, which allows for more flexibility choosing which sub- > testsuites should run for the current device autotest. > The introduction of sub-testsuites also allows for more precise > setup/teardown functions, rather than general ones loaded with conditions > as was seen with the initial setup function used for all crypto testsuites. > > For example, when running the cryptodev_aesni_mb_autotest, the AESNI > MB parent test suite has its own setup function to initialise the AESNI MB > device. > Various sub-testsuites are added to the parent test suite, such as some of > the static suites that were once in the cryptodev_testsuite, and blockcipher > suites. > The unit test runner can then run the AESNI MB parent test suite, which in > turn will run the sub-testsuites. > > Documentation will be added in a later version of the patchset, adding to the > test document that isn't yet merged. [1] > > --- > [1] > https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1- > aconole@redhat.com/ > > Ciara Power (6): > app/test: refactor of unit test suite runner > test: introduce parent testsuite format > test/crypto: refactor to use sub-testsuites > test/crypto: move testsuite params to header file > test/crypto: dynamically build blockcipher suite > doc: add unit test suite change to release notes > > app/test/test.c | 168 +- > app/test/test.h | 22 +- > app/test/test_bitratestats.c | 4 +- > app/test/test_compressdev.c | 4 +- > app/test/test_cryptodev.c | 2020 ++++++++++++++++++------ > app/test/test_cryptodev.h | 20 + > app/test/test_cryptodev_asym.c | 105 +- > app/test/test_cryptodev_blockcipher.c | 121 +- > app/test/test_cryptodev_blockcipher.h | 12 +- > app/test/test_ethdev_link.c | 4 +- > app/test/test_event_crypto_adapter.c | 4 +- > app/test/test_event_eth_rx_adapter.c | 8 +- > app/test/test_event_eth_tx_adapter.c | 4 +- > app/test/test_event_timer_adapter.c | 4 +- > app/test/test_eventdev.c | 4 +- > app/test/test_fbarray.c | 4 +- > app/test/test_fib.c | 8 +- > app/test/test_fib6.c | 8 +- > app/test/test_graph.c | 4 +- > app/test/test_graph_perf.c | 4 +- > app/test/test_ipfrag.c | 4 +- > app/test/test_ipsec.c | 36 +- > app/test/test_ipsec_sad.c | 4 +- > app/test/test_latencystats.c | 4 +- > app/test/test_link_bonding.c | 4 +- > app/test/test_link_bonding_mode4.c | 4 +- > app/test/test_link_bonding_rssconf.c | 4 +- > app/test/test_metrics.c | 4 +- > app/test/test_pmd_ring.c | 4 +- > app/test/test_reorder.c | 4 +- > app/test/test_rib.c | 8 +- > app/test/test_rib6.c | 8 +- > app/test/test_security.c | 4 +- > app/test/test_service_cores.c | 4 +- > app/test/test_trace.c | 4 +- > doc/guides/rel_notes/release_21_05.rst | 5 + > 36 files changed, 1898 insertions(+), 739 deletions(-) > > -- > 2.25.1 Tested against armv8crypto PMD and the result looks good. Tested-by: Ruifeng Wang <ruifeng.wang@arm.com>
>-----Original Message----- >From: Ruifeng Wang <Ruifeng.Wang@arm.com> >Sent: Thursday 1 April 2021 04:14 >To: Power, Ciara <ciara.power@intel.com>; dev@dpdk.org >Cc: Doherty, Declan <declan.doherty@intel.com>; nd <nd@arm.com> >Subject: RE: [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test framework > >> -----Original Message----- >> From: dev <dev-bounces@dpdk.org> On Behalf Of Ciara Power >> Sent: Tuesday, March 16, 2021 10:33 PM >> To: dev@dpdk.org >> Cc: declan.doherty@intel.com; Ciara Power <ciara.power@intel.com> >> Subject: [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test >> framework >> >> The current crypto unit test framework is not granular enough to >> accurately track unit test results. This is caused by one testcase in >> a suite actually running multiple testcases, but only returning one result. >> >> The approach taken in this patchset allows a test suite have either a >> list of sub-testsuites, or a list of testcases as previously used. >> The unit test suite runner can then recursively iterate and run the >> sub- testsuites, until it reaches a suite with testcases, and it then >> runs each testcase as it had done previously. >> >> By allowing this further breakdown into sub-testsuites, a refactor of >> the crypto unit tests solves the issue of inaccurate reporting, as >> sub-testsuites can be used in place of the testcases that had multiple >> testcases hidden on a sub level. >> The blockcipher tests previously had these hidden testcases, but are >> now sub-testsuites that are dynamically created and added to a parent >> test suite, allowing for each testcase status to be reported directly to the >runner. >> The cryptodev test suite is broken down into smaller suites that are >> used as sub-testsuites, which allows for more flexibility choosing >> which sub- testsuites should run for the current device autotest. >> The introduction of sub-testsuites also allows for more precise >> setup/teardown functions, rather than general ones loaded with >> conditions as was seen with the initial setup function used for all crypto >testsuites. >> >> For example, when running the cryptodev_aesni_mb_autotest, the AESNI >> MB parent test suite has its own setup function to initialise the >> AESNI MB device. >> Various sub-testsuites are added to the parent test suite, such as >> some of the static suites that were once in the cryptodev_testsuite, >> and blockcipher suites. >> The unit test runner can then run the AESNI MB parent test suite, >> which in turn will run the sub-testsuites. >> >> Documentation will be added in a later version of the patchset, adding >> to the test document that isn't yet merged. [1] >> >> --- >> [1] >> https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536- >1- >> aconole@redhat.com/ >> >> Ciara Power (6): >> app/test: refactor of unit test suite runner >> test: introduce parent testsuite format >> test/crypto: refactor to use sub-testsuites >> test/crypto: move testsuite params to header file >> test/crypto: dynamically build blockcipher suite >> doc: add unit test suite change to release notes >> >> app/test/test.c | 168 +- >> app/test/test.h | 22 +- >> app/test/test_bitratestats.c | 4 +- >> app/test/test_compressdev.c | 4 +- >> app/test/test_cryptodev.c | 2020 ++++++++++++++++++------ >> app/test/test_cryptodev.h | 20 + >> app/test/test_cryptodev_asym.c | 105 +- >> app/test/test_cryptodev_blockcipher.c | 121 +- >> app/test/test_cryptodev_blockcipher.h | 12 +- >> app/test/test_ethdev_link.c | 4 +- >> app/test/test_event_crypto_adapter.c | 4 +- >> app/test/test_event_eth_rx_adapter.c | 8 +- >> app/test/test_event_eth_tx_adapter.c | 4 +- >> app/test/test_event_timer_adapter.c | 4 +- >> app/test/test_eventdev.c | 4 +- >> app/test/test_fbarray.c | 4 +- >> app/test/test_fib.c | 8 +- >> app/test/test_fib6.c | 8 +- >> app/test/test_graph.c | 4 +- >> app/test/test_graph_perf.c | 4 +- >> app/test/test_ipfrag.c | 4 +- >> app/test/test_ipsec.c | 36 +- >> app/test/test_ipsec_sad.c | 4 +- >> app/test/test_latencystats.c | 4 +- >> app/test/test_link_bonding.c | 4 +- >> app/test/test_link_bonding_mode4.c | 4 +- >> app/test/test_link_bonding_rssconf.c | 4 +- >> app/test/test_metrics.c | 4 +- >> app/test/test_pmd_ring.c | 4 +- >> app/test/test_reorder.c | 4 +- >> app/test/test_rib.c | 8 +- >> app/test/test_rib6.c | 8 +- >> app/test/test_security.c | 4 +- >> app/test/test_service_cores.c | 4 +- >> app/test/test_trace.c | 4 +- >> doc/guides/rel_notes/release_21_05.rst | 5 + >> 36 files changed, 1898 insertions(+), 739 deletions(-) >> >> -- >> 2.25.1 > >Tested against armv8crypto PMD and the result looks good. >Tested-by: Ruifeng Wang <ruifeng.wang@arm.com> Thanks for testing this Ruifeng. I have just sent a v2 of the patchset but didn't add the "Tested-by" tag, as it has quite a number of changes. I don't think the changes should have affected armv8crypto PMD, but if you could test with the new version that would be really appreciated. https://patchwork.dpdk.org/project/dpdk/list/?series=16085 Thanks! Ciara
Hi Aaron, >-----Original Message----- >From: Aaron Conole <aconole@redhat.com> >Sent: Wednesday 31 March 2021 15:43 >To: Doherty, Declan <declan.doherty@intel.com> >Cc: Power, Ciara <ciara.power@intel.com>; dev@dpdk.org >Subject: Re: [PATCH 0/6] test: refactor crypto unit test framework > >"Doherty, Declan" <declan.doherty@intel.com> writes: > >> Hey Aaron, >> >> based on the work you've been doing on the unit test documentation we >> would appreciate if you could take a look over this patchset and get >> your thoughts. The primary drive here is to make it easier to get a >> clear picture of what is being executed in the cryptodev testsuite, as >> at the moment there are test suites masquerading as unit tests and the >> output doesn't reflect the actual number of unit tests being executed. > >I sent some comments - thanks :) > Thanks for the review! I have sent a v2 including your suggested changes incorporated. https://patchwork.dpdk.org/project/dpdk/list/?series=16085 >> Thanks >> Declan >> >> On 16/03/2021 2:32 PM, Ciara Power wrote: >>> The current crypto unit test framework is not granular enough to >>> accurately track unit test results. This is caused by one testcase in >>> a suite actually running multiple testcases, but only returning one >>> result. >>> The approach taken in this patchset allows a test suite have either >>> a list of sub-testsuites, or a list of testcases as previously used. >>> The unit test suite runner can then recursively iterate and run the >>> sub-testsuites, until it reaches a suite with testcases, and it then >>> runs each testcase as it had done previously. >>> By allowing this further breakdown into sub-testsuites, a refactor >>> of the crypto unit tests solves the issue of inaccurate reporting, as >>> sub-testsuites can be used in place of the testcases that had >>> multiple testcases hidden on a sub level. >>> The blockcipher tests previously had these hidden testcases, but are >>> now sub-testsuites that are dynamically created and added to a parent >>> test suite, allowing for each testcase status to be reported directly >>> to the runner. >>> The cryptodev test suite is broken down into smaller suites that are >>> used as sub-testsuites, which allows for more flexibility choosing >>> which sub-testsuites should run for the current device autotest. >>> The introduction of sub-testsuites also allows for more precise >>> setup/teardown functions, rather than general ones loaded with >>> conditions as was seen with the initial setup function used for all >>> crypto testsuites. >>> For example, when running the cryptodev_aesni_mb_autotest, the >>> AESNI MB parent test suite has its own setup function to initialise >>> the AESNI MB device. >>> Various sub-testsuites are added to the parent test suite, such as >>> some of the static suites that were once in the cryptodev_testsuite, >>> and blockcipher suites. >>> The unit test runner can then run the AESNI MB parent test suite, >>> which in turn will run the sub-testsuites. >>> Documentation will be added in a later version of the patchset, >>> adding to the test document that isn't yet merged. [1] >>> >>> --- >>> [1] >>> https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536- >1 >>> -aconole@redhat.com/ >>> >>> Ciara Power (6): >>> app/test: refactor of unit test suite runner >>> test: introduce parent testsuite format >>> test/crypto: refactor to use sub-testsuites >>> test/crypto: move testsuite params to header file >>> test/crypto: dynamically build blockcipher suite >>> doc: add unit test suite change to release notes >>> >>> app/test/test.c | 168 +- >>> app/test/test.h | 22 +- >>> app/test/test_bitratestats.c | 4 +- >>> app/test/test_compressdev.c | 4 +- >>> app/test/test_cryptodev.c | 2020 ++++++++++++++++++------ >>> app/test/test_cryptodev.h | 20 + >>> app/test/test_cryptodev_asym.c | 105 +- >>> app/test/test_cryptodev_blockcipher.c | 121 +- >>> app/test/test_cryptodev_blockcipher.h | 12 +- >>> app/test/test_ethdev_link.c | 4 +- >>> app/test/test_event_crypto_adapter.c | 4 +- >>> app/test/test_event_eth_rx_adapter.c | 8 +- >>> app/test/test_event_eth_tx_adapter.c | 4 +- >>> app/test/test_event_timer_adapter.c | 4 +- >>> app/test/test_eventdev.c | 4 +- >>> app/test/test_fbarray.c | 4 +- >>> app/test/test_fib.c | 8 +- >>> app/test/test_fib6.c | 8 +- >>> app/test/test_graph.c | 4 +- >>> app/test/test_graph_perf.c | 4 +- >>> app/test/test_ipfrag.c | 4 +- >>> app/test/test_ipsec.c | 36 +- >>> app/test/test_ipsec_sad.c | 4 +- >>> app/test/test_latencystats.c | 4 +- >>> app/test/test_link_bonding.c | 4 +- >>> app/test/test_link_bonding_mode4.c | 4 +- >>> app/test/test_link_bonding_rssconf.c | 4 +- >>> app/test/test_metrics.c | 4 +- >>> app/test/test_pmd_ring.c | 4 +- >>> app/test/test_reorder.c | 4 +- >>> app/test/test_rib.c | 8 +- >>> app/test/test_rib6.c | 8 +- >>> app/test/test_security.c | 4 +- >>> app/test/test_service_cores.c | 4 +- >>> app/test/test_trace.c | 4 +- >>> doc/guides/rel_notes/release_21_05.rst | 5 + >>> 36 files changed, 1898 insertions(+), 739 deletions(-) >>>