[RFC] app/test: allow passing a parameter string to autotests
Checks
Commit Message
Sometimes it can be nice to have autotests which can take a parameter,
or can be tweaked in some ways, e.g. adjust the number of iterations, or
the burst size used in the test. Currently there is no way to do so -
all test parameters are hardcoded, which makes sense for a generic
regression test to be run quickly, but is a bit lacking for those
looking to use unit tests for investigations.
To that end, we can all the commandline to accept an additional string
parameter after each autotest name, and make that available via global
variable. Any test which wishes to make use of this can do so, to allow
overriding test defaults.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/commands.c | 33 ++++++++++++++++++++++++++-
app/test/test.h | 2 ++
doc/guides/contributing/unit_test.rst | 15 ++++++++++++
3 files changed, 49 insertions(+), 1 deletion(-)
Comments
> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Friday, 15 December 2023 14.07
>
> Sometimes it can be nice to have autotests which can take a parameter,
> or can be tweaked in some ways, e.g. adjust the number of iterations,
> or
> the burst size used in the test. Currently there is no way to do so -
> all test parameters are hardcoded, which makes sense for a generic
> regression test to be run quickly, but is a bit lacking for those
> looking to use unit tests for investigations.
>
> To that end, we can all the commandline to accept an additional string
> parameter after each autotest name, and make that available via global
> variable. Any test which wishes to make use of this can do so, to allow
> overriding test defaults.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
This would be useful for the mempool perf test. :-)
It would be nice standardizing on a parameter syntax, though.
Preferably: <param> ["=" <value>] {" " <param> ["=" <value>]}
Acked-by: Morten Brørup <mb@smartsharesystems.com>
On Fri, 15 Dec 2023 13:06:56 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:
> Sometimes it can be nice to have autotests which can take a parameter,
> or can be tweaked in some ways, e.g. adjust the number of iterations, or
> the burst size used in the test. Currently there is no way to do so -
> all test parameters are hardcoded, which makes sense for a generic
> regression test to be run quickly, but is a bit lacking for those
> looking to use unit tests for investigations.
>
> To that end, we can all the commandline to accept an additional string
> parameter after each autotest name, and make that available via global
> variable. Any test which wishes to make use of this can do so, to allow
> overriding test defaults.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
LGTM
Maybe add an example to an existing test
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
@@ -40,6 +40,8 @@
#include "test.h"
+char *test_parameter = NULL;
+
/****************/
static struct test_commands_list commands_list =
@@ -55,14 +57,25 @@ struct cmd_autotest_result {
cmdline_fixed_string_t autotest;
};
+struct cmd_autotest_param_result {
+ cmdline_fixed_string_t autotest;
+ cmdline_fixed_string_t param;
+};
+
static void cmd_autotest_parsed(void *parsed_result,
__rte_unused struct cmdline *cl,
- __rte_unused void *data)
+ void *data)
{
struct test_command *t;
struct cmd_autotest_result *res = parsed_result;
int ret = 0;
+ test_parameter = NULL;
+ if (data == (void *)1) {
+ struct cmd_autotest_param_result *pres = parsed_result;
+ test_parameter = pres->param;
+ }
+
TAILQ_FOREACH(t, &commands_list, next) {
if (!strcmp(res->autotest, t->command))
ret = t->callback();
@@ -92,6 +105,22 @@ cmdline_parse_inst_t cmd_autotest = {
},
};
+cmdline_parse_token_string_t cmd_autotest_param_autotest =
+ TOKEN_STRING_INITIALIZER(struct cmd_autotest_param_result, autotest, "");
+cmdline_parse_token_string_t cmd_autotest_param_param =
+ TOKEN_STRING_INITIALIZER(struct cmd_autotest_param_result, param, NULL);
+
+cmdline_parse_inst_t cmd_autotest_with_param = {
+ .f = cmd_autotest_parsed,
+ .data = (void *)1,
+ .help_str = "launch autotest, providing a string parameter",
+ .tokens = {
+ (void *)&cmd_autotest_param_autotest,
+ (void *)&cmd_autotest_param_param,
+ NULL,
+ },
+};
+
/****************/
struct cmd_dump_result {
@@ -348,6 +377,7 @@ cmdline_parse_inst_t cmd_set_rxtx_sc = {
cmdline_parse_ctx_t main_ctx[] = {
(cmdline_parse_inst_t *)&cmd_autotest,
+ (cmdline_parse_inst_t *)&cmd_autotest_with_param,
(cmdline_parse_inst_t *)&cmd_dump,
(cmdline_parse_inst_t *)&cmd_dump_one,
(cmdline_parse_inst_t *)&cmd_quit,
@@ -378,5 +408,6 @@ int commands_init(void)
}
cmd_autotest_autotest.string_data.str = commands;
+ cmd_autotest_param_autotest.string_data.str = commands;
return 0;
}
@@ -27,6 +27,8 @@
#include <rte_test.h>
+extern char *test_parameter;
+
#define TEST_ASSERT RTE_TEST_ASSERT
#define TEST_ASSERT_EQUAL RTE_TEST_ASSERT_EQUAL
@@ -374,6 +374,21 @@ Example::
...
+Optional test parameters
+-------------------------
+
+Each unit test suite must be runable without any additional test parameters,
+since automated testing via ``meson test``, and many DPDK Continuous Integration(CI) frameworks,
+will just call each test via cmdline-parameter to the ``dpdk-test`` binary
+or by setting the test name in the ``DPDK_TEST`` environment variable.
+
+However, when run interactively via ``dpdk-test`` CLI,
+an additional parameter may be passed to the autotest command.
+This parameter will be available to the test in the ``test_parameter`` global variable.
+This support is intended for developer use only as a convenience.
+It is up to the each test how to use this parameter and what form it may have.
+
+
Checking code coverage
----------------------