[v2,5/6] dts: add conditional behavior for test suite
Checks
Commit Message
There is some odd functionality/behavior in how the --test-suite
parameters interacts in conjunction with the 'test_suites' attribute in
the config file. If a user leaves an empty list underneath
'test_suites,' or if they negate the attribute entirely, even if said
user adds test suites via the --test-suite parameter, a schema violation
is thrown.
This patch mitigates this, by removing the schema requirement if the
user has indicated test suites within main.py parameters, allowing for
the 'test_suites' attribute to be optional.
Bugzilla ID: 1360
Signed-off-by: Nicholas Pratte <npratte@iol.unh.edu>
---
dts/framework/config/__init__.py | 7 ++++++-
dts/framework/runner.py | 2 +-
2 files changed, 7 insertions(+), 2 deletions(-)
Comments
I think it makes sense to allow users to not specify any test suites
in the config file if they specify them through other means, so I like
this change.
On Fri, Jul 5, 2024 at 1:20 PM Nicholas Pratte <npratte@iol.unh.edu> wrote:
>
> There is some odd functionality/behavior in how the --test-suite
> parameters interacts in conjunction with the 'test_suites' attribute in
> the config file. If a user leaves an empty list underneath
> 'test_suites,' or if they negate the attribute entirely, even if said
> user adds test suites via the --test-suite parameter, a schema violation
> is thrown.
>
> This patch mitigates this, by removing the schema requirement if the
> user has indicated test suites within main.py parameters, allowing for
> the 'test_suites' attribute to be optional.
>
> Bugzilla ID: 1360
> Signed-off-by: Nicholas Pratte <npratte@iol.unh.edu>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
On 5. 7. 2024 19:13, Nicholas Pratte wrote:
> There is some odd functionality/behavior in how the --test-suite
> parameters interacts in conjunction with the 'test_suites' attribute in
> the config file. If a user leaves an empty list underneath
> 'test_suites,' or if they negate the attribute entirely, even if said
> user adds test suites via the --test-suite parameter, a schema violation
> is thrown.
>
> This patch mitigates this, by removing the schema requirement if the
> user has indicated test suites within main.py parameters, allowing for
> the 'test_suites' attribute to be optional.
>
Nice idea, doesn't look like there's any hard in adding it. Should we
document this (the fact that it could be optional under certain
circumstances) somewhere, like add something to the description in schema?
> Bugzilla ID: 1360
> Signed-off-by: Nicholas Pratte <npratte@iol.unh.edu>
> ---
> dts/framework/config/__init__.py | 7 ++++++-
> dts/framework/runner.py | 2 +-
> 2 files changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py
> index ed1c979fb6..82182b5c99 100644
> --- a/dts/framework/config/__init__.py
> +++ b/dts/framework/config/__init__.py
> @@ -553,7 +553,7 @@ def from_dict(cls, d: ConfigurationDict) -> Self:
> return cls(test_runs=test_runs)
>
>
> -def load_config(config_file_path: Path) -> Configuration:
> +def load_config(config_file_path: Path, test_suites: list[TestSuiteConfig]) -> Configuration:
The variable should maybe contain that these test suites are not from
config_file_path. Maybe just rename to other_test_suites? Or something
like that.
Also, it needs to be added to Args:
@@ -553,7 +553,7 @@ def from_dict(cls, d: ConfigurationDict) -> Self:
return cls(test_runs=test_runs)
-def load_config(config_file_path: Path) -> Configuration:
+def load_config(config_file_path: Path, test_suites: list[TestSuiteConfig]) -> Configuration:
"""Load DTS test run configuration from a file.
Load the YAML test run configuration file
@@ -576,6 +576,11 @@ def load_config(config_file_path: Path) -> Configuration:
with open(schema_path, "r") as f:
schema = json.load(f)
+ if test_suites:
+ schema["properties"]["test_runs"]["items"]["required"].remove("test_suites")
+ for test_run in config_data["test_runs"]:
+ if not hasattr(test_run, "test_suites"):
+ test_run["test_suites"] = []
config = warlock.model_factory(schema, name="_Config")(config_data)
config_obj: Configuration = Configuration.from_dict(dict(config)) # type: ignore[arg-type]
return config_obj
@@ -85,7 +85,7 @@ class DTSRunner:
def __init__(self):
"""Initialize the instance with configuration, logger, result and string constants."""
- self._configuration = load_config(SETTINGS.config_file_path)
+ self._configuration = load_config(SETTINGS.config_file_path, SETTINGS.test_suites)
self._logger = get_dts_logger()
if not os.path.exists(SETTINGS.output_dir):
os.makedirs(SETTINGS.output_dir)