[V1,0/2] Rewrite test suite for checksum_offload

Message ID 20230712203334.2032445-1-ke1.xu@intel.com (mailing list archive)


Ke Xu July 12, 2023, 8:33 p.m. UTC
  Checksum offload cases were in bad situation. Cases are not well organized
 in structure. Cases were added one by one along with the development of

The very beginning cases are designed to verifying RX Checksum Offload
 enabling / disabling. The followed cases are to verify the TX Offload, and
 found a gap when TX introduces differences to IP and L4. Then IP and L4 RX
 cases are added again. There were also a gap found on the flag verifying
 and a seperate ol_flag case was added.

To better organizing the checksum offload cases. I introduced the scenario
 based common cases and specific cases named with sufix to cover specific
 scenarios of special requirements. And we placed all available test scopes
 introduced in any existing cases to each of our common test execution to
 ensure no leakage to our test scopes.

For scenarios, we introduce the very commonly used configurations for
 checksum as our test scenarios. These scenarios cover combined usage of HW
 checksum Offload, SW checksum and differently configured tunneling scenario,
 w/wo VLAN. These cases formed a organized web for future cases that if a new
 configuration is introduced, cases of new configuration combinations are to
 be added but existing configurations for different scopes are not to be added.

For test scopes, we introduce verifying ol_flag, fwd stats, other verbose lines,
 and forwarded packet capturing and analyzing. All these scopes are recorder for
 verifying in each common checksum offload execution. This ensures no scope would
 be missed in this rewrite and in future case updating.

The newly organized cases also helps better covering our existing test
 requirements. A common packet sending and recording helps cut down the repeating
 time cost of different executions for scope based redundant cases.

This rewrite also helps formating a data-process-seperated suite structure. The
 test parameters and required packets are listed in front and cases are referring
 to the configured parameters. This helps better reading the suite code.

In the test plan, new cases are added and old ones are merged with the new cases.
 Packets, ol_flags, stats and related test scopes are listed in table to show where
 the expected result is from.

Generally speaking, this case rewrite helps formating the case structure and helps
 better maintaining the cases in future.

Ke Xu (2):
  test_plans/checksum_offload: Refactor checksum_offload case.
  tests/checksum_offload: Refactor checksum_offload case.

 test_plans/checksum_offload_test_plan.rst | 1478 +++++++----------
 tests/TestSuite_checksum_offload.py       | 1766 +++++++++++----------
 2 files changed, 1549 insertions(+), 1695 deletions(-)