[0/2] Add ABI Version Testing to app/test

  This patchset adds ABI Version Testing functionality to the app/test unit
test framework.

The patchset is intended to address two issues previously raised during ML
conversations on ABI Stability;
1. How do we unit test still supported previous ABI Versions.
2. How to we unit test inline functions from still supported previous ABI
Versions.

The more obvious way to achieve both of the above is to simply archive
pre-built binaries compiled against previous versions of DPDK for use unit
testing previous ABI Versions, and while this should still be done as an
additional check, this approach does not scale well, must every DPDK
developer have a local copy of these binaries to test with, before
upstreaming changes?

Instead starting with rte_lpm, I did the following:-

* I reproduced mostly unmodified unit tests from previous ABI Versions,
  in this case v2.0 and v16.04
* I reproduced the rte_lpm interface header from these previous ABI
  Versions,including the inline functions and remapping symbols to
  appropriate versions.
* I added support for multiple abi versions to the app/test unit test
  framework to allow users to switch between abi versions (set_abi_version),
  without further polluting the already long list of unit tests available in
  app/test.

The intention here is that, in future as developers need to depreciate
APIs, their associated unit tests may move into the ABI Version testing
mechanism of the app/test instead of being replaced by the latest set of
unit tests as would be the case today.

ToDo:
* Refactor the v2.0 and v16.04 unit tests to separate functional and
  performance test cases.
* Add support for trigger ABI Version unit tests from the app/test command
  line.

Ray Kinsella (2):
  app/test: Add ABI Version Testing functionality
  app/test: LPMv4 ABI Version Testing

 app/test/Makefile              |   12 +-
 app/test/commands.c            |  131 ++-
 app/test/meson.build           |    5 +
 app/test/test.c                |    2 +
 app/test/test.h                |   52 +-
 app/test/test_lpm.c            |    1 +
 app/test/test_lpm_perf.c       |  293 +------
 app/test/test_lpm_routes.c     |  287 +++++++
 app/test/test_lpm_routes.h     |   25 +
 app/test/v16.04/dcompat.h      |   23 +
 app/test/v16.04/rte_lpm.h      |  463 +++++++++++
 app/test/v16.04/rte_lpm_neon.h |  119 +++
 app/test/v16.04/rte_lpm_sse.h  |  120 +++
 app/test/v16.04/test_lpm.c     | 1405 ++++++++++++++++++++++++++++++++
 app/test/v16.04/test_v1604.c   |   14 +
 app/test/v2.0/dcompat.h        |   23 +
 app/test/v2.0/rte_lpm.h        |  443 ++++++++++
 app/test/v2.0/test_lpm.c       | 1306 +++++++++++++++++++++++++++++
 app/test/v2.0/test_v20.c       |   14 +
 19 files changed, 4420 insertions(+), 318 deletions(-)
 create mode 100644 app/test/test_lpm_routes.c
 create mode 100644 app/test/test_lpm_routes.h
 create mode 100644 app/test/v16.04/dcompat.h
 create mode 100644 app/test/v16.04/rte_lpm.h
 create mode 100644 app/test/v16.04/rte_lpm_neon.h
 create mode 100644 app/test/v16.04/rte_lpm_sse.h
 create mode 100644 app/test/v16.04/test_lpm.c
 create mode 100644 app/test/v16.04/test_v1604.c
 create mode 100644 app/test/v2.0/dcompat.h
 create mode 100644 app/test/v2.0/rte_lpm.h
 create mode 100644 app/test/v2.0/test_lpm.c
 create mode 100644 app/test/v2.0/test_v20.c
  

Someone kindly approved it, and saved me sending it again from the right
email a/c, thank you.

On 28/05/2019 13:08, Bruce Richardson wrote:
> On Tue, May 28, 2019 at 12:51:56PM +0100, Ray Kinsella wrote:
>> This patchset adds ABI Version Testing functionality to the app/test unit
>> test framework.
>>
>> The patchset is intended to address two issues previously raised during ML
>> conversations on ABI Stability;
>> 1. How do we unit test still supported previous ABI Versions.
>> 2. How to we unit test inline functions from still supported previous ABI
>> Versions.
>>
>> The more obvious way to achieve both of the above is to simply archive
>> pre-built binaries compiled against previous versions of DPDK for use unit
>> testing previous ABI Versions, and while this should still be done as an
>> additional check, this approach does not scale well, must every DPDK
>> developer have a local copy of these binaries to test with, before
>> upstreaming changes?
>>
>> Instead starting with rte_lpm, I did the following:-
>>
>> * I reproduced mostly unmodified unit tests from previous ABI Versions,
>>   in this case v2.0 and v16.04
>> * I reproduced the rte_lpm interface header from these previous ABI
>>   Versions,including the inline functions and remapping symbols to
>>   appropriate versions.
>> * I added support for multiple abi versions to the app/test unit test
>>   framework to allow users to switch between abi versions (set_abi_version),
>>   without further polluting the already long list of unit tests available in
>>   app/test.
>>
>> The intention here is that, in future as developers need to depreciate
>> APIs, their associated unit tests may move into the ABI Version testing
>> mechanism of the app/test instead of being replaced by the latest set of
>> unit tests as would be the case today.
>>
>> ToDo:
>> * Refactor the v2.0 and v16.04 unit tests to separate functional and
>>   performance test cases.
>> * Add support for trigger ABI Version unit tests from the app/test command
>>   line.
>>
> While I admire the goal, given the amount of code that seems to be involved
> here, I'm not sure if the "test" binary is the place to put this. I think
> it might be better as a separate ABI compatiblity test app.

I did think about that also - the test binary, similar to testpmd is
very busy. I sought to mitigate that with set_abi_version command,
mitigating unit test name collisions etc.

I would have a huge concern about putting it into a separate binary, as
these tcs would quickly become forgotten about.

> 
> A separate question is whether this is really necessary to ensure ABI
> compatibility? 

I would argue that we currently have no idea if ABI versioned functions
actually work? Why offer backward compatibility, if we don't test it?

> Do other projects do this? 

The C++ stdlib is many ways similar to DPDK, ABI compatibility is a well
understood problem over there. See the following presentation Slide 20.

https://accu.org/content/conf2015/JonathanWakely-What%20Is%20An%20ABI%20And%20Why%20Is%20It%20So%20Complicated.pdf

They have a small corpus of TC's for this

https://github.com/gcc-mirror/gcc/tree/master/libstdc%2B%2B-v3/testsuite/[backward,abi]

> Is the info from the abi
> compatiblity checker script already in DPDK, or from other
> already-available tools not sufficient?

Well this just tells you that you ABI has changed. We don't have
anything at the moment that tells us that the ABI compatible functions
actually work ... ?

> 
> /Bruce
>