[v1,0/4,RFC] Testpmd RPC API

  From: Owen Hilyard <ohilyard@iol.unh.edu>

    Currently, DTS uses Testpmd for most of its testing. This has been successful in reducing the need to create more test apps, but it has a few drawbacks. First, if some part of DPDK is not exposed via Testpmd or one of the example applications, for the purposes of DTS it is not testable. This is a situation I’d like to avoid. However, adding new functionality to Testpmd is labor-intensive. Testpmd currently uses a hand-written LL(1) parser (https://en.wikipedia.org/wiki/LL_parser) to parse command line options. This makes adding new functionality difficult since the parser is stored as a series of several thousand line long lookup tables. To look at it another way, 64% of the 52238 lines in Testpmd are related to command line input in some way. The command line interface of testpmd also presents several challenges for the underlying implementation, since it requires that everything a user might want to reference is identified via something that is reasonable to ask a user to type. As of right now, this is handled via either strings or integers. This can be handled by creating a global registry for objects, but it is still extra work that I think can be avoided. In addition, this leads to more places where things can go wrong. 

This is what DTS running a single command in testpmd looks like right now:
https://drive.google.com/file/d/1hvTcjfVdh8-I3CUNoq6bx82EuNQSK6qW/view?usp=sharing

    This approach has a number of disadvantages. First, it requires assembling all commands as strings inside of the test suite and sending them through a full round trip of SSH. This means that any non-trivial command, such as creating an RTE flow, will involve a lot of string templating. This normally wouldn’t be a big issue, except that some of the test suites are designed to hundreds of commands over the course of a test, paying the cost of an SSH round trip for each. Once Testpmd has the commands, it will then call the appropriate functions inside of DPDK, and then print out all of the state to standard out. All of this is sent back to DTS, where the author of the test case then needs to handle all possible outputs of Trex, often by either declaring the presence of a single word or short phrase in the output as meaning success or failure. In my opinion, this is something that is perfectly fine for humans to interact with, but it causes a lot of issues with automation due to its inherent inflexibility and the less-than-ideal methods of information transfer. This is why I am proposing the creation of an automation-oriented pmd, with a focus on exposing as much.

https://drive.google.com/file/d/1wj4-RnFPVERCzM8b68VJswAOEI9cg-X8/view?usp=sharing 

	That diagram is a high-level overview of the design, which explicitly excludes implementation details. However, it already has some benefits. First, making DPDK do something is a normal method call, instead of needing to format things into a string. This provides a much better interface for people working in both DTS and DPDK. Second, the ability to return structured data means that there won’t be parsers on both sides of communication anymore. Structured data also allows much more verbosity, since it is no longer an interface designed for humans. If a test case author needs to return the bytes of every received packet back to DTS for comparison with the expected value, they can. If you need to return a pointer for DTS to use later, that becomes reasonable. Simply moving to shuffling structured data around and using RPC already provides a lot of benefits. 
	The next obvious question would be what to use for the implementation. The initial attempt was made using Python on both sides and the standard library xmlrpc module. The RPC aspect of this approach worked very well, with the ability to send arbitrary python objects back and forth between DTS and app. However, having Python interacting with DPDK has a few issues. First, DPDK is generally very multi-threaded and the most common implementation of Python, CPython, does not have concurrency. It has something known as the global interpretr lock, which is a global mutex. This makes it very difficult to interact with blocking, multi-threaded code. The other issue is that I was not able to find a binding generator that I feel would be sufficient for DPDK. Many generators assumed sizeof(int) == 4 or had other portability issues such as assuming GCC or Clang as a C compiler. Others focused on some subset of C, meaning they would throw errors on alignment annotations. 
    Given this, I decided to look for cross-language RPC libraries. Although libraries exist for performing xmlrpc in C, they generally appeared quite difficult to use and required a lot of manual work. The next best option was gRPC. gRPC allows using a simple language, protobuf, with a language extension for rpc. It provides code generation to make it easy to use multiple languages together, since it was developed to make polyglot microservice interaction easier. The only drawback is that it considers C++ good enough for C support. In this case, I was able to easily integrate DPDK with C++, so that isn’t much of a concern. I used C++17 in the attached patches, but the minimum requirements are C++11. If there is concern about modern C++ causing too much mental overhead, a “C with classes” subset of C++ could easily be used. I also added an on-by-default option to use a C++ compiler, allowing anyone who does not have a C++ compiler available to them to turn off everything that uses C++. This disables the application written for this RFC.
    One of the major benefits of gRPC is the asynchronous API. This allows streaming data on both sides of an RPC call. This allows streaming logs back to DTS, streaming large amounts of data from low-memory systems back to DTS for processing, and would allow DTS to easily feed DPDK data, ending the test quickly on a failure. Currently, due to the overhead of sending data to Testpmd, it is common to just send all of the commands over and run everything since that will be much faster when the test passes, but it can cost a lot of time in the event of a failure. There are also optional security features for requiring authentication before allowing code execution. I think that a discussion on whether using them for DTS is necessary is warranted, although I personally think that it’s not worth the effort given the type of environment this sample application is expected to run in. 
    For this RFC, I ported test-acl because it was mostly self-contained and was something I could run on my laptop. It should be fairly easy to see how you would expand this proof of concept to cover more of DPDK, and I think that most of the functions currently used in testpmd could be ported over to this approach, saving a lot of development time. However, I would like to see some more interest before I take on a task like that. This will require a lot of work on the DTS side to implement, but it will make it much easier to add new features to DTS. 

Owen Hilyard (4):
  app/test-pmd-api: Add C++ Compiler
  app/test-pmd-api: Add POC with gRPC deps
  app/test-pmd-api: Add protobuf file
  app/test-pmd-api: Implementation files for the API

 app/meson.build              |   17 +
 app/test-pmd-api/api.proto   |   12 +
 app/test-pmd-api/api_impl.cc | 1160 ++++++++++++++++++++++++++++++++++
 app/test-pmd-api/api_impl.h  |   10 +
 app/test-pmd-api/main.c      |   11 +
 app/test-pmd-api/meson.build |   96 +++
 meson.build                  |    3 +
 meson_options.txt            |    2 +
 8 files changed, 1311 insertions(+)
 create mode 100644 app/test-pmd-api/api.proto
 create mode 100644 app/test-pmd-api/api_impl.cc
 create mode 100644 app/test-pmd-api/api_impl.h
 create mode 100644 app/test-pmd-api/main.c
 create mode 100644 app/test-pmd-api/meson.build
  

On Fri, Apr 8, 2022 at 3:17 AM <ohilyard@iol.unh.edu> wrote:
>
> From: Owen Hilyard <ohilyard@iol.unh.edu>
>
>     Currently, DTS uses Testpmd for most of its testing. This has been successful in reducing the need to create more test apps, but it has a few drawbacks. First, if some part of DPDK is not exposed via Testpmd or one of the example applications, for the purposes of DTS it is not testable. This is a situation I’d like to avoid. However, adding new functionality to Testpmd is labor-intensive. Testpmd currently uses a hand-written LL(1) parser (https://en.wikipedia.org/wiki/LL_parser) to parse command line options. This makes adding new functionality difficult since the parser is stored as a series of several thousand line long lookup tables. To look at it another way, 64% of the 52238 lines in Testpmd are related to command line input in some way. The command line interface of testpmd also presents several challenges for the underlying implementation, since it requires that everything a user might want to reference is identified via something that is reasonable to ask a user to type. As of right now, this is handled via either strings or integers. This can be handled by creating a global registry for objects, but it is still extra work that I think can be avoided. In addition, this leads to more places where things can go wrong.
>
> This is what DTS running a single command in testpmd looks like right now:
> https://drive.google.com/file/d/1hvTcjfVdh8-I3CUNoq6bx82EuNQSK6qW/view?usp=sharing
>
>     This approach has a number of disadvantages. First, it requires assembling all commands as strings inside of the test suite and sending them through a full round trip of SSH. This means that any non-trivial command, such as creating an RTE flow, will involve a lot of string templating. This normally wouldn’t be a big issue, except that some of the test suites are designed to hundreds of commands over the course of a test, paying the cost of an SSH round trip for each. Once Testpmd has the commands, it will then call the appropriate functions inside of DPDK, and then print out all of the state to standard out. All of this is sent back to DTS, where the author of the test case then needs to handle all possible outputs of Trex, often by either declaring the presence of a single word or short phrase in the output as meaning success or failure. In my opinion, this is something that is perfectly fine for humans to interact with, but it causes a lot of issues with automation due to its inherent inflexibility and the less-than-ideal methods of information transfer. This is why I am proposing the creation of an automation-oriented pmd, with a focus on exposing as much.
>
> https://drive.google.com/file/d/1wj4-RnFPVERCzM8b68VJswAOEI9cg-X8/view?usp=sharing
>
>         That diagram is a high-level overview of the design, which explicitly excludes implementation details. However, it already has some benefits. First, making DPDK do something is a normal method call, instead of needing to format things into a string. This provides a much better interface for people working in both DTS and DPDK. Second, the ability to return structured data means that there won’t be parsers on both sides of communication anymore. Structured data also allows much more verbosity, since it is no longer an interface designed for humans. If a test case author needs to return the bytes of every received packet back to DTS for comparison with the expected value, they can. If you need to return a pointer for DTS to use later, that becomes reasonable. Simply moving to shuffling structured data around and using RPC already provides a lot of benefits.
>         The next obvious question would be what to use for the implementation. The initial attempt was made using Python on both sides and the standard library xmlrpc module. The RPC aspect of this approach worked very well, with the ability to send arbitrary python objects back and forth between DTS and app. However, having Python interacting with DPDK has a few issues. First, DPDK is generally very multi-threaded and the most common implementation of Python, CPython, does not have concurrency. It has something known as the global interpretr lock, which is a global mutex. This makes it very difficult to interact with blocking, multi-threaded code. The other issue is that I was not able to find a binding generator that I feel would be sufficient for DPDK. Many generators assumed sizeof(int) == 4 or had other portability issues such as assuming GCC or Clang as a C compiler. Others focused on some subset of C, meaning they would throw errors on alignment annotations.
>     Given this, I decided to look for cross-language RPC libraries. Although libraries exist for performing xmlrpc in C, they generally appeared quite difficult to use and required a lot of manual work. The next best option was gRPC. gRPC allows using a simple language, protobuf, with a language extension for rpc. It provides code generation to make it easy to use multiple languages together, since it was developed to make polyglot microservice interaction easier. The only drawback is that it considers C++ good enough for C support. In this case, I was able to easily integrate DPDK with C++, so that isn’t much of a concern. I used C++17 in the attached patches, but the minimum requirements are C++11. If there is concern about modern C++ causing too much mental overhead, a “C with classes” subset of C++ could easily be used. I also added an on-by-default option to use a C++ compiler, allowing anyone who does not have a C++ compiler available to them to turn off everything that uses C++. This disables the application written for this RFC.
>     One of the major benefits of gRPC is the asynchronous API. This allows streaming data on both sides of an RPC call. This allows streaming logs back to DTS, streaming large amounts of data from low-memory systems back to DTS for processing, and would allow DTS to easily feed DPDK data, ending the test quickly on a failure. Currently, due to the overhead of sending data to Testpmd, it is common to just send all of the commands over and run everything since that will be much faster when the test passes, but it can cost a lot of time in the event of a failure. There are also optional security features for requiring authentication before allowing code execution. I think that a discussion on whether using them for DTS is necessary is warranted, although I personally think that it’s not worth the effort given the type of environment this sample application is expected to run in.
>     For this RFC, I ported test-acl because it was mostly self-contained and was something I could run on my laptop. It should be fairly easy to see how you would expand this proof of concept to cover more of DPDK, and I think that most of the functions currently used in testpmd could be ported over to this approach, saving a lot of development time. However, I would like to see some more interest before I take on a task like that. This will require a lot of work on the DTS side to implement, but it will make it much easier to add new features to DTS.


Thanks, Owen for POC.

In my view, May using this scheme is probably over-engineered. The
reason for thinking so is
-Now that, Test code is also part of DPDK, Exposing as services may
not be required.
-Now in DPDK, we have two types of existing test cases to verify the API
-- Noninteractive  - These test cases, can simply run over ssh with
bash invocation and return the test from a remote PC,
-- Interactive - Testpmd one, I believe, Feeding stdin
programmatically would suffice to test all the combinations.
-We need to add all test cases in this model and we need to maintain
two sets of programs.(Traditional tests and gRPC model-based tests).

Just my 2c.


>
> Owen Hilyard (4):
>   app/test-pmd-api: Add C++ Compiler
>   app/test-pmd-api: Add POC with gRPC deps
>   app/test-pmd-api: Add protobuf file
>   app/test-pmd-api: Implementation files for the API
>
>  app/meson.build              |   17 +
>  app/test-pmd-api/api.proto   |   12 +
>  app/test-pmd-api/api_impl.cc | 1160 ++++++++++++++++++++++++++++++++++
>  app/test-pmd-api/api_impl.h  |   10 +
>  app/test-pmd-api/main.c      |   11 +
>  app/test-pmd-api/meson.build |   96 +++
>  meson.build                  |    3 +
>  meson_options.txt            |    2 +
>  8 files changed, 1311 insertions(+)
>  create mode 100644 app/test-pmd-api/api.proto
>  create mode 100644 app/test-pmd-api/api_impl.cc
>  create mode 100644 app/test-pmd-api/api_impl.h
>  create mode 100644 app/test-pmd-api/main.c
>  create mode 100644 app/test-pmd-api/meson.build
>
> --
> 2.30.2
>
  

>
> scheme is probably over-engineered


I tried my hardest to keep this as simple as possible. The requirements
imposed by DTS being a distributed system in Python restricted what I could
do a lot. Needing to be compatible with DPDK's license also got rid of a
lot of options. Binding generators are made for simple projects, and DPDK
is not a simple project. There were some other options related to choice in
the RPC framework, but very few RPC protocols seem to work well with C and
be usable from Python, which is why I ended up using C++ with gRPC. Most of
the code in api_impl.cc is taken from /app/test-acl/main.c, and the new
part is mostly the C++ class at the bottom. Overall, this proposal comes
out to ~100 lines of new C++, 9 lines of C, 12 lines of gRPC Protobuf and
100 lines of Meson. gRPC may be able to do a lot more than I need it to for
the proof of concept, but many of the features that are not used, like
bi-directional streaming, become very useful in writing more complicated
tests. Overall, this solution is probably more capable than we need it to
be, but I think that those extra capabilities don't come at a very large
cost.


> Now that, Test code is also part of DPDK.
>

DTS is pure python. I tried to use FFI to call directly into DPDK from
Python and then use xmlrpc from the python standard library. As mentioned
in the writeup, I couldn't find a binding generator that would properly
handle DPDK's allocators, which made it so that anything passed to DPDK
using python was allocated using the system malloc. I don't think it is
wise to attempt to programmatically re-write the generated code to allow
for custom allocators. The original reason for needing to have DTS and DPDK
in the same repository was so that tests could be committed and run
alongside the feature patch.

Interactive - Testpmd one, I believe, Feeding stdin programmatically would
> suffice to test all the combinations.
>

One of the issues this is trying to address is that human-readable strings
are a poor way to pass complex information between two programs. DTS is a
distributed system, and it can have up to 3 physical servers involved in
any given test. This means that it's not stdin via a pipe, it's an entire
SSH session. This adds a noticeable amount of overhead when trying to send
and verify the result of sending 1,000+ packets, since the lack of
structured output means each packet must be checked before the next can be
sent. This might be solvable by adding a structured output mode to testpmd,
but that would involve committing to writing output twice for every
function in testpmd forever.

We need to add all test cases in this model and we need to maintain two
> sets of programs.(Traditional tests and gRPC model-based tests).
>

Assuming by traditional tests you mean the unit tests run by Meson, I would
argue that we are already maintaining 2 kinds of tests. The unit tests, and
the python-based DTS tests. My intention is to create a thin wrapper around
DPDK that would be exposed via gRPC, like you see here, and use that as
midware. Then, we would have two front-ends. Testpmd, which takes text and
then calls midware as it does now, and the gRPC frontend, which parses
messages from the RPC server and runs the midware. This would enable
testpmd to still be used to sanity check a DPDK installation, but we would
not need to continually expand Testpmd. The primary issue is that, right
now, anything not included in Testpmd is not really testable by DTS. This
includes portions of the RTE Flow API, which was part of my reason for
proposing this. The RTE Flow API would, in my estimation, if fully
implemented into Testpmd, probably add at least another 10,000 lines of
code. As mentioned in my proposal, Testpmd already does more parsing and
lexing than it does interaction with DPDK by line count. Also, since I am
proposing making this a separate application, we would be able to gradually
migrate the tests inside of DTS. This would have no effect on anything
except for Testpmd, the new application and the addition of a flag to
toggle the use of a C++ compiler.

I'm not sure exactly what you mean by gRPC model-based tests. gRPC uses
classes to model services, but for this usecase we are essentially using it
to transfer function arguments across the internet and then pass the return
value back. Any RPC framework would function similarly if I ignored the
restrictions of which languages to use, and the choice is not important to
how tests are conducted. Put another way, how you write a test for DTS will
not change much if you are using this or testpmd, it's just how you
transfer data and get it back that I want to change.
  

On Mon, Apr 11, 2022 at 11:19 PM Owen Hilyard <ohilyard@iol.unh.edu> wrote:
>>
>> scheme is probably over-engineered
>
>
> I tried my hardest to keep this as simple as possible. The requirements imposed by DTS being a distributed system in Python restricted what I could do a lot. Needing to be compatible with DPDK's license also got rid of a lot of options. Binding generators are made for simple projects, and DPDK is not a simple project. There were some other options related to choice in the RPC framework, but very few RPC protocols seem to work well with C and be usable from Python, which is why I ended up using C++ with gRPC. Most of the code in api_impl.cc is taken from /app/test-acl/main.c, and the new part is mostly the C++ class at the bottom. Overall, this proposal comes out to ~100 lines of new C++, 9 lines of C, 12 lines of gRPC Protobuf and 100 lines of Meson. gRPC may be able to do a lot more than I need it to for the proof of concept, but many of the features that are not used, like bi-directional streaming, become very useful in writing more complicated tests. Overall, this solution is probably more capable than we need it to be, but I think that those extra capabilities don't come at a very large cost.


Now it is clear, I was carried away with the POC test application and
I was not knowing existing DTS tests are based on python

Is below a fair summary?

1) DPDK has interactive test cases and no interactive test cases.

For The interactive test case like testpmd, I agree that we can enable
RPC service via gRPC server in C++ as  and client in Python, and
something along the lines of exposing the existing test-pmd command
line function as service
to avoid command line parsing and reuse the existing python test suite.

If so, I think, gRPC service would be along with existing testpmd
functions, like start_packet_forwarding(). Also, We don't need to
rewrite the existing testpmd,
Instead, RPC service, we can add in existing app/test-pmd/ and hook to
existing core testpmd functions to bypass the command-line parsing in
C and control from python client as needed as service.

Also, I agree that pulling in gRPC C++ server boilerplate and hooking
to C functions is a good idea as it is the best C-based RPC scheme
available today.

2)I think, DPDK has only one interactive test case which is testpmd,
Remaining test cases are non-interactive, non-interactive test cases
can simply run over ssh with passwordless login. Right?
Do we need gRPC for that? Will the following scheme suffice? If not,
How you are planning to do noninteractive test cases?
i.e
a)Copy test to target
b) result=`ssh username@IP /path/to/testapp/in/target`

I think, key should be leveraging existing test cases as much as
possible and make easy for developers to add new test cases.


>>
>> Now that, Test code is also part of DPDK.
>
>
> DTS is pure python. I tried to use FFI to call directly into DPDK from Python and then use xmlrpc from the python standard library. As mentioned in the writeup, I couldn't find a binding generator that would properly handle DPDK's allocators, which made it so that anything passed to DPDK using python was allocated using the system malloc. I don't think it is wise to attempt to programmatically re-write the generated code to allow for custom allocators. The original reason for needing to have DTS and DPDK in the same repository was so that tests could be committed and run alongside the feature patch.
>
>> Interactive - Testpmd one, I believe, Feeding stdin programmatically would suffice to test all the combinations.
>
>
> One of the issues this is trying to address is that human-readable strings are a poor way to pass complex information between two programs. DTS is a distributed system, and it can have up to 3 physical servers involved in any given test. This means that it's not stdin via a pipe, it's an entire SSH session. This adds a noticeable amount of overhead when trying to send and verify the result of sending 1,000+ packets, since the lack of structured output means each packet must be checked before the next can be sent. This might be solvable by adding a structured output mode to testpmd, but that would involve committing to writing output twice for every function in testpmd forever.
>
>> We need to add all test cases in this model and we need to maintain two sets of programs.(Traditional tests and gRPC model-based tests).
>
>
> Assuming by traditional tests you mean the unit tests run by Meson, I would argue that we are already maintaining 2 kinds of tests. The unit tests, and the python-based DTS tests. My intention is to create a thin wrapper around DPDK that would be exposed via gRPC, like you see here, and use that as midware. Then, we would have two front-ends. Testpmd, which takes text and then calls midware as it does now, and the gRPC frontend, which parses messages from the RPC server and runs the midware. This would enable testpmd to still be used to sanity check a DPDK installation, but we would not need to continually expand Testpmd. The primary issue is that, right now, anything not included in Testpmd is not really testable by DTS. This includes portions of the RTE Flow API, which was part of my reason for proposing this. The RTE Flow API would, in my estimation, if fully implemented into Testpmd, probably add at least another 10,000 lines of code. As mentioned in my proposal, Testpmd already does more parsing and lexing than it does interaction with DPDK by line count. Also, since I am proposing making this a separate application, we would be able to gradually migrate the tests inside of DTS. This would have no effect on anything except for Testpmd, the new application and the addition of a flag to toggle the use of a C++ compiler.
>
> I'm not sure exactly what you mean by gRPC model-based tests. gRPC uses classes to model services, but for this usecase we are essentially using it to transfer function arguments across the internet and then pass the return value back. Any RPC framework would function similarly if I ignored the restrictions of which languages to use, and the choice is not important to how tests are conducted. Put another way, how you write a test for DTS will not change much if you are using this or testpmd, it's just how you transfer data and get it back that I want to change.
  

>
> If so, I think, gRPC service would be along with existing
> testpmd functions, like start_packet_forwarding().


It was my intention to re-use existing functions. I used the ACL tests as
an example because they are more self-contained then Testpmd, which made
creating the proof of concept much easier.

Also, We don't need to rewrite the existing testpmd, Instead, RPC service,
> we can add in existing app/test-pmd/
>

The reason that I split out the services is that there doesn't seem to be a
way to produce multiple binaries without re-writing that section of the
build system. I wanted to avoid the hard requirement of having a C++
compiler available in order to be able to use testpmd, since that may
affect what platforms Testpmd can run on and I want to avoid this being any
kind of breaking change. If we decide to go the route of putting it all in
a single application, we would need to conditionally enable the gRPC
service at build time. Meson's current lack of support for conditionally
detecting compilers causes issues here.

I think, DPDK has only one interactive test case which is testpmd,
>

Could you point me to that test case? Either invocation or source is ok. I
can't see anything that would lead me to assume use of testpmd in "meson
test --list". To my knowledge, all of the test cases that use testpmd are
in DTS. If there is a test that uses testpmd but is not part of DTS, I
think it would be a candidate for moving into DTS assuming it's not a unit
test.

How you are planning to do noninteractive test cases?


I'm not planning to make any change to unit testing, you can read more
about how testing is currently conducted here:
https://www.dpdk.org/blog/2021/07/05/dpdk-testing-approaches/

If there is a unit test that involves testpmd, there are two possibilities.
1. If we are making a separate application for Testpmd with the gRPC api,
then nothing changes except for possibly changing where some of the testpmd
source lives in order to enable code reuse between the two applications.
2. If gRPC is being added to Testpmd, then the unit test should still
function as it previously did if I do any necessary refactoring as
correctly.

I think, key should be leveraging existing test cases as much as possible
> and make easy for developers to add new test cases.


That is part of the reason why I want to be able to do this. Adding a new
test in DTS is very easy if the functionality needed already exists in
Testpmd. If the functionality does not exist, then adding the test becomes
difficult, due to the required modifications to the Testpmd lexer and
parser to accommodate the new command. My plan is to leave unit testing in
C, but help make it easier to expose C functions to Python for integration
testing. This gives us the best of both worlds in terms of access to DPDK
and the ability to use a high-level language to write the tests.

On Tue, Apr 12, 2022 at 2:07 AM Jerin Jacob <jerinjacobk@gmail.com> wrote:

> On Mon, Apr 11, 2022 at 11:19 PM Owen Hilyard <ohilyard@iol.unh.edu>
> wrote:
> >>
> >> scheme is probably over-engineered
> >
> >
> > I tried my hardest to keep this as simple as possible. The requirements
> imposed by DTS being a distributed system in Python restricted what I could
> do a lot. Needing to be compatible with DPDK's license also got rid of a
> lot of options. Binding generators are made for simple projects, and DPDK
> is not a simple project. There were some other options related to choice in
> the RPC framework, but very few RPC protocols seem to work well with C and
> be usable from Python, which is why I ended up using C++ with gRPC. Most of
> the code in api_impl.cc is taken from /app/test-acl/main.c, and the new
> part is mostly the C++ class at the bottom. Overall, this proposal comes
> out to ~100 lines of new C++, 9 lines of C, 12 lines of gRPC Protobuf and
> 100 lines of Meson. gRPC may be able to do a lot more than I need it to for
> the proof of concept, but many of the features that are not used, like
> bi-directional streaming, become very useful in writing more complicated
> tests. Overall, this solution is probably more capable than we need it to
> be, but I think that those extra capabilities don't come at a very large
> cost.
>
>
> Now it is clear, I was carried away with the POC test application and
> I was not knowing existing DTS tests are based on python
>
> Is below a fair summary?
>
> 1) DPDK has interactive test cases and no interactive test cases.
>
> For The interactive test case like testpmd, I agree that we can enable
> RPC service via gRPC server in C++ as  and client in Python, and
> something along the lines of exposing the existing test-pmd command
> line function as service
> to avoid command line parsing and reuse the existing python test suite.
>
> If so, I think, gRPC service would be along with existing testpmd
> functions, like start_packet_forwarding(). Also, We don't need to
> rewrite the existing testpmd,
> Instead, RPC service, we can add in existing app/test-pmd/ and hook to
> existing core testpmd functions to bypass the command-line parsing in
> C and control from python client as needed as service.
>
> Also, I agree that pulling in gRPC C++ server boilerplate and hooking
> to C functions is a good idea as it is the best C-based RPC scheme
> available today.
>
> 2)I think, DPDK has only one interactive test case which is testpmd,
> Remaining test cases are non-interactive, non-interactive test cases
> can simply run over ssh with passwordless login. Right?
> Do we need gRPC for that? Will the following scheme suffice? If not,
> How you are planning to do noninteractive test cases?
> i.e
> a)Copy test to target
> b) result=`ssh username@IP /path/to/testapp/in/target`
>
> I think, key should be leveraging existing test cases as much as
> possible and make easy for developers to add new test cases.
>
>
> >>
> >> Now that, Test code is also part of DPDK.
> >
> >
> > DTS is pure python. I tried to use FFI to call directly into DPDK from
> Python and then use xmlrpc from the python standard library. As mentioned
> in the writeup, I couldn't find a binding generator that would properly
> handle DPDK's allocators, which made it so that anything passed to DPDK
> using python was allocated using the system malloc. I don't think it is
> wise to attempt to programmatically re-write the generated code to allow
> for custom allocators. The original reason for needing to have DTS and DPDK
> in the same repository was so that tests could be committed and run
> alongside the feature patch.
> >
> >> Interactive - Testpmd one, I believe, Feeding stdin programmatically
> would suffice to test all the combinations.
> >
> >
> > One of the issues this is trying to address is that human-readable
> strings are a poor way to pass complex information between two programs.
> DTS is a distributed system, and it can have up to 3 physical servers
> involved in any given test. This means that it's not stdin via a pipe, it's
> an entire SSH session. This adds a noticeable amount of overhead when
> trying to send and verify the result of sending 1,000+ packets, since the
> lack of structured output means each packet must be checked before the next
> can be sent. This might be solvable by adding a structured output mode to
> testpmd, but that would involve committing to writing output twice for
> every function in testpmd forever.
> >
> >> We need to add all test cases in this model and we need to maintain two
> sets of programs.(Traditional tests and gRPC model-based tests).
> >
> >
> > Assuming by traditional tests you mean the unit tests run by Meson, I
> would argue that we are already maintaining 2 kinds of tests. The unit
> tests, and the python-based DTS tests. My intention is to create a thin
> wrapper around DPDK that would be exposed via gRPC, like you see here, and
> use that as midware. Then, we would have two front-ends. Testpmd, which
> takes text and then calls midware as it does now, and the gRPC frontend,
> which parses messages from the RPC server and runs the midware. This would
> enable testpmd to still be used to sanity check a DPDK installation, but we
> would not need to continually expand Testpmd. The primary issue is that,
> right now, anything not included in Testpmd is not really testable by DTS.
> This includes portions of the RTE Flow API, which was part of my reason for
> proposing this. The RTE Flow API would, in my estimation, if fully
> implemented into Testpmd, probably add at least another 10,000 lines of
> code. As mentioned in my proposal, Testpmd already does more parsing and
> lexing than it does interaction with DPDK by line count. Also, since I am
> proposing making this a separate application, we would be able to gradually
> migrate the tests inside of DTS. This would have no effect on anything
> except for Testpmd, the new application and the addition of a flag to
> toggle the use of a C++ compiler.
> >
> > I'm not sure exactly what you mean by gRPC model-based tests. gRPC uses
> classes to model services, but for this usecase we are essentially using it
> to transfer function arguments across the internet and then pass the return
> value back. Any RPC framework would function similarly if I ignored the
> restrictions of which languages to use, and the choice is not important to
> how tests are conducted. Put another way, how you write a test for DTS will
> not change much if you are using this or testpmd, it's just how you
> transfer data and get it back that I want to change.
>
  

Hi everyone,

First of all thanks Owen for stepping forward with this RFC.
Few thoughts on this subject below.
Konstantin   

> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Thursday, April 14, 2022 12:59 PM
> To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Subject: FW: [PATCH v1 0/4] [RFC] Testpmd RPC API
> 
> 
> 
> From: Owen Hilyard <ohilyard@iol.unh.edu>
> Sent: Wednesday, April 13, 2022 1:47 PM
> To: Jerin Jacob <jerinjacobk@gmail.com>
> Cc: dpdk-dev <dev@dpdk.org>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Thomas Monjalon <thomas@monjalon.net>
> Subject: Re: [PATCH v1 0/4] [RFC] Testpmd RPC API
> 
> If so, I think, gRPC service would be along with existing testpmd functions, like start_packet_forwarding().
> 
> It was my intention to re-use existing functions. I used the ACL tests as an example because they are more self-contained then Testpmd,
> which made creating the proof of concept much easier.
> 
> Also, We don't need to rewrite the existing testpmd, Instead, RPC service, we can add in existing app/test-pmd/
> 
> The reason that I split out the services is that there doesn't seem to be a way to produce multiple binaries without re-writing that section of
> the build system. I wanted to avoid the hard requirement of having a C++ compiler available in order to be able to use testpmd, since that
> may affect what platforms Testpmd can run on and I want to avoid this being any kind of breaking change. If we decide to go the route of
> putting it all in a single application, we would need to conditionally enable the gRPC service at build time. Meson's current lack of support
> for conditionally detecting compilers causes issues here.
> 
> I think, DPDK has only one interactive test case which is testpmd,
> 
> Could you point me to that test case? Either invocation or source is ok. I can't see anything that would lead me to assume use of testpmd in
> "meson test --list". To my knowledge, all of the test cases that use testpmd are in DTS. If there is a test that uses testpmd but is not part of
> DTS, I think it would be a candidate for moving into DTS assuming it's not a unit test.
> 
> How you are planning to do noninteractive test cases?
> 
> I'm not planning to make any change to unit testing, you can read more about how testing is currently conducted
> here: https://www.dpdk.org/blog/2021/07/05/dpdk-testing-approaches/
> 
> If there is a unit test that involves testpmd, there are two possibilities.
> 1. If we are making a separate application for Testpmd with the gRPC api, then nothing changes except for possibly changing where some
> of the testpmd source lives in order to enable code reuse between the two applications.
> 2. If gRPC is being added to Testpmd, then the unit test should still function as it previously did if I do any necessary refactoring as correctly.
> 
> I think, key should be leveraging existing test cases as much as possible and make easy for developers to add new test cases.
> 
> That is part of the reason why I want to be able to do this. Adding a new test in DTS is very easy if the functionality needed already exists in
> Testpmd. If the functionality does not exist, then adding the test becomes difficult, due to the required modifications to the Testpmd lexer
> and parser to accommodate the new command. My plan is to leave unit testing in C, but help make it easier to expose C functions to Python
> for integration testing. This gives us the best of both worlds in terms of access to DPDK and the ability to use a high-level language to write
> the tests.
> 
> On Tue, Apr 12, 2022 at 2:07 AM Jerin Jacob <mailto:jerinjacobk@gmail.com> wrote:
> On Mon, Apr 11, 2022 at 11:19 PM Owen Hilyard <mailto:ohilyard@iol.unh.edu> wrote:
> >>
> >> scheme is probably over-engineered
> >
> >
> > I tried my hardest to keep this as simple as possible. The requirements imposed by DTS being a distributed system in Python restricted
> what I could do a lot. Needing to be compatible with DPDK's license also got rid of a lot of options. Binding generators are made for simple
> projects, and DPDK is not a simple project. There were some other options related to choice in the RPC framework, but very few RPC
> protocols seem to work well with C and be usable from Python, which is why I ended up using C++ with gRPC. Most of the code in
> api_impl.cc is taken from /app/test-acl/main.c, and the new part is mostly the C++ class at the bottom. Overall, this proposal comes out to
> ~100 lines of new C++, 9 lines of C, 12 lines of gRPC Protobuf and 100 lines of Meson. gRPC may be able to do a lot more than I need it to
> for the proof of concept, but many of the features that are not used, like bi-directional streaming, become very useful in writing more
> complicated tests. Overall, this solution is probably more capable than we need it to be, but I think that those extra capabilities don't come
> at a very large cost.
> 
> 
> Now it is clear, I was carried away with the POC test application and
> I was not knowing existing DTS tests are based on python
> 
> Is below a fair summary?
> 
> 1) DPDK has interactive test cases and no interactive test cases.
> 
> For The interactive test case like testpmd, I agree that we can enable
> RPC service via gRPC server in C++ as  and client in Python, and
> something along the lines of exposing the existing test-pmd command
> line function as service
> to avoid command line parsing and reuse the existing python test suite.
> 
> If so, I think, gRPC service would be along with existing testpmd
> functions, like start_packet_forwarding(). Also, We don't need to
> rewrite the existing testpmd,
> Instead, RPC service, we can add in existing app/test-pmd/ and hook to
> existing core testpmd functions to bypass the command-line parsing in
> C and control from python client as needed as service.
> 
> Also, I agree that pulling in gRPC C++ server boilerplate and hooking
> to C functions is a good idea as it is the best C-based RPC scheme
> available today.
> 
> 2)I think, DPDK has only one interactive test case which is testpmd,
> Remaining test cases are non-interactive, non-interactive test cases
> can simply run over ssh with passwordless login. Right?
> Do we need gRPC for that? Will the following scheme suffice? If not,
> How you are planning to do noninteractive test cases?
> i.e
> a)Copy test to target
> b) result=`ssh username@IP /path/to/testapp/in/target`
> 
> I think, key should be leveraging existing test cases as much as
> possible and make easy for developers to add new test cases.
> 
> 
> >>
> >> Now that, Test code is also part of DPDK.
> >
> >
> > DTS is pure python. I tried to use FFI to call directly into DPDK from Python and then use xmlrpc from the python standard library. As
> mentioned in the writeup, I couldn't find a binding generator that would properly handle DPDK's allocators, which made it so that anything
> passed to DPDK using python was allocated using the system malloc. I don't think it is wise to attempt to programmatically re-write the
> generated code to allow for custom allocators. The original reason for needing to have DTS and DPDK in the same repository was so that
> tests could be committed and run alongside the feature patch.
> >
> >> Interactive - Testpmd one, I believe, Feeding stdin programmatically would suffice to test all the combinations.
> >
> >
> > One of the issues this is trying to address is that human-readable strings are a poor way to pass complex information between two
> programs. DTS is a distributed system, and it can have up to 3 physical servers involved in any given test. This means that it's not stdin via a
> pipe, it's an entire SSH session. This adds a noticeable amount of overhead when trying to send and verify the result of sending 1,000+
> packets, since the lack of structured output means each packet must be checked before the next can be sent. This might be solvable by
> adding a structured output mode to testpmd, but that would involve committing to writing output twice for every function in testpmd
> forever.
> >
> >> We need to add all test cases in this model and we need to maintain two sets of programs.(Traditional tests and gRPC model-based
> tests).
> >
> >
> > Assuming by traditional tests you mean the unit tests run by Meson, I would argue that we are already maintaining 2 kinds of tests. The
> unit tests, and the python-based DTS tests. My intention is to create a thin wrapper around DPDK that would be exposed via gRPC, like you
> see here, and use that as midware. Then, we would have two front-ends. Testpmd, which takes text and then calls midware as it does now,
> and the gRPC frontend, which parses messages from the RPC server and runs the midware. This would enable testpmd to still be used to
> sanity check a DPDK installation, but we would not need to continually expand Testpmd. The primary issue is that, right now, anything not
> included in Testpmd is not really testable by DTS. This includes portions of the RTE Flow API, which was part of my reason for proposing this.
> The RTE Flow API would, in my estimation, if fully implemented into Testpmd, probably add at least another 10,000 lines of code. As
> mentioned in my proposal, Testpmd already does more parsing and lexing than it does interaction with DPDK by line count. Also, since I am
> proposing making this a separate application, we would be able to gradually migrate the tests inside of DTS. This would have no effect on
> anything except for Testpmd, the new application and the addition of a flag to toggle the use of a C++ compiler.
> >
> > I'm not sure exactly what you mean by gRPC model-based tests. gRPC uses classes to model services, but for this usecase we are
> essentially using it to transfer function arguments across the internet and then pass the return value back. Any RPC framework would
> function similarly if I ignored the restrictions of which languages to use, and the choice is not important to how tests are conducted. Put
> another way, how you write a test for DTS will not change much if you are using this or testpmd, it's just how you transfer data and get it
> back that I want to change.

- In general I think it is a good idea to adding gRPC binding to testpmd to expose/improve testing automation.
  Though I think we shouldn’t remove existing CLI interface.
  Ideally I’d like to have both – CLI and gRPC for all commands.
  Don’t know how realistic is that, but at least for major commands -  port/queue configure, start/stop, etc.
- Conditional compilation (new meson flag or so) is probably good enough for this case.   
- About RFC itself - I understand that you choose testacl for simplicity, but in fact, it is a standalone application
  that has not much common with testpmd itself and the problems that you mentioned:
  interactive commands, parameter and results parsing, etc.
  Would it be possible to try implement something more realistic with testpmd itself,
  like simple test-pmd port/queue configure, start,  result collection, etc.?
  To get a better idea how it is going to work and how complicated it would be.
  

>
>   Though I think we shouldn’t remove existing CLI interface.
>

I agree, it's a very useful debugging tool for validating environments. I
think having two "frontends", the CLI and API, which both consume one
"backend" testpmd library would be the easiest way to go about doing that
while keeping long-term maintenance low.

Conditional compilation (new meson flag or so) is probably good enough for
> this case.
>

One of the changes I made was an on-by-default meson flag to enable C++
compilation. If that flag is on, and all dependencies are present, then the
application will be built.

Would it be possible to try implement something more realistic with testpmd
> itself


I would consider it a "phase 2" version of this RFC. The hard part was
getting gRPC working inside of Meson, which is why I picked a simple app to
port. If this RFC moves forward, I can look at porting the functionality
needed for the nic single core performance test (
http://git.dpdk.org/tools/dts/tree/test_plans/nic_single_core_perf_test_plan.rst
).

On Thu, Apr 14, 2022 at 8:08 AM Ananyev, Konstantin <
konstantin.ananyev@intel.com> wrote:

>
> Hi everyone,
>
> First of all thanks Owen for stepping forward with this RFC.
> Few thoughts on this subject below.
> Konstantin
>
> > -----Original Message-----
> > From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> > Sent: Thursday, April 14, 2022 12:59 PM
> > To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> > Subject: FW: [PATCH v1 0/4] [RFC] Testpmd RPC API
> >
> >
> >
> > From: Owen Hilyard <ohilyard@iol.unh.edu>
> > Sent: Wednesday, April 13, 2022 1:47 PM
> > To: Jerin Jacob <jerinjacobk@gmail.com>
> > Cc: dpdk-dev <dev@dpdk.org>; Honnappa Nagarahalli <
> Honnappa.Nagarahalli@arm.com>; Thomas Monjalon <thomas@monjalon.net>
> > Subject: Re: [PATCH v1 0/4] [RFC] Testpmd RPC API
> >
> > If so, I think, gRPC service would be along with existing testpmd
> functions, like start_packet_forwarding().
> >
> > It was my intention to re-use existing functions. I used the ACL tests
> as an example because they are more self-contained then Testpmd,
> > which made creating the proof of concept much easier.
> >
> > Also, We don't need to rewrite the existing testpmd, Instead, RPC
> service, we can add in existing app/test-pmd/
> >
> > The reason that I split out the services is that there doesn't seem to
> be a way to produce multiple binaries without re-writing that section of
> > the build system. I wanted to avoid the hard requirement of having a C++
> compiler available in order to be able to use testpmd, since that
> > may affect what platforms Testpmd can run on and I want to avoid this
> being any kind of breaking change. If we decide to go the route of
> > putting it all in a single application, we would need to conditionally
> enable the gRPC service at build time. Meson's current lack of support
> > for conditionally detecting compilers causes issues here.
> >
> > I think, DPDK has only one interactive test case which is testpmd,
> >
> > Could you point me to that test case? Either invocation or source is ok.
> I can't see anything that would lead me to assume use of testpmd in
> > "meson test --list". To my knowledge, all of the test cases that use
> testpmd are in DTS. If there is a test that uses testpmd but is not part of
> > DTS, I think it would be a candidate for moving into DTS assuming it's
> not a unit test.
> >
> > How you are planning to do noninteractive test cases?
> >
> > I'm not planning to make any change to unit testing, you can read more
> about how testing is currently conducted
> > here: https://www.dpdk.org/blog/2021/07/05/dpdk-testing-approaches/
> >
> > If there is a unit test that involves testpmd, there are two
> possibilities.
> > 1. If we are making a separate application for Testpmd with the gRPC
> api, then nothing changes except for possibly changing where some
> > of the testpmd source lives in order to enable code reuse between the
> two applications.
> > 2. If gRPC is being added to Testpmd, then the unit test should still
> function as it previously did if I do any necessary refactoring as
> correctly.
> >
> > I think, key should be leveraging existing test cases as much as
> possible and make easy for developers to add new test cases.
> >
> > That is part of the reason why I want to be able to do this. Adding a
> new test in DTS is very easy if the functionality needed already exists in
> > Testpmd. If the functionality does not exist, then adding the test
> becomes difficult, due to the required modifications to the Testpmd lexer
> > and parser to accommodate the new command. My plan is to leave unit
> testing in C, but help make it easier to expose C functions to Python
> > for integration testing. This gives us the best of both worlds in terms
> of access to DPDK and the ability to use a high-level language to write
> > the tests.
> >
> > On Tue, Apr 12, 2022 at 2:07 AM Jerin Jacob <mailto:
> jerinjacobk@gmail.com> wrote:
> > On Mon, Apr 11, 2022 at 11:19 PM Owen Hilyard <mailto:
> ohilyard@iol.unh.edu> wrote:
> > >>
> > >> scheme is probably over-engineered
> > >
> > >
> > > I tried my hardest to keep this as simple as possible. The
> requirements imposed by DTS being a distributed system in Python restricted
> > what I could do a lot. Needing to be compatible with DPDK's license also
> got rid of a lot of options. Binding generators are made for simple
> > projects, and DPDK is not a simple project. There were some other
> options related to choice in the RPC framework, but very few RPC
> > protocols seem to work well with C and be usable from Python, which is
> why I ended up using C++ with gRPC. Most of the code in
> > api_impl.cc is taken from /app/test-acl/main.c, and the new part is
> mostly the C++ class at the bottom. Overall, this proposal comes out to
> > ~100 lines of new C++, 9 lines of C, 12 lines of gRPC Protobuf and 100
> lines of Meson. gRPC may be able to do a lot more than I need it to
> > for the proof of concept, but many of the features that are not used,
> like bi-directional streaming, become very useful in writing more
> > complicated tests. Overall, this solution is probably more capable than
> we need it to be, but I think that those extra capabilities don't come
> > at a very large cost.
> >
> >
> > Now it is clear, I was carried away with the POC test application and
> > I was not knowing existing DTS tests are based on python
> >
> > Is below a fair summary?
> >
> > 1) DPDK has interactive test cases and no interactive test cases.
> >
> > For The interactive test case like testpmd, I agree that we can enable
> > RPC service via gRPC server in C++ as  and client in Python, and
> > something along the lines of exposing the existing test-pmd command
> > line function as service
> > to avoid command line parsing and reuse the existing python test suite.
> >
> > If so, I think, gRPC service would be along with existing testpmd
> > functions, like start_packet_forwarding(). Also, We don't need to
> > rewrite the existing testpmd,
> > Instead, RPC service, we can add in existing app/test-pmd/ and hook to
> > existing core testpmd functions to bypass the command-line parsing in
> > C and control from python client as needed as service.
> >
> > Also, I agree that pulling in gRPC C++ server boilerplate and hooking
> > to C functions is a good idea as it is the best C-based RPC scheme
> > available today.
> >
> > 2)I think, DPDK has only one interactive test case which is testpmd,
> > Remaining test cases are non-interactive, non-interactive test cases
> > can simply run over ssh with passwordless login. Right?
> > Do we need gRPC for that? Will the following scheme suffice? If not,
> > How you are planning to do noninteractive test cases?
> > i.e
> > a)Copy test to target
> > b) result=`ssh username@IP /path/to/testapp/in/target`
> >
> > I think, key should be leveraging existing test cases as much as
> > possible and make easy for developers to add new test cases.
> >
> >
> > >>
> > >> Now that, Test code is also part of DPDK.
> > >
> > >
> > > DTS is pure python. I tried to use FFI to call directly into DPDK from
> Python and then use xmlrpc from the python standard library. As
> > mentioned in the writeup, I couldn't find a binding generator that would
> properly handle DPDK's allocators, which made it so that anything
> > passed to DPDK using python was allocated using the system malloc. I
> don't think it is wise to attempt to programmatically re-write the
> > generated code to allow for custom allocators. The original reason for
> needing to have DTS and DPDK in the same repository was so that
> > tests could be committed and run alongside the feature patch.
> > >
> > >> Interactive - Testpmd one, I believe, Feeding stdin programmatically
> would suffice to test all the combinations.
> > >
> > >
> > > One of the issues this is trying to address is that human-readable
> strings are a poor way to pass complex information between two
> > programs. DTS is a distributed system, and it can have up to 3 physical
> servers involved in any given test. This means that it's not stdin via a
> > pipe, it's an entire SSH session. This adds a noticeable amount of
> overhead when trying to send and verify the result of sending 1,000+
> > packets, since the lack of structured output means each packet must be
> checked before the next can be sent. This might be solvable by
> > adding a structured output mode to testpmd, but that would involve
> committing to writing output twice for every function in testpmd
> > forever.
> > >
> > >> We need to add all test cases in this model and we need to maintain
> two sets of programs.(Traditional tests and gRPC model-based
> > tests).
> > >
> > >
> > > Assuming by traditional tests you mean the unit tests run by Meson, I
> would argue that we are already maintaining 2 kinds of tests. The
> > unit tests, and the python-based DTS tests. My intention is to create a
> thin wrapper around DPDK that would be exposed via gRPC, like you
> > see here, and use that as midware. Then, we would have two front-ends.
> Testpmd, which takes text and then calls midware as it does now,
> > and the gRPC frontend, which parses messages from the RPC server and
> runs the midware. This would enable testpmd to still be used to
> > sanity check a DPDK installation, but we would not need to continually
> expand Testpmd. The primary issue is that, right now, anything not
> > included in Testpmd is not really testable by DTS. This includes
> portions of the RTE Flow API, which was part of my reason for proposing
> this.
> > The RTE Flow API would, in my estimation, if fully implemented into
> Testpmd, probably add at least another 10,000 lines of code. As
> > mentioned in my proposal, Testpmd already does more parsing and lexing
> than it does interaction with DPDK by line count. Also, since I am
> > proposing making this a separate application, we would be able to
> gradually migrate the tests inside of DTS. This would have no effect on
> > anything except for Testpmd, the new application and the addition of a
> flag to toggle the use of a C++ compiler.
> > >
> > > I'm not sure exactly what you mean by gRPC model-based tests. gRPC
> uses classes to model services, but for this usecase we are
> > essentially using it to transfer function arguments across the internet
> and then pass the return value back. Any RPC framework would
> > function similarly if I ignored the restrictions of which languages to
> use, and the choice is not important to how tests are conducted. Put
> > another way, how you write a test for DTS will not change much if you
> are using this or testpmd, it's just how you transfer data and get it
> > back that I want to change.
>
> - In general I think it is a good idea to adding gRPC binding to testpmd
> to expose/improve testing automation.
>   Though I think we shouldn’t remove existing CLI interface.
>   Ideally I’d like to have both – CLI and gRPC for all commands.
>   Don’t know how realistic is that, but at least for major commands -
> port/queue configure, start/stop, etc.
> - Conditional compilation (new meson flag or so) is probably good enough
> for this case.
> - About RFC itself - I understand that you choose testacl for simplicity,
> but in fact, it is a standalone application
>   that has not much common with testpmd itself and the problems that you
> mentioned:
>   interactive commands, parameter and results parsing, etc.
>   Would it be possible to try implement something more realistic with
> testpmd itself,
>   like simple test-pmd port/queue configure, start,  result collection,
> etc.?
>   To get a better idea how it is going to work and how complicated it
> would be.
>
>
>