diff mbox series

[v5,04/10] app/test: build using per-file dependency matrix

Message ID 20230815151053.996469-5-bruce.richardson@intel.com (mailing list archive)
State New
Headers
Series expand list of optional libraries |

Commit Message

Bruce Richardson Aug. 15, 2023, 3:10 p.m. UTC 
  Rather than using if-else constructs to selectively add or remove files
from the UT build, switch to a table-based approach where each file
lists out what libs or drivers it depends upon.

Initial version of this table was generated via analysis of the header
files included in each C file. The basic dependencies of the test binary
[cmdline, ring, mempool and mbuf] were then removed from the per-file
lists, as there is no point in checking them as the whole app will be
disabled if they are not present.

With the file list, the dependencies for the "utility" C-files are kept
separate, so that other tests which depend on the functions provided by
those files can have that dependency recorded properly. The basic
cryptodev tests also fall into this category as functions from the main
cryptodev test file are used by other crypto tests.

As well as the main table for internal dependencies, some test
files have separate external components too. A second, much smaller
table lists these dependencies.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/meson.build      |   8 +-
 app/test/meson.build | 823 ++++++++++++-------------------------------
 2 files changed, 238 insertions(+), 593 deletions(-)

Comments

Patrick Robb Aug. 15, 2023, 7:05 p.m. UTC | #1 
Adam from our team just raised something important about this patch and UNH
CI which I missed during the RFC discussion.
Presently, eal_flags_file_prefix_autotest fails on arm tx2 systems (arm
people are aware of this, they think it might be a memory leak from mlx5
driver), so at their request we are disabling this unit test (on arm only)
for them for the time being. This failure was discovered when we were
initially standing up arm unit testing on tx2 servers earlier this year -
previously there was no coverage for this. We do this filtering by
going through the /app/test/meson.build file and resetting
the eal_flags_file_prefix_autotest line to an empty string. This process is
broken by your patch series. Again, I'm sorry I didn't catch this concern
when discussing it during the RFC. This is why you are not getting unit
test results for arm64, we can't run unit tests given the changes in this
patch.

Your refactor likely means that going forward, we will no longer be able to
tailor the fast test suite (or any suite) per vendor request. That might
actually be a good thing. In any case, if this is merged now, it is going
to put the tree in a state where our CI doesn't run any unit testing on
ARM. I don't know how close this patch series is to possibly hitting
mainline, but, if possible, can that be delayed, pending us figuring out
how we will respond to this situation?
Bruce Richardson Aug. 16, 2023, 10:56 a.m. UTC | #2 
On Tue, Aug 15, 2023 at 03:05:08PM -0400, Patrick Robb wrote:
>    Adam from our team just raised something important about this patch and
>    UNH CI which I missed during the RFC discussion.
>    Presently, eal_flags_file_prefix_autotest fails on arm tx2 systems (arm
>    people are aware of this, they think it might be a memory leak from
>    mlx5 driver), so at their request we are disabling this unit test (on
>    arm only) for them for the time being. This failure was discovered when
>    we were initially standing up arm unit testing on tx2 servers earlier
>    this year - previously there was no coverage for this. We do this
>    filtering by going through the /app/test/meson.build file and resetting
>    the eal_flags_file_prefix_autotest line to an empty string. This
>    process is broken by your patch series. Again, I'm sorry I didn't catch
>    this concern when discussing it during the RFC. This is why you are not
>    getting unit test results for arm64, we can't run unit tests given the
>    changes in this patch.

If this is a (semi)permanent issue, would it not be better to actually
patch in the disabling of this test into the test sources, i.e. if ARM64
have the test return TEST_SKIPPED?

>    Your refactor likely means that going forward, we will no longer be
>    able to tailor the fast test suite (or any suite) per vendor request.
>    That might actually be a good thing. In any case, if this is merged
>    now, it is going to put the tree in a state where our CI doesn't run
>    any unit testing on ARM. I don't know how close this patch series is to
>    possibly hitting mainline, but, if possible, can that be delayed,
>    pending us figuring out how we will respond to this situation?

There is at least one more issue I am aware of with this set - the failing
mempool tests - which needs to be fixed, so there will be at least one more
version.

If this is likely a common occurrance where we need to disable one
particular test for a particular platform, would it be worth designing such
a feature into the rework. For example, it should be fairly easy to have
the testname-extraction script read a blocklist from the environment, and
omit informing meson of those test cases. Would that work?

/Bruce
Bruce Richardson Aug. 16, 2023, 12:55 p.m. UTC | #3 
On Tue, Aug 15, 2023 at 03:05:08PM -0400, Patrick Robb wrote:
>    Adam from our team just raised something important about this patch and
>    UNH CI which I missed during the RFC discussion.
>    Presently, eal_flags_file_prefix_autotest fails on arm tx2 systems (arm
>    people are aware of this, they think it might be a memory leak from
>    mlx5 driver), so at their request we are disabling this unit test (on
>    arm only) for them for the time being.

Does the test suite pass if the mlx5 driver is disabled in the build? That
could confirm or refute the suspicion of where the issue is, and also
provide a temporary workaround while this set is merged (possibly including
support for disabling specific tests, as I suggested in my other email).

/Bruce

PS: Are there any other workarounds inside the test/DTS/CI systems that
involve patching sources? If so, it would be good to get a list that we can
work through removing by putting place proper fixes or workarounds, as
changing sources for testing like this blocks future patch acceptance.
David Marchand Aug. 16, 2023, 2:40 p.m. UTC | #4 
Patrick, Bruce,

On Wed, Aug 16, 2023 at 2:57 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Tue, Aug 15, 2023 at 03:05:08PM -0400, Patrick Robb wrote:
> >    Adam from our team just raised something important about this patch and
> >    UNH CI which I missed during the RFC discussion.
> >    Presently, eal_flags_file_prefix_autotest fails on arm tx2 systems (arm
> >    people are aware of this, they think it might be a memory leak from

That's sad to read.

> >    mlx5 driver), so at their request we are disabling this unit test (on
> >    arm only) for them for the time being.

If it was reported, I either missed it or forgot about it, sorry.
Can you (re)share the context?

There was a couple of mem leak fixes for mlx5 in v23.07.
It may be worth running this test again and providing fresh traces.


>
> Does the test suite pass if the mlx5 driver is disabled in the build? That
> could confirm or refute the suspicion of where the issue is, and also
> provide a temporary workaround while this set is merged (possibly including
> support for disabling specific tests, as I suggested in my other email).

Or disabling the driver as Bruce proposes.

>
> /Bruce
>
> PS: Are there any other workarounds inside the test/DTS/CI systems that
> involve patching sources? If so, it would be good to get a list that we can
> work through removing by putting place proper fixes or workarounds, as
> changing sources for testing like this blocks future patch acceptance.

Patching sources from the test tool is a poor solution.
In general, developers won't be aware of source patching and will
waste time trying to understand why they can't reproduce what the CI
reports (it happened to me with DTS on the interrupt stuff with vhost,
at least).

For this specific case of skipping a test, if nobody can fix the
issue, I prefer if the CI can skip some "known broken in my lab" tests
via some meson configuration.
And, such configuration should be easy to catch in the test report.
Patrick Robb Aug. 16, 2023, 6:29 p.m. UTC | #5 
On Wed, Aug 16, 2023 at 10:40 AM David Marchand <david.marchand@redhat.com>
wrote:

> Patrick, Bruce,
>
> If it was reported, I either missed it or forgot about it, sorry.
> Can you (re)share the context?


> >
> > Does the test suite pass if the mlx5 driver is disabled in the build?
> That
> > could confirm or refute the suspicion of where the issue is, and also
> > provide a temporary workaround while this set is merged (possibly
> including
> > support for disabling specific tests, as I suggested in my other email).
>
> Or disabling the driver as Bruce proposes.
>
 Okay, we ran the test with the mlx5 driver disabled, and it still fails.
So, this might be more of an ARM architecture issue. Ruifeng, are you still
seeing this on your test bed?

@David you didn't miss anything, we had a unicast with ARM when setting up
the new arm container runners for unit testing a few months back. Ruifeng
also noticed the same issue and speculated about mlx5 memory leaks. He
raised the possibility of disabling the mlx5 driver too, but that option
isn't great since we want to have a uniform build process (as much as
possible) for our unit testing. Anyways, now we know that that isn't
relevant. I'll forward the thread to you in any case - let me know if you
have any ideas.

>
> >
> > /Bruce
> >
> > PS: Are there any other workarounds inside the test/DTS/CI systems that
> > involve patching sources? If so, it would be good to get a list that we
> can
> > work through removing by putting place proper fixes or workarounds, as
> > changing sources for testing like this blocks future patch acceptance.


> Patching sources from the test tool is a poor solution.
> In general, developers won't be aware of source patching and will
> waste time trying to understand why they can't reproduce what the CI
> reports (it happened to me with DTS on the interrupt stuff with vhost,
> at least).
>
> For this specific case of skipping a test, if nobody can fix the
> issue, I prefer if the CI can skip some "known broken in my lab" tests
> via some meson configuration.
> And, such configuration should be easy to catch in the test report.
>
> I strongly agree on all points, which is why I said it was probably a good
thing anyhow for us to lose this ability. In the case of the disabled
fast-test for arm, that was a new discovery coming from adding new
environments, not a regression introduced by a patch, and I don't think it
made sense then to block the introduction of the entire unit test coverage
for arm while they looked into this issue. If it's possible to introduce
meson configure functionality to disable specific tests, that does give us
more flexibility. And it's obviously a better process than us doing it at
the CI end.

We don't currently patch source in any other way in our CI testing.
David Marchand Aug. 16, 2023, 7:26 p.m. UTC | #6 
On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb@iol.unh.edu> wrote:
> On Wed, Aug 16, 2023 at 10:40 AM David Marchand <david.marchand@redhat.com> wrote:
>>
>> Patrick, Bruce,
>>
>> If it was reported, I either missed it or forgot about it, sorry.
>> Can you (re)share the context?
>>
>>
>> >
>> > Does the test suite pass if the mlx5 driver is disabled in the build? That
>> > could confirm or refute the suspicion of where the issue is, and also
>> > provide a temporary workaround while this set is merged (possibly including
>> > support for disabling specific tests, as I suggested in my other email).
>>
>> Or disabling the driver as Bruce proposes.
>
>  Okay, we ran the test with the mlx5 driver disabled, and it still fails. So, this might be more of an ARM architecture issue. Ruifeng, are you still seeing this on your test bed?
>
> @David you didn't miss anything, we had a unicast with ARM when setting up the new arm container runners for unit testing a few months back. Ruifeng also noticed the same issue and speculated about mlx5 memory leaks. He raised the possibility of disabling the mlx5 driver too, but that option isn't great since we want to have a uniform build process (as much as possible) for our unit testing. Anyways, now we know that that isn't relevant. I'll forward the thread to you in any case - let me know if you have any ideas.

The mention of "memtest1" in the mails rings a bell.
I will need more detailed logs, or ideally an env where it is reproduced.


One thing bothers me.. why are we not seeing this failure with ARM for
Bruce v6 series?
Just looking at patchwork, I would think that I can merge Bruce series as is.
https://patchwork.dpdk.org/project/dpdk/patch/20230816153439.551501-12-bruce.richardson@intel.com/
Patrick Robb Aug. 16, 2023, 8:38 p.m. UTC | #7 
On Wed, Aug 16, 2023 at 3:26 PM David Marchand <david.marchand@redhat.com>
wrote:

> On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb@iol.unh.edu> wrote:
> > On Wed, Aug 16, 2023 at 10:40 AM David Marchand <
> david.marchand@redhat.com> wrote:
> >>
> >> Patrick, Bruce,
> >>
> >> If it was reported, I either missed it or forgot about it, sorry.
> >> Can you (re)share the context?
> >>
> >>
> >> >
> >> > Does the test suite pass if the mlx5 driver is disabled in the build?
> That
> >> > could confirm or refute the suspicion of where the issue is, and also
> >> > provide a temporary workaround while this set is merged (possibly
> including
> >> > support for disabling specific tests, as I suggested in my other
> email).
> >>
> >> Or disabling the driver as Bruce proposes.
> >
> >  Okay, we ran the test with the mlx5 driver disabled, and it still
> fails. So, this might be more of an ARM architecture issue. Ruifeng, are
> you still seeing this on your test bed?
> >
> > @David you didn't miss anything, we had a unicast with ARM when setting
> up the new arm container runners for unit testing a few months back.
> Ruifeng also noticed the same issue and speculated about mlx5 memory leaks.
> He raised the possibility of disabling the mlx5 driver too, but that option
> isn't great since we want to have a uniform build process (as much as
> possible) for our unit testing. Anyways, now we know that that isn't
> relevant. I'll forward the thread to you in any case - let me know if you
> have any ideas.
>
> The mention of "memtest1" in the mails rings a bell.
> I will need more detailed logs, or ideally an env where it is reproduced.
>
> meson-logs/ for the unit tests run with eal_flags_file_prefix_autotest
included shared with you via slack. I also shared the meson test summary,
but of course it's the detailed testlog.txt you care about.

>
> One thing bothers me.. why are we not seeing this failure with ARM for
> Bruce v6 series?
> Just looking at patchwork, I would think that I can merge Bruce series as
> is.
>
> https://patchwork.dpdk.org/project/dpdk/patch/20230816153439.551501-12-bruce.richardson@intel.com/
>
> So, this is a niche edge case, but because we fail to apply the fast-test
filtering script in our jenkinsfile script, we exit without doing any unit
testing and don't save or report any results. Almost always if we fail
doing "unh jenkins scipt" stuff, it's an infra failure, not a problem with
a patch, and we don't want to report a false positive failure result there.
It does further exemplify the danger in our current process, of course.
I'll be glad to not have to do this anymore. I did try to make this point
above, but I don't think I explained it too well.

The only other thing I'll add is that we are going to change our reporting
process soon, to begin our pipeline run on a test/environment combo by
reporting a "pending" result on that test/environent. Then we will
overwrite it with a PASS or FAIL at the end. This helps protect us from
situations like this. For instance, the way this would have played out is
your would have had a label (iol-unit-arm64-testing) which would have had
the initial "PENDING" result reported to it, but it never would have been
updated from pending. So, you would know the CI results were incomplete.
David Marchand Aug. 17, 2023, 7:27 a.m. UTC | #8 
On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb@iol.unh.edu> wrote:
>> Patching sources from the test tool is a poor solution.
>> In general, developers won't be aware of source patching and will
>> waste time trying to understand why they can't reproduce what the CI
>> reports (it happened to me with DTS on the interrupt stuff with vhost,
>> at least).
>>
>> For this specific case of skipping a test, if nobody can fix the
>> issue, I prefer if the CI can skip some "known broken in my lab" tests
>> via some meson configuration.
>> And, such configuration should be easy to catch in the test report.
>>
> I strongly agree on all points, which is why I said it was probably a good thing anyhow for us to lose this ability. In the case of the disabled fast-test for arm, that was a new discovery coming from adding new environments, not a regression introduced by a patch, and I don't think it made sense then to block the introduction of the entire unit test coverage for arm while they looked into this issue. If it's possible to introduce meson configure functionality to disable specific tests, that does give us more flexibility. And it's obviously a better process than us doing it at the CI end.
>
> We don't currently patch source in any other way in our CI testing.

It is possible to list tests from meson.
Why not simply list the existing tests and filter the non working one?

$ meson test -C <build_dir> --suite fast-tests --list
$ meson test -C <build_dir> <explicit list> --test-args ...

That requires no change in DPDK.
Bruce Richardson Aug. 17, 2023, 8:46 a.m. UTC | #9 
On Thu, Aug 17, 2023 at 09:27:50AM +0200, David Marchand wrote:
> On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb@iol.unh.edu> wrote:
> >> Patching sources from the test tool is a poor solution.
> >> In general, developers won't be aware of source patching and will
> >> waste time trying to understand why they can't reproduce what the CI
> >> reports (it happened to me with DTS on the interrupt stuff with vhost,
> >> at least).
> >>
> >> For this specific case of skipping a test, if nobody can fix the
> >> issue, I prefer if the CI can skip some "known broken in my lab" tests
> >> via some meson configuration.
> >> And, such configuration should be easy to catch in the test report.
> >>
> > I strongly agree on all points, which is why I said it was probably a good thing anyhow for us to lose this ability. In the case of the disabled fast-test for arm, that was a new discovery coming from adding new environments, not a regression introduced by a patch, and I don't think it made sense then to block the introduction of the entire unit test coverage for arm while they looked into this issue. If it's possible to introduce meson configure functionality to disable specific tests, that does give us more flexibility. And it's obviously a better process than us doing it at the CI end.
> >
> > We don't currently patch source in any other way in our CI testing.
> 
> It is possible to list tests from meson.
> Why not simply list the existing tests and filter the non working one?
> 
> $ meson test -C <build_dir> --suite fast-tests --list
> $ meson test -C <build_dir> <explicit list> --test-args ...
> 
> That requires no change in DPDK.
> 
The small issue with that is that the broken test is hidden from reports.
While it unblocks things here without having to change DPDK, I wonder if
changing DPDK to support this scenario might be worthwhile. For example, one
idea I'm now thinking of is to have the test binary itself look in the
environment for a DPDK_TEST_SKIPPED variable, and - when processing test
names passed on command-line or environment - report any on that list as
SKIPPED. This at least flags in the reports that a test is not running, and
the test log gives a reason. 

Worth doing? Or is adding a new environment with broken tests rare enough
we don't need this?

Of course, this also seems like a feature request for meson. When running
all tests, meson allows you to skip a test suite, but doesn't allow
skipping a test within a suite.

/Bruce
Patrick Robb Aug. 17, 2023, 6:30 p.m. UTC | #10 
On Thu, Aug 17, 2023 at 3:28 AM David Marchand <david.marchand@redhat.com>
wrote:

> It is possible to list tests from meson.
> Why not simply list the existing tests and filter the non working one?
>
> $ meson test -C <build_dir> --suite fast-tests --list
> $ meson test -C <build_dir> <explicit list> --test-args ...
>
> That requires no change in DPDK.
>
>
> Thanks. I am seeing now that this is possible. So, we could easily solve
this problem using this approach. Thank you David.

At the same time, I do like Bruce's new RFC patch for excluding tests with
env variables. It is a bit cleaner on the testing experience side (I don't
have to list 100 tests in the command) and it makes clear in the logs what
is being skipped which typically would be run. Making that info visible
could be valuable in some cases.

Ultimately, we can proceed on the Community Lab side either way. I guess
the maintainers must decide whether adding in the extra complexity to the
test app is justified in this case.

Also, Bruce, I do see that the symlink you used is working for the case
where we have to (for now) directly use the dpdk-test app. I'm sorry I
didn't have that info on hand this morning during the ci meeting.
David Marchand Aug. 18, 2023, 7:07 a.m. UTC | #11 
On Wed, Aug 16, 2023 at 9:26 PM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb@iol.unh.edu> wrote:
> > On Wed, Aug 16, 2023 at 10:40 AM David Marchand <david.marchand@redhat.com> wrote:
> >>
> >> Patrick, Bruce,
> >>
> >> If it was reported, I either missed it or forgot about it, sorry.
> >> Can you (re)share the context?
> >>
> >>
> >> >
> >> > Does the test suite pass if the mlx5 driver is disabled in the build? That
> >> > could confirm or refute the suspicion of where the issue is, and also
> >> > provide a temporary workaround while this set is merged (possibly including
> >> > support for disabling specific tests, as I suggested in my other email).
> >>
> >> Or disabling the driver as Bruce proposes.
> >
> >  Okay, we ran the test with the mlx5 driver disabled, and it still fails. So, this might be more of an ARM architecture issue. Ruifeng, are you still seeing this on your test bed?
> >
> > @David you didn't miss anything, we had a unicast with ARM when setting up the new arm container runners for unit testing a few months back. Ruifeng also noticed the same issue and speculated about mlx5 memory leaks. He raised the possibility of disabling the mlx5 driver too, but that option isn't great since we want to have a uniform build process (as much as possible) for our unit testing. Anyways, now we know that that isn't relevant. I'll forward the thread to you in any case - let me know if you have any ideas.
>
> The mention of "memtest1" in the mails rings a bell.
> I will need more detailed logs, or ideally an env where it is reproduced.

It is a "recurring" yet not so well known issue.
This unit test fails if any part of the DPDK did not release all
(hugepage backed) memory and associated hugepages before exiting.

In your case here, there is a virtio-net device that the container
tries to get its hands on because DPDK scans and probes all available
resources by default (and fails to, in this case, but that's not
important).
Triggering this virtio-net probing makes ethdev allocate its shared
memzone for port data, but nothing in ethdev releases the memzone when
exiting.
Fixing this could be tricky... as the current ethdev code is really
vague around which locks protect what (if anything..).

I think we hit this issue in the past, and avoided it by running the
tests with dynamically linked DPDK binaries (and by doing this, avoid
the net drivers get loaded).

I can see that you are running the unit tests with a static binary in
the report you sent.
I think the default is shared mode, so I wonder what could be the
reason why UNH builds with static here.
In any case, could you have a try and switch to
-Ddefault_library=shared (or remove forcing to static mode)?
Patrick Robb Aug. 18, 2023, 10:33 p.m. UTC | #12 
On Fri, Aug 18, 2023 at 3:07 AM David Marchand <david.marchand@redhat.com>
wrote:

> It is a "recurring" yet not so well known issue.
> This unit test fails if any part of the DPDK did not release all
> (hugepage backed) memory and associated hugepages before exiting.
>
> In your case here, there is a virtio-net device that the container
> tries to get its hands on because DPDK scans and probes all available
> resources by default (and fails to, in this case, but that's not
> important).
> Triggering this virtio-net probing makes ethdev allocate its shared
> memzone for port data, but nothing in ethdev releases the memzone when
> exiting.
> Fixing this could be tricky... as the current ethdev code is really
> vague around which locks protect what (if anything..).
>
> I think we hit this issue in the past, and avoided it by running the
> tests with dynamically linked DPDK binaries (and by doing this, avoid
> the net drivers get loaded).
>
> I can see that you are running the unit tests with a static binary in
> the report you sent.
> I think the default is shared mode, so I wonder what could be the
> reason why UNH builds with static here.
> In any case, could you have a try and switch to
> -Ddefault_library=shared (or remove forcing to static mode)?
>
> Thanks for helping debug this David.

I think the default library actually is static, but let me know if I
misunderstand: https://git.dpdk.org/dpdk/tree/meson.build. If you meson
configure on the output of 'meson setup build' it says static is the
default.

```
        default_options: [
            'buildtype=release',
            'c_std=c11',
            'default_library=static',
            'warning_level=2',
        ],
```

I tried a build forcing default_library=shared, and all fast-tests unit
tests failed. I can look further into this if you think shared mode might
be the best way forward though. But I am assuming for now if static is
indeed the default, that is the way we should build for testing.

For your patch, it looks like there are actually more eal flags test
failures: http://mails.dpdk.org/archives/test-report/2023-August/444170.html.
In any case, for debugging purposes, I will do a run with
the eal_flags_file_prefix_autotest also included and send you the meson
test detail log on slack.
Patrick Robb Aug. 18, 2023, 11:26 p.m. UTC | #13 
On Fri, Aug 18, 2023 at 6:33 PM Patrick Robb <probb@iol.unh.edu> wrote:

>
> For your patch, it looks like there are actually more eal flags test
> failures:
> http://mails.dpdk.org/archives/test-report/2023-August/444170.html. In
> any case, for debugging purposes, I will do a run with
> the eal_flags_file_prefix_autotest also included and send you the meson
> test detail log on slack.
>
> Quick follow up - I see with your patch it does pass on
eal_flags_file_prefix_autotest, but as stated fails some others now. The
detailed slack logs are DM'ed to you on slack. Thanks!
David Marchand Aug. 21, 2023, 7:12 a.m. UTC | #14 
On Sat, Aug 19, 2023 at 1:26 AM Patrick Robb <probb@iol.unh.edu> wrote:
> On Fri, Aug 18, 2023 at 6:33 PM Patrick Robb <probb@iol.unh.edu> wrote:
>>
>>
>> For your patch, it looks like there are actually more eal flags test failures: http://mails.dpdk.org/archives/test-report/2023-August/444170.html. In any case, for debugging purposes, I will do a run with the eal_flags_file_prefix_autotest also included and send you the meson test detail log on slack.
>>
> Quick follow up - I see with your patch it does pass on eal_flags_file_prefix_autotest, but as stated fails some others now. The detailed slack logs are DM'ed to you on slack. Thanks!

Thanks for the report.
Interestingly those failures pointed out that I broke secondary
processes when no port gets initialised in the primary process.

I'll send a v3.
Patrick Robb Aug. 21, 2023, 2:32 p.m. UTC | #15 
Hi David,

Responding to your Slack request about merging Bruce's patch ahead of yours
here so there is more visibility.

That is fine, but as you point out it will mean a temporary loss of test
coverage, as it will disable arm unit testing until either 1. your patch
fixing the flags test hits mainline and we can re-enable testing without
failing everyone's patchseries or 2. Bruce's RFC ([RFC PATCH] app/test: add
support for skipping tests) is implemented and made a part of mainline. I
agree with you this is the best thing to do so as to not hold up Bruce and
his series.

On Mon, Aug 21, 2023 at 3:13 AM David Marchand <david.marchand@redhat.com>
wrote:

> On Sat, Aug 19, 2023 at 1:26 AM Patrick Robb <probb@iol.unh.edu> wrote:
> > On Fri, Aug 18, 2023 at 6:33 PM Patrick Robb <probb@iol.unh.edu> wrote:
> >>
> >>
> >> For your patch, it looks like there are actually more eal flags test
> failures:
> http://mails.dpdk.org/archives/test-report/2023-August/444170.html. In
> any case, for debugging purposes, I will do a run with the
> eal_flags_file_prefix_autotest also included and send you the meson test
> detail log on slack.
> >>
> > Quick follow up - I see with your patch it does pass on
> eal_flags_file_prefix_autotest, but as stated fails some others now. The
> detailed slack logs are DM'ed to you on slack. Thanks!
>
> Thanks for the report.
> Interestingly those failures pointed out that I broke secondary
> processes when no port gets initialised in the primary process.
>
> I'll send a v3.
>
>
> --
> David Marchand
>
>
diff mbox series

Patch

diff --git a/app/meson.build b/app/meson.build
index 4fc1a83eba..0d8b618e7f 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -32,6 +32,11 @@  apps = [
         'test-security-perf',
 ]
 
+if get_option('tests')
+# build the auto test app if enabled.
+    apps += 'test'
+endif
+
 default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
 default_ldflags = []
 if get_option('default_library') == 'static' and not is_windows
@@ -106,6 +111,3 @@  foreach app:apps
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
 endforeach
-
-# special case the autotests
-subdir('test')
diff --git a/app/test/meson.build b/app/test/meson.build
index 66897c14a3..2b885ae273 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -1,446 +1,230 @@ 
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
-
-if not get_option('tests')
-    subdir_done()
-endif
-
-test_sources = files(
-        'commands.c',
-        'packet_burst_generator.c',
-        'test.c',
-        'test_acl.c',
-        'test_alarm.c',
-        'test_atomic.c',
-        'test_barrier.c',
-        'test_bitops.c',
-        'test_bitmap.c',
-        'test_bpf.c',
-        'test_byteorder.c',
-        'test_cksum.c',
-        'test_cksum_perf.c',
-        'test_cmdline.c',
-        'test_cmdline_cirbuf.c',
-        'test_cmdline_etheraddr.c',
-        'test_cmdline_ipaddr.c',
-        'test_cmdline_lib.c',
-        'test_cmdline_num.c',
-        'test_cmdline_portlist.c',
-        'test_cmdline_string.c',
-        'test_common.c',
-        'test_cpuflags.c',
-        'test_crc.c',
-        'test_cryptodev.c',
-        'test_cryptodev_asym.c',
-        'test_cryptodev_blockcipher.c',
-        'test_cryptodev_crosscheck.c',
-        'test_cryptodev_security_ipsec.c',
-        'test_cryptodev_security_pdcp.c',
-        'test_cycles.c',
-        'test_debug.c',
-        'test_devargs.c',
-        'test_distributor.c',
-        'test_distributor_perf.c',
-        'test_dmadev.c',
-        'test_dmadev_api.c',
-        'test_eal_flags.c',
-        'test_eal_fs.c',
-        'test_efd.c',
-        'test_efd_perf.c',
-        'test_errno.c',
-        'test_ethdev_link.c',
-        'test_event_crypto_adapter.c',
-        'test_event_eth_rx_adapter.c',
-        'test_event_ring.c',
-        'test_event_timer_adapter.c',
-        'test_eventdev.c',
-        'test_external_mem.c',
-        'test_fbarray.c',
-        'test_fib.c',
-        'test_fib_perf.c',
-        'test_fib6.c',
-        'test_fib6_perf.c',
-        'test_func_reentrancy.c',
-        'test_hash.c',
-        'test_hash_functions.c',
-        'test_hash_multiwriter.c',
-        'test_hash_readwrite.c',
-        'test_hash_perf.c',
-        'test_hash_readwrite_lf_perf.c',
-        'test_interrupts.c',
-        'test_ipfrag.c',
-        'test_ipsec.c',
-        'test_ipsec_sad.c',
-        'test_ipsec_perf.c',
-        'test_kvargs.c',
-        'test_lcores.c',
-        'test_logs.c',
-        'test_lpm.c',
-        'test_lpm6.c',
-        'test_lpm6_perf.c',
-        'test_lpm_perf.c',
-        'test_malloc.c',
-        'test_malloc_perf.c',
-        'test_mbuf.c',
-        'test_member.c',
-        'test_member_perf.c',
-        'test_memcpy.c',
-        'test_memcpy_perf.c',
-        'test_memory.c',
-        'test_mempool.c',
-        'test_mempool_perf.c',
-        'test_memzone.c',
-        'test_meter.c',
-        'test_mcslock.c',
-        'test_mp_secondary.c',
-        'test_per_lcore.c',
-        'test_pflock.c',
-        'test_pmd_perf.c',
-        'test_power.c',
-        'test_power_cpufreq.c',
-        'test_power_kvm_vm.c',
-        'test_power_intel_uncore.c',
-        'test_prefetch.c',
-        'test_rand_perf.c',
-        'test_rawdev.c',
-        'test_rcu_qsbr.c',
-        'test_rcu_qsbr_perf.c',
-        'test_reassembly_perf.c',
-        'test_reciprocal_division.c',
-        'test_reciprocal_division_perf.c',
-        'test_red.c',
-        'test_pie.c',
-        'test_reorder.c',
-        'test_rib.c',
-        'test_rib6.c',
-        'test_ring.c',
-        'test_ring_mpmc_stress.c',
-        'test_ring_hts_stress.c',
-        'test_ring_mt_peek_stress.c',
-        'test_ring_mt_peek_stress_zc.c',
-        'test_ring_perf.c',
-        'test_ring_rts_stress.c',
-        'test_ring_st_peek_stress.c',
-        'test_ring_st_peek_stress_zc.c',
-        'test_ring_stress.c',
-        'test_rwlock.c',
-        'test_sched.c',
-        'test_security.c',
-        'test_security_inline_macsec.c',
-        'test_security_inline_proto.c',
-        'test_seqlock.c',
-        'test_service_cores.c',
-        'test_spinlock.c',
-        'test_stack.c',
-        'test_stack_perf.c',
-        'test_string_fns.c',
-        'test_tailq.c',
-        'test_thash.c',
-        'test_thash_perf.c',
-        'test_threads.c',
-        'test_timer.c',
-        'test_timer_perf.c',
-        'test_timer_racecond.c',
-        'test_timer_secondary.c',
-        'test_ticketlock.c',
-        'test_trace.c',
-        'test_trace_register.c',
-        'test_trace_perf.c',
-        'test_version.c',
-        'virtual_pmd.c',
-)
-
-test_deps = dpdk_libs_enabled
-# as well as libs, the pci and vdev bus drivers are needed for a lot of tests
-test_deps += ['bus_pci', 'bus_vdev']
-
-# Each test is marked with flags:
-# - the first flag indicates whether the test can run in no-huge mode,
-# - the second flag indicates whether the test can run with ASan enabled,
-fast_tests = [
-        ['acl_autotest', true, true],
-        ['atomic_autotest', false, true],
-        ['bitmap_autotest', true, true],
-        ['bpf_autotest', true, true],
-        ['bpf_convert_autotest', true, true],
-        ['bitops_autotest', true, true],
-        ['byteorder_autotest', true, true],
-        ['cksum_autotest', true, true],
-        ['cmdline_autotest', true, true],
-        ['common_autotest', true, true],
-        ['cpuflags_autotest', true, true],
-        ['debug_autotest', true, true],
-        ['devargs_autotest', true, true],
-        ['eal_flags_c_opt_autotest', false, false],
-        ['eal_flags_main_opt_autotest', false, false],
-        ['eal_flags_n_opt_autotest', false, false],
-        ['eal_flags_hpet_autotest', false, false],
-        ['eal_flags_no_huge_autotest', false, false],
-        ['eal_flags_a_opt_autotest', false, false],
-        ['eal_flags_b_opt_autotest', false, false],
-        ['eal_flags_vdev_opt_autotest', false, false],
-        ['eal_flags_r_opt_autotest', false, false],
-        ['eal_flags_mem_autotest', false, false],
-        ['eal_flags_file_prefix_autotest', false, false],
-        ['eal_flags_misc_autotest', false, false],
-        ['eal_fs_autotest', true, true],
-        ['errno_autotest', true, true],
-        ['ethdev_link_status', true, true],
-        ['event_ring_autotest', true, true],
-        ['fib_autotest', true, true],
-        ['fib6_autotest', true, true],
-        ['func_reentrancy_autotest', false, true],
-        ['hash_autotest', true, true],
-        ['interrupt_autotest', true, true],
-        ['ipfrag_autotest', false, true],
-        ['lcores_autotest', true, true],
-        ['logs_autotest', true, true],
-        ['lpm_autotest', true, true],
-        ['lpm6_autotest', true, true],
-        ['malloc_autotest', false, true],
-        ['mbuf_autotest', false, true],
-        ['mcslock_autotest', false, true],
-        ['memcpy_autotest', true, true],
-        ['memory_autotest', false, true],
-        ['mempool_autotest', false, true],
-        ['memzone_autotest', false, true],
-        ['meter_autotest', true, true],
-        ['multiprocess_autotest', false, false],
-        ['per_lcore_autotest', true, true],
-        ['pflock_autotest', true, true],
-        ['prefetch_autotest', true, true],
-        ['rcu_qsbr_autotest', true, true],
-        ['pie_autotest', true, true],
-        ['rib_autotest', true, true],
-        ['rib6_autotest', true, true],
-        ['ring_autotest', true, true],
-        ['rwlock_test1_autotest', true, true],
-        ['rwlock_rda_autotest', true, true],
-        ['rwlock_rds_wrm_autotest', true, true],
-        ['rwlock_rde_wro_autotest', true, true],
-        ['sched_autotest', true, true],
-        ['security_autotest', false, true],
-        ['seqlock_autotest', true, true],
-        ['spinlock_autotest', true, true],
-        ['stack_autotest', false, true],
-        ['stack_lf_autotest', false, true],
-        ['string_autotest', true, true],
-        ['tailq_autotest', true, true],
-        ['ticketlock_autotest', true, true],
-        ['timer_autotest', false, true],
-        ['user_delay_us', true, true],
-        ['version_autotest', true, true],
-        ['crc_autotest', true, true],
-        ['distributor_autotest', false, true],
-        ['eventdev_common_autotest', true, true],
-        ['fbarray_autotest', true, true],
-        ['hash_readwrite_func_autotest', false, true],
-        ['ipsec_autotest', true, true],
-        ['kvargs_autotest', true, true],
-        ['member_autotest', true, true],
-        ['power_cpufreq_autotest', false, true],
-        ['power_autotest', true, true],
-        ['power_kvm_vm_autotest', false, true],
-        ['power_intel_uncore_autotest', true, true],
-        ['reorder_autotest', true, true],
-        ['service_autotest', true, true],
-        ['thash_autotest', true, true],
-        ['threads_autotest', true, true],
-        ['trace_autotest', true, true],
-]
-
-# Tests known to have issues or which don't belong in other tests lists.
-extra_test_names = [
-        'alarm_autotest', # ee00af60170b ("test: remove strict timing requirements some tests")
-        'red_autotest', # https://bugs.dpdk.org/show_bug.cgi?id=826
-]
-
-perf_test_names = [
-        'ring_perf_autotest',
-        'malloc_perf_autotest',
-        'mempool_perf_autotest',
-        'memcpy_perf_autotest',
-        'hash_perf_autotest',
-        'timer_perf_autotest',
-        'reciprocal_division',
-        'reciprocal_division_perf',
-        'lpm_perf_autotest',
-        'rib_slow_autotest',
-        'fib_slow_autotest',
-        'fib_perf_autotest',
-        'red_all',
-        'pie_all',
-        'barrier_autotest',
-        'hash_multiwriter_autotest',
-        'timer_racecond_autotest',
-        'efd_autotest',
-        'hash_functions_autotest',
-        'member_perf_autotest',
-        'efd_perf_autotest',
-        'lpm6_perf_autotest',
-        'rib6_slow_autotest',
-        'fib6_slow_autotest',
-        'fib6_perf_autotest',
-        'rcu_qsbr_perf_autotest',
-        'red_perf',
-        'pie_perf',
-        'distributor_perf_autotest',
-        'pmd_perf_autotest',
-        'service_perf_autotest',
-        'stack_perf_autotest',
-        'stack_lf_perf_autotest',
-        'rand_perf_autotest',
-        'hash_readwrite_perf_autotest',
-        'hash_readwrite_lf_perf_autotest',
-        'trace_perf_autotest',
-        'ipsec_perf_autotest',
-        'thash_perf_autotest',
-        'reassembly_perf_autotest',
-]
-
-driver_test_names = [
-        'cryptodev_aesni_gcm_autotest',
-        'cryptodev_aesni_mb_autotest',
-        'cryptodev_chacha_poly_mb_autotest',
-        'cryptodev_cn10k_autotest',
-        'cryptodev_cn9k_autotest',
-        'cryptodev_cpu_aesni_mb_autotest',
-        'cryptodev_cpu_aesni_gcm_autotest',
-        'cryptodev_dpaa2_sec_autotest',
-        'cryptodev_dpaa_sec_autotest',
-        'cryptodev_null_autotest',
-        'cryptodev_openssl_autotest',
-        'cryptodev_qat_autotest',
-        'cryptodev_qat_asym_autotest',
-        'cryptodev_qat_raw_api_autotest',
-        'cryptodev_sw_armv8_autotest',
-        'cryptodev_sw_kasumi_autotest',
-        'cryptodev_sw_mvsam_autotest',
-        'cryptodev_sw_snow3g_autotest',
-        'cryptodev_sw_zuc_autotest',
-        'cryptodev_uadk_autotest',
-        'dmadev_autotest',
-]
-
-dump_test_names = []
-
-if not is_windows
-    driver_test_names += [
-            'cryptodev_openssl_asym_autotest',
-            'eventdev_selftest_octeontx',
-            'eventdev_selftest_sw',
-    ]
-
-    dump_test_names += [
-            'dump_struct_sizes',
-            'dump_mempool',
-            'dump_malloc_stats',
-            'dump_devargs',
-            'dump_log_types',
-            'dump_ring',
-            'dump_physmem',
-            'dump_memzone',
-    ]
-endif
-
-# The following linkages are an exception to allow running the
-# unit tests without requiring that the developer install the
-# DPDK libraries.  Explicit linkage of drivers (plugin libraries)
-# in applications should not be used.
-if dpdk_conf.has('RTE_MEMPOOL_RING')
-    test_deps += 'mempool_ring'
-endif
-if dpdk_conf.has('RTE_MEMPOOL_STACK')
-    test_deps += 'mempool_stack'
-endif
-if dpdk_conf.has('RTE_EVENT_SKELETON')
-    test_deps += 'event_skeleton'
-endif
-
-if dpdk_conf.has('RTE_LIB_GRAPH')
-    test_sources += 'test_graph.c'
-    fast_tests += [['graph_autotest', true, true]]
-    fast_tests += [['node_list_dump', true, true]]
-    test_sources += 'test_graph_perf.c'
-    perf_test_names += 'graph_perf_autotest'
-endif
-if dpdk_conf.has('RTE_LIB_METRICS')
-    test_sources += ['test_metrics.c']
-    fast_tests += [['metrics_autotest', true, true]]
-endif
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_sources += ['test_telemetry_json.c', 'test_telemetry_data.c']
-    fast_tests += [['telemetry_json_autotest', true, true]]
-    fast_tests += [['telemetry_data_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_LIB_PIPELINE')
-# pipeline lib depends on port and table libs, so those must be present
-# if pipeline library is.
-    test_sources += [
-            'test_table.c',
-            'test_table_acl.c',
-            'test_table_combined.c',
-            'test_table_pipeline.c',
-            'test_table_ports.c',
-            'test_table_tables.c',
-    ]
-    fast_tests += [['table_autotest', true, true]]
-endif
-
-# The following linkages of drivers are required because
-# they are used via a driver-specific API.
-if dpdk_conf.has('RTE_NET_BOND')
-    test_deps += 'net_bond'
-    test_sources += ['test_link_bonding.c', 'test_link_bonding_rssconf.c']
-    driver_test_names += ['link_bonding_autotest', 'link_bonding_rssconf_autotest']
-    if dpdk_conf.has('RTE_NET_RING')
-        test_sources += 'test_link_bonding_mode4.c'
-        driver_test_names += 'link_bonding_mode4_autotest'
-    endif
-endif
-if dpdk_conf.has('RTE_LIB_EVENTDEV') and dpdk_conf.has('RTE_NET_RING')
-    test_deps += 'net_ring'
-    test_sources += 'test_pmd_ring_perf.c'
-    test_sources += 'test_pmd_ring.c'
-    test_sources += 'test_event_eth_tx_adapter.c'
-    test_sources += 'sample_packet_forward.c'
-    fast_tests += [['ring_pmd_autotest', true, true]]
-    perf_test_names += 'ring_pmd_perf_autotest'
-    fast_tests += [['event_eth_tx_adapter_autotest', false, true]]
-    if dpdk_conf.has('RTE_LIB_BITRATESTATS')
-        test_sources += 'test_bitratestats.c'
-        fast_tests += [['bitratestats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_LATENCYSTATS')
-        test_sources += 'test_latencystats.c'
-        fast_tests += [['latencystats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_PDUMP')
-        test_sources += 'test_pdump.c'
-        fast_tests += [['pdump_autotest', true, false]]
-    endif
-endif
-if dpdk_conf.has('RTE_NET_NULL')
-    test_deps += 'net_null'
-    test_sources += 'test_vdev.c'
-    fast_tests += [['vdev_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_RAW_SKELETON')
-    test_deps += 'raw_skeleton'
-    fast_tests += [['rawdev_autotest', true, true]]
-endif
-
-if dpdk_conf.has('RTE_HAS_LIBPCAP')
-    ext_deps += pcap_dep
-    if dpdk_conf.has('RTE_LIB_PCAPNG')
-        test_sources += 'test_pcapng.c'
+# Copyright(c) 2017-2023 Intel Corporation
+
+# the main test files [test.c and commands.c] relies on these libraries
+deps += ['cmdline', 'ring', 'mempool', 'mbuf']
+sources += files('commands.c', 'test.c')
+
+# some other utility C files, providing functions used by various tests
+# so we need to include these deps in the dependency list for the files using those fns.
+packet_burst_generator_deps = ['net']
+sample_packet_forward_deps = ['net_ring', 'ethdev', 'bus_vdev']
+virtual_pmd_deps = ['ethdev', 'net', 'bus_pci']
+# test_cryptodev has material that other crypto tests need
+test_cryptodev_deps = ['bus_vdev', 'net', 'cryptodev', 'crypto_scheduler', 'security']
+
+source_file_deps = {
+    # The C files providing functionality to other test cases
+    'packet_burst_generator.c': packet_burst_generator_deps,
+#    'resource.c': [],          # unused currently.
+    'sample_packet_forward.c': sample_packet_forward_deps,
+    'virtual_pmd.c': virtual_pmd_deps,
+
+    # the various test_*.c files
+    'test_acl.c': ['net', 'acl'],
+    'test_alarm.c': [],
+    'test_atomic.c': ['hash'],
+    'test_barrier.c': [],
+    'test_bitmap.c': [],
+    'test_bitops.c': [],
+    'test_bitratestats.c': ['metrics', 'bitratestats', 'ethdev'] + sample_packet_forward_deps,
+    'test_bpf.c': ['bpf', 'net'],
+    'test_byteorder.c': [],
+#    'test_cfgfile.c': ['cfgfile'],
+    'test_cksum.c': ['net'],
+    'test_cksum_perf.c': ['net'],
+    'test_cmdline.c': [],
+    'test_cmdline_cirbuf.c': [],
+    'test_cmdline_etheraddr.c': ['net'],
+    'test_cmdline_ipaddr.c': [],
+    'test_cmdline_lib.c': [],
+    'test_cmdline_num.c': [],
+    'test_cmdline_portlist.c': [],
+    'test_cmdline_string.c': [],
+    'test_common.c': [],
+    'test_compressdev.c': ['compressdev'],
+    'test_cpuflags.c': [],
+    'test_crc.c': ['net'],
+    'test_cryptodev.c': test_cryptodev_deps,
+    'test_cryptodev_asym.c': ['bus_vdev'] + test_cryptodev_deps,
+    'test_cryptodev_blockcipher.c': test_cryptodev_deps,
+    'test_cryptodev_crosscheck.c': test_cryptodev_deps,
+    'test_cryptodev_security_ipsec.c': test_cryptodev_deps,
+    'test_cryptodev_security_pdcp.c': test_cryptodev_deps,
+    'test_cycles.c': [],
+    'test_debug.c': [],
+    'test_devargs.c': ['kvargs'],
+    'test_distributor.c': ['distributor'],
+    'test_distributor_perf.c': ['distributor'],
+    'test_dmadev.c': ['dmadev', 'bus_vdev'],
+    'test_dmadev_api.c': ['dmadev'],
+    'test_eal_flags.c': [],
+    'test_eal_fs.c': [],
+    'test_efd.c': ['efd', 'net'],
+    'test_efd_perf.c': ['efd', 'hash'],
+    'test_errno.c': [],
+    'test_ethdev_link.c': ['ethdev'],
+    'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
+    'test_event_ring.c': ['eventdev'],
+    'test_event_timer_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_eventdev.c': ['eventdev', 'bus_vdev'],
+    'test_external_mem.c': [],
+    'test_fbarray.c': [],
+    'test_fib.c': ['net', 'fib'],
+    'test_fib6.c': ['rib', 'fib'],
+    'test_fib6_perf.c': ['fib'],
+    'test_fib_perf.c': ['net', 'fib'],
+    'test_flow_classify.c': ['net', 'acl', 'table', 'ethdev', 'flow_classify'],
+    'test_func_reentrancy.c': ['hash', 'lpm'],
+    'test_graph.c': ['graph'],
+    'test_graph_perf.c': ['graph'],
+    'test_hash.c': ['net', 'hash'],
+    'test_hash_functions.c': ['hash'],
+    'test_hash_multiwriter.c': ['hash'],
+    'test_hash_perf.c': ['hash'],
+    'test_hash_readwrite.c': ['hash'],
+    'test_hash_readwrite_lf_perf.c': ['hash'],
+    'test_interrupts.c': [],
+    'test_ipfrag.c': ['net', 'ip_frag'],
+    'test_ipsec.c': ['bus_vdev', 'net', 'cryptodev', 'ipsec', 'security'],
+    'test_ipsec_perf.c': ['net', 'ipsec'],
+    'test_ipsec_sad.c': ['ipsec'],
+    'test_kvargs.c': ['kvargs'],
+    'test_latencystats.c': ['ethdev', 'latencystats', 'metrics'] + sample_packet_forward_deps,
+    'test_lcores.c': [],
+    'test_link_bonding.c': ['ethdev', 'net_bond',
+        'net'] + packet_burst_generator_deps + virtual_pmd_deps,
+    'test_link_bonding_mode4.c': ['ethdev', 'net_ring', 'net_bond',
+        'net'] + packet_burst_generator_deps,
+    'test_link_bonding_rssconf.c': ['ethdev', 'bus_vdev', 'net_bond'],
+    'test_logs.c': [],
+    'test_lpm.c': ['net', 'lpm'],
+    'test_lpm6.c': ['lpm'],
+    'test_lpm6_perf.c': ['lpm'],
+    'test_lpm_perf.c': ['net', 'lpm'],
+    'test_malloc.c': [],
+    'test_malloc_perf.c': [],
+    'test_mbuf.c': ['net'],
+    'test_mcslock.c': [],
+    'test_member.c': ['member', 'net'],
+    'test_member_perf.c': ['hash', 'member'],
+    'test_memcpy.c': [],
+    'test_memcpy_perf.c': [],
+    'test_memory.c': [],
+    'test_mempool.c': [],
+    'test_mempool_perf.c': [],
+    'test_memzone.c': [],
+    'test_meter.c': ['meter'],
+    'test_metrics.c': ['metrics'],
+    'test_mp_secondary.c': ['hash', 'lpm'],
+    'test_pcapng.c': ['ethdev', 'net', 'pcapng'],
+    'test_pdcp.c': ['eventdev', 'pdcp', 'net', 'timer', 'security'],
+    'test_pdump.c': ['pdump'] + sample_packet_forward_deps,
+    'test_per_lcore.c': [],
+    'test_pflock.c': [],
+    'test_pie.c': ['sched'],
+    'test_pmd_perf.c': ['ethdev', 'net'] + packet_burst_generator_deps,
+    'test_pmd_ring.c': ['net_ring', 'ethdev', 'bus_vdev'],
+    'test_pmd_ring_perf.c': ['ethdev', 'net_ring', 'bus_vdev'],
+    'test_power.c': ['power'],
+    'test_power_cpufreq.c': ['power'],
+    'test_power_intel_uncore.c': ['power'],
+    'test_power_kvm_vm.c': ['power'],
+    'test_prefetch.c': [],
+    'test_rand_perf.c': [],
+    'test_rawdev.c': ['rawdev', 'bus_vdev'],
+    'test_rcu_qsbr.c': ['rcu', 'hash'],
+    'test_rcu_qsbr_perf.c': ['rcu', 'hash'],
+    'test_reassembly_perf.c': ['net', 'ip_frag'],
+    'test_reciprocal_division.c': [],
+    'test_reciprocal_division_perf.c': [],
+    'test_red.c': ['sched'],
+    'test_reorder.c': ['reorder'],
+#    'test_resource.c': [],
+    'test_rib.c': ['net', 'rib'],
+    'test_rib6.c': ['net', 'rib'],
+    'test_ring.c': [],
+    'test_ring_hts_stress.c': [],
+    'test_ring_mpmc_stress.c': [],
+    'test_ring_mt_peek_stress.c': [],
+    'test_ring_mt_peek_stress_zc.c': [],
+    'test_ring_perf.c': [],
+    'test_ring_rts_stress.c': [],
+    'test_ring_st_peek_stress.c': [],
+    'test_ring_st_peek_stress_zc.c': [],
+    'test_ring_stress.c': [],
+    'test_rwlock.c': [],
+    'test_sched.c': ['net', 'sched'],
+    'test_security.c': ['net', 'security'],
+    'test_security_inline_macsec.c': ['ethdev', 'security'],
+    'test_security_inline_proto.c': ['ethdev', 'security', 'eventdev'] + test_cryptodev_deps,
+    'test_seqlock.c': [],
+    'test_service_cores.c': [],
+    'test_spinlock.c': [],
+    'test_stack.c': ['stack'],
+    'test_stack_perf.c': ['stack'],
+    'test_string_fns.c': [],
+    'test_table.c': ['table', 'pipeline', 'port'],
+    'test_table_acl.c': ['net', 'table', 'pipeline', 'port'],
+    'test_table_combined.c': ['table', 'pipeline', 'port'],
+    'test_table_pipeline.c': ['pipeline', 'table', 'port'],
+    'test_table_ports.c': ['table', 'pipeline', 'port'],
+    'test_table_tables.c': ['table', 'pipeline', 'port'],
+    'test_tailq.c': [],
+    'test_telemetry_data.c': ['telemetry'],
+    'test_telemetry_json.c': ['telemetry'],
+    'test_thash.c': ['net', 'hash'],
+    'test_thash_perf.c': ['hash'],
+    'test_threads.c': [],
+    'test_ticketlock.c': [],
+    'test_timer.c': ['timer'],
+    'test_timer_perf.c': ['timer'],
+    'test_timer_racecond.c': ['timer'],
+    'test_timer_secondary.c': ['timer'],
+    'test_trace.c': [],
+    'test_trace_perf.c': [],
+    'test_trace_register.c': [],
+    'test_vdev.c': ['kvargs', 'bus_vdev'],
+    'test_version.c': [],
+}
+
+source_file_ext_deps = {
+    'test_compressdev.c': ['zlib'],
+    'test_pcapng.c': ['pcap'],
+}
+
+def_lib = get_option('default_library')
+foreach f, f_deps : source_file_deps
+    has_deps = true
+    foreach d : f_deps
+        if not is_variable(def_lib + '_rte_' + d)
+            has_deps = false
+            break
+        else
+            # technically we might not need this dep, but adding it is harmless
+            if d not in deps
+                deps += d
+            endif
+        endif
+    endforeach
+    # check for any external dependencies for this file
+    if source_file_ext_deps.has_key(f)
+        foreach d: source_file_ext_deps.get(f)
+            dep = dependency(d, required: false, method: 'pkg-config')
+            if not dep.found()
+                message('Skipping test file @0@ due to missing external dependency @1@'.format(f, d))
+                has_deps = false
+            else
+                ext_deps += dep
+            endif
+        endforeach
+    endif
+    if has_deps
+        sources += files(f)
     endif
-endif
-
-if dpdk_conf.has('RTE_LIB_PDCP')
-    test_sources += 'test_pdcp.c'
-    fast_tests += [['pdcp_autotest', false, true]]
-endif
+endforeach
 
 if cc.has_argument('-Wno-format-truncation')
     cflags += '-Wno-format-truncation'
@@ -450,154 +234,13 @@  endif
 cflags += '-fno-strict-aliasing'
 
 # Enable using internal APIs in unit tests
-cflags += ['-DALLOW_INTERNAL_API']
-
-test_dep_objs = []
-if dpdk_conf.has('RTE_LIB_COMPRESSDEV')
-    compress_test_dep = dependency('zlib', required: false, method: 'pkg-config')
-    if compress_test_dep.found()
-        test_dep_objs += compress_test_dep
-        test_sources += 'test_compressdev.c'
-        fast_tests += [['compressdev_autotest', false, true]]
-    endif
-endif
-
-if dpdk_conf.has('RTE_CRYPTO_SCHEDULER')
-    driver_test_names += 'cryptodev_scheduler_autotest'
-    test_deps += 'crypto_scheduler'
-endif
-
-foreach d:test_deps
-    def_lib = get_option('default_library')
-    test_dep_objs += get_variable(def_lib + '_rte_' + d)
-endforeach
-
-link_libs = []
-if get_option('default_library') == 'static'
-    link_libs = dpdk_static_libraries + dpdk_drivers
-endif
-
-dpdk_test = executable('dpdk-test',
-        test_sources,
-        link_whole: link_libs,
-        dependencies: test_dep_objs + ext_deps,
-        c_args: cflags,
-        install_rpath: join_paths(get_option('prefix'),
-             driver_install_path),
-        install: true)
-
-has_hugepage = run_command(py3, files('has_hugepage.py'), check: true).stdout().strip() != '0'
-message('hugepage availability: @0@'.format(has_hugepage))
-
-# some perf tests (eg: memcpy perf autotest)take very long
-# to complete, so timeout to 10 minutes
-timeout_seconds = 600
-timeout_seconds_fast = 10
-
-test_no_huge_args = ['--no-huge', '-m', '2048']
-
-foreach arg : fast_tests
-    test_args = []
-    run_test = true
-    if not has_hugepage
-        if arg[1]
-            test_args += test_no_huge_args
-        else
-            run_test = false
-        endif
-    endif
-
-    if get_option('b_sanitize') == 'address' or get_option('b_sanitize') == 'address,undefined'
-        if not arg[2]
-            run_test = false
-        endif
-    endif
-
-    if (get_option('default_library') == 'shared' and
-        arg[0] == 'event_eth_tx_adapter_autotest')
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if is_linux
-        test_args += ['--file-prefix=@0@'.format(arg[0])]
-    endif
-
-    if run_test
-        test(arg[0], dpdk_test,
-                env : ['DPDK_TEST=' + arg[0]],
-                args : test_args,
-                timeout : timeout_seconds_fast,
-                is_parallel : false,
-                suite : 'fast-tests')
-        if not is_windows and arg[0] == 'trace_autotest'
-            test_args += ['--trace=.*']
-            test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
-            test(arg[0] + '_with_traces', dpdk_test,
-                    env : ['DPDK_TEST=' + arg[0]],
-                    args : test_args,
-                    timeout : timeout_seconds_fast,
-                    is_parallel : false,
-                    suite : 'fast-tests')
-        endif
-    endif
-endforeach
+cflags += '-DALLOW_INTERNAL_API'
 
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_args = [dpdk_test]
-    test_args += test_no_huge_args
-    if get_option('default_library') == 'shared'
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if dpdk_conf.has('RTE_CRYPTO_NULL')
-        test_args += ['--vdev=crypto_null0']
-    endif
-    if dpdk_conf.has('RTE_DMA_SKELETON')
-        test_args += ['--vdev=dma_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_EVENT_SKELETON')
-        test_args += ['--vdev=event_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_NET_NULL')
-        test_args += ['--vdev=net_null0']
-    endif
-    if dpdk_conf.has('RTE_RAW_SKELETON')
-        test_args += ['--vdev=rawdev_skeleton0']
-    endif
-    test_args += ['-a', '0000:00:00.0']
-    test('telemetry_all', find_program('test_telemetry.sh'),
-            args: test_args,
-            timeout : timeout_seconds_fast,
-            is_parallel : false,
-            suite : 'fast-tests')
+# create a symlink in the app/test directory for the binary, for backward compatibility
+if not is_windows
+    custom_target('test_symlink',
+            output: 'dpdk-test',
+            command: ['ln', '-sf', '../dpdk-test', '@OUTPUT@'],
+            build_by_default: true,
+            install: false)
 endif
-
-foreach arg : perf_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'perf-tests')
-endforeach
-
-foreach arg : driver_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'driver-tests')
-endforeach
-
-foreach arg : dump_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'debug-tests')
-endforeach
-
-foreach arg : extra_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'extra-tests')
-endforeach