[v3,0/8] eal: cleanup resources on shutdown

  Started using valgrind with DPDK, and there are lots of leftover
memory and file descriptors. This makes it hard to find application
leaks versus DPDK leaks.

The DPDK has a function that applications can use to tell it
to cleanup resources on shutdown (rte_eal_cleanup). But the
current coverage of that API is spotty. Many internal parts of
DPDK leave files and allocated memory behind.

This patch set is a first step at getting the sub-parts of
DPDK to cleanup after themselves. These are the easier ones,
the harder and more critical ones are in the drivers
and the memory subsystem.

There are no new exposed API or ABI changes here.

v3
 - fix a couple of minor checkpatch complaints

v2
 - rebase after 20.05 file renames
 - incorporate review comment feedback
 - hold off some of the more involved patches for later

Stephen Hemminger (8):
  eal: log: close on cleanup
  eal: log: free dynamic state on cleanup
  eal: alarm: close file on cleanup
  eal: cleanup threads
  eal: mp: end the multiprocess thread during cleanup
  eal: vfio: cleanup the mp sync handle
  eal: hotplug: cleanup multiprocess resources
  eal: malloc: cleanup mp resources

 lib/librte_eal/common/eal_common_log.c  | 33 ++++++++++++++++++++++++-
 lib/librte_eal/common/eal_common_proc.c | 17 +++++++++----
 lib/librte_eal/common/eal_private.h     | 20 +++++++++++++++
 lib/librte_eal/common/hotplug_mp.c      |  5 ++++
 lib/librte_eal/common/hotplug_mp.h      |  6 +++++
 lib/librte_eal/common/malloc_heap.c     |  6 +++++
 lib/librte_eal/common/malloc_heap.h     |  3 +++
 lib/librte_eal/common/malloc_mp.c       | 12 +++++++++
 lib/librte_eal/common/malloc_mp.h       |  3 +++
 lib/librte_eal/freebsd/eal.c            |  1 +
 lib/librte_eal/freebsd/eal_alarm.c      | 10 ++++++++
 lib/librte_eal/linux/eal.c              | 20 +++++++++++++++
 lib/librte_eal/linux/eal_alarm.c        | 11 +++++++++
 lib/librte_eal/linux/eal_log.c          | 14 +++++++++++
 lib/librte_eal/linux/eal_vfio.h         |  1 +
 lib/librte_eal/linux/eal_vfio_mp_sync.c |  8 ++++++
 16 files changed, 164 insertions(+), 6 deletions(-)
  

On Wed, Apr 29, 2020 at 1:58 AM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> Started using valgrind with DPDK, and there are lots of leftover
> memory and file descriptors. This makes it hard to find application
> leaks versus DPDK leaks.
>
> The DPDK has a function that applications can use to tell it
> to cleanup resources on shutdown (rte_eal_cleanup). But the
> current coverage of that API is spotty. Many internal parts of
> DPDK leave files and allocated memory behind.
>
> This patch set is a first step at getting the sub-parts of
> DPDK to cleanup after themselves. These are the easier ones,
> the harder and more critical ones are in the drivers
> and the memory subsystem.
>
> There are no new exposed API or ABI changes here.
>
> v3
>  - fix a couple of minor checkpatch complaints
>
> v2
>  - rebase after 20.05 file renames
>  - incorporate review comment feedback
>  - hold off some of the more involved patches for later

Same segfault as v1.

$ ./devtools/test-null.sh ./build/app/dpdk-testpmd 0x3 --plop
./build/app/dpdk-testpmd: unrecognized option '--plop'
EAL: Detected 8 lcore(s)
EAL: Detected 1 NUMA nodes

Usage: ./build/app/dpdk-testpmd [options]

(snip)

EAL: FATAL: Invalid 'command line' arguments.
EAL: Invalid 'command line' arguments.
EAL: Error - exiting with code: 1
  Cause: Cannot init EAL: Invalid argument
./devtools/test-null.sh: line 32: 23134 Broken pipe             (
sleep 1 && echo stop )
     23135 Segmentation fault      (core dumped) | $testpmd -c
$coremask --no-huge -m 20 $libs -w 0:0.0 --vdev net_null1 --vdev
net_null2 $eal_options -- --no-mlockall --total-num-mbufs=2048
$testpmd_options -ia
  

That's a pity this patchset is not concluded.
Please Stephen, could you respin with a fix?


03/05/2020 19:21, David Marchand:
> On Wed, Apr 29, 2020 at 1:58 AM Stephen Hemminger
> <stephen@networkplumber.org> wrote:
> >
> > Started using valgrind with DPDK, and there are lots of leftover
> > memory and file descriptors. This makes it hard to find application
> > leaks versus DPDK leaks.
> >
> > The DPDK has a function that applications can use to tell it
> > to cleanup resources on shutdown (rte_eal_cleanup). But the
> > current coverage of that API is spotty. Many internal parts of
> > DPDK leave files and allocated memory behind.
> >
> > This patch set is a first step at getting the sub-parts of
> > DPDK to cleanup after themselves. These are the easier ones,
> > the harder and more critical ones are in the drivers
> > and the memory subsystem.
> >
> > There are no new exposed API or ABI changes here.
> >
> > v3
> >  - fix a couple of minor checkpatch complaints
> >
> > v2
> >  - rebase after 20.05 file renames
> >  - incorporate review comment feedback
> >  - hold off some of the more involved patches for later
> 
> Same segfault as v1.
> 
> $ ./devtools/test-null.sh ./build/app/dpdk-testpmd 0x3 --plop
> ./build/app/dpdk-testpmd: unrecognized option '--plop'
> EAL: Detected 8 lcore(s)
> EAL: Detected 1 NUMA nodes
> 
> Usage: ./build/app/dpdk-testpmd [options]
> 
> (snip)
> 
> EAL: FATAL: Invalid 'command line' arguments.
> EAL: Invalid 'command line' arguments.
> EAL: Error - exiting with code: 1
>   Cause: Cannot init EAL: Invalid argument
> ./devtools/test-null.sh: line 32: 23134 Broken pipe             (
> sleep 1 && echo stop )
>      23135 Segmentation fault      (core dumped) | $testpmd -c
> $coremask --no-huge -m 20 $libs -w 0:0.0 --vdev net_null1 --vdev
> net_null2 $eal_options -- --no-mlockall --total-num-mbufs=2048
> $testpmd_options -ia
> 
> 
>
  

Ping

20/10/2020 00:24, Thomas Monjalon:
> That's a pity this patchset is not concluded.
> Please Stephen, could you respin with a fix?
> 
> 
> 03/05/2020 19:21, David Marchand:
> > On Wed, Apr 29, 2020 at 1:58 AM Stephen Hemminger
> > <stephen@networkplumber.org> wrote:
> > >
> > > Started using valgrind with DPDK, and there are lots of leftover
> > > memory and file descriptors. This makes it hard to find application
> > > leaks versus DPDK leaks.
> > >
> > > The DPDK has a function that applications can use to tell it
> > > to cleanup resources on shutdown (rte_eal_cleanup). But the
> > > current coverage of that API is spotty. Many internal parts of
> > > DPDK leave files and allocated memory behind.
> > >
> > > This patch set is a first step at getting the sub-parts of
> > > DPDK to cleanup after themselves. These are the easier ones,
> > > the harder and more critical ones are in the drivers
> > > and the memory subsystem.
> > >
> > > There are no new exposed API or ABI changes here.
> > >
> > > v3
> > >  - fix a couple of minor checkpatch complaints
> > >
> > > v2
> > >  - rebase after 20.05 file renames
> > >  - incorporate review comment feedback
> > >  - hold off some of the more involved patches for later
> > 
> > Same segfault as v1.
> > 
> > $ ./devtools/test-null.sh ./build/app/dpdk-testpmd 0x3 --plop
> > ./build/app/dpdk-testpmd: unrecognized option '--plop'
> > EAL: Detected 8 lcore(s)
> > EAL: Detected 1 NUMA nodes
> > 
> > Usage: ./build/app/dpdk-testpmd [options]
> > 
> > (snip)
> > 
> > EAL: FATAL: Invalid 'command line' arguments.
> > EAL: Invalid 'command line' arguments.
> > EAL: Error - exiting with code: 1
> >   Cause: Cannot init EAL: Invalid argument
> > ./devtools/test-null.sh: line 32: 23134 Broken pipe             (
> > sleep 1 && echo stop )
> >      23135 Segmentation fault      (core dumped) | $testpmd -c
> > $coremask --no-huge -m 20 $libs -w 0:0.0 --vdev net_null1 --vdev
> > net_null2 $eal_options -- --no-mlockall --total-num-mbufs=2048
> > $testpmd_options -ia
> > 
> > 
> > 
> 
> 
> 
> 
> 
>