[v1,0/5] introduce IWYU

This patchset introduces the include-what-you-use script which removes
unused header includes. IWYU GitHub:

https://github.com/include-what-you-use/include-what-you-use

Along with the script there are some patches which make a start on
removing unneeded headers.

Sean Morrissey (5):
  devtools: script to remove unused headers includes
  lib/telemetry: remove unneeded header includes
  lib/ring: remove unneeded header includes
  lib/kvargs: remove unneeded header includes
  lib/eal: remove unneeded header includes

 devtools/process_iwyu.py               | 109 +++++++++++++++++++++++++
 lib/eal/common/eal_common_dev.c        |   5 --
 lib/eal/common/eal_common_devargs.c    |   1 -
 lib/eal/common/eal_common_errno.c      |   4 -
 lib/eal/common/eal_common_fbarray.c    |   3 -
 lib/eal/common/eal_common_hexdump.c    |   3 -
 lib/eal/common/eal_common_launch.c     |   6 --
 lib/eal/common/eal_common_lcore.c      |   6 --
 lib/eal/common/eal_common_log.c        |   2 -
 lib/eal/common/eal_common_memalloc.c   |   3 -
 lib/eal/common/eal_common_memory.c     |   5 --
 lib/eal/common/eal_common_memzone.c    |   4 -
 lib/eal/common/eal_common_options.c    |   2 -
 lib/eal/common/eal_common_proc.c       |   2 -
 lib/eal/common/eal_common_string_fns.c |   2 -
 lib/eal/common/eal_common_tailqs.c     |  11 ---
 lib/eal/common/eal_common_thread.c     |   3 -
 lib/eal/common/eal_common_timer.c      |   6 --
 lib/eal/common/eal_common_trace.c      |   1 -
 lib/eal/common/hotplug_mp.h            |   1 -
 lib/eal/common/malloc_elem.c           |   6 --
 lib/eal/common/malloc_heap.c           |   5 --
 lib/eal/common/malloc_mp.c             |   1 -
 lib/eal/common/malloc_mp.h             |   2 -
 lib/eal/common/rte_malloc.c            |   6 --
 lib/eal/common/rte_random.c            |   3 -
 lib/eal/common/rte_service.c           |   6 --
 lib/eal/include/rte_version.h          |   2 -
 lib/eal/linux/eal.c                    |  10 ---
 lib/eal/linux/eal_alarm.c              |   7 --
 lib/eal/linux/eal_cpuflags.c           |   2 -
 lib/eal/linux/eal_debug.c              |   5 --
 lib/eal/linux/eal_dev.c                |   4 -
 lib/eal/linux/eal_hugepage_info.c      |   8 --
 lib/eal/linux/eal_interrupts.c         |   8 --
 lib/eal/linux/eal_lcore.c              |   7 --
 lib/eal/linux/eal_log.c                |  11 +--
 lib/eal/linux/eal_memalloc.c           |   8 --
 lib/eal/linux/eal_memory.c             |   9 --
 lib/eal/linux/eal_thread.c             |   5 --
 lib/eal/linux/eal_timer.c              |  15 ----
 lib/eal/linux/eal_vfio_mp_sync.c       |   1 -
 lib/eal/unix/eal_file.c                |   1 -
 lib/eal/unix/rte_thread.c              |   1 -
 lib/eal/x86/rte_cycles.c               |   1 -
 lib/kvargs/rte_kvargs.c                |   1 -
 lib/ring/rte_ring.c                    |   7 --
 lib/telemetry/telemetry.c              |   1 -
 lib/telemetry/telemetry_data.h         |   1 -
 49 files changed, 110 insertions(+), 213 deletions(-)
 create mode 100755 devtools/process_iwyu.py
  

On Mon,  4 Oct 2021 10:10:53 +0000
Sean Morrissey <sean.morrissey@intel.com> wrote:

> This patchset introduces the include-what-you-use script which removes
> unused header includes. IWYU GitHub:
> 
> https://github.com/include-what-you-use/include-what-you-use
> 
> Along with the script there are some patches which make a start on
> removing unneeded headers.
> 
> Sean Morrissey (5):
>   devtools: script to remove unused headers includes
>   lib/telemetry: remove unneeded header includes
>   lib/ring: remove unneeded header includes
>   lib/kvargs: remove unneeded header includes
>   lib/eal: remove unneeded header includes
> 
>  devtools/process_iwyu.py               | 109 +++++++++++++++++++++++++
>  lib/eal/common/eal_common_dev.c        |   5 --
>  lib/eal/common/eal_common_devargs.c    |   1 -
>  lib/eal/common/eal_common_errno.c      |   4 -
>  lib/eal/common/eal_common_fbarray.c    |   3 -
>  lib/eal/common/eal_common_hexdump.c    |   3 -
>  lib/eal/common/eal_common_launch.c     |   6 --
>  lib/eal/common/eal_common_lcore.c      |   6 --
>  lib/eal/common/eal_common_log.c        |   2 -
>  lib/eal/common/eal_common_memalloc.c   |   3 -
>  lib/eal/common/eal_common_memory.c     |   5 --
>  lib/eal/common/eal_common_memzone.c    |   4 -
>  lib/eal/common/eal_common_options.c    |   2 -
>  lib/eal/common/eal_common_proc.c       |   2 -
>  lib/eal/common/eal_common_string_fns.c |   2 -
>  lib/eal/common/eal_common_tailqs.c     |  11 ---
>  lib/eal/common/eal_common_thread.c     |   3 -
>  lib/eal/common/eal_common_timer.c      |   6 --
>  lib/eal/common/eal_common_trace.c      |   1 -
>  lib/eal/common/hotplug_mp.h            |   1 -
>  lib/eal/common/malloc_elem.c           |   6 --
>  lib/eal/common/malloc_heap.c           |   5 --
>  lib/eal/common/malloc_mp.c             |   1 -
>  lib/eal/common/malloc_mp.h             |   2 -
>  lib/eal/common/rte_malloc.c            |   6 --
>  lib/eal/common/rte_random.c            |   3 -
>  lib/eal/common/rte_service.c           |   6 --
>  lib/eal/include/rte_version.h          |   2 -
>  lib/eal/linux/eal.c                    |  10 ---
>  lib/eal/linux/eal_alarm.c              |   7 --
>  lib/eal/linux/eal_cpuflags.c           |   2 -
>  lib/eal/linux/eal_debug.c              |   5 --
>  lib/eal/linux/eal_dev.c                |   4 -
>  lib/eal/linux/eal_hugepage_info.c      |   8 --
>  lib/eal/linux/eal_interrupts.c         |   8 --
>  lib/eal/linux/eal_lcore.c              |   7 --
>  lib/eal/linux/eal_log.c                |  11 +--
>  lib/eal/linux/eal_memalloc.c           |   8 --
>  lib/eal/linux/eal_memory.c             |   9 --
>  lib/eal/linux/eal_thread.c             |   5 --
>  lib/eal/linux/eal_timer.c              |  15 ----
>  lib/eal/linux/eal_vfio_mp_sync.c       |   1 -
>  lib/eal/unix/eal_file.c                |   1 -
>  lib/eal/unix/rte_thread.c              |   1 -
>  lib/eal/x86/rte_cycles.c               |   1 -
>  lib/kvargs/rte_kvargs.c                |   1 -
>  lib/ring/rte_ring.c                    |   7 --
>  lib/telemetry/telemetry.c              |   1 -
>  lib/telemetry/telemetry_data.h         |   1 -
>  49 files changed, 110 insertions(+), 213 deletions(-)
>  create mode 100755 devtools/process_iwyu.py
> 

There is a risk of breaking builds on other platforms.
How can you be sure the include files (especially auto generated list)
are the same in Linux, FreeBSD and Windows as well as the special
versions of libc (musl etc).
  

On Mon, Oct 04, 2021 at 08:55:35AM -0700, Stephen Hemminger wrote:
> On Mon,  4 Oct 2021 10:10:53 +0000
> Sean Morrissey <sean.morrissey@intel.com> wrote:
> 
> > This patchset introduces the include-what-you-use script which removes
> > unused header includes. IWYU GitHub:
> > 
> > https://github.com/include-what-you-use/include-what-you-use
> > 
> > Along with the script there are some patches which make a start on
> > removing unneeded headers.
> > 
> > Sean Morrissey (5):
> >   devtools: script to remove unused headers includes
> >   lib/telemetry: remove unneeded header includes
> >   lib/ring: remove unneeded header includes
> >   lib/kvargs: remove unneeded header includes
> >   lib/eal: remove unneeded header includes
> > 
> >  devtools/process_iwyu.py               | 109 +++++++++++++++++++++++++
<snip>
> >  49 files changed, 110 insertions(+), 213 deletions(-)
> >  create mode 100755 devtools/process_iwyu.py
> > 
> 
> There is a risk of breaking builds on other platforms.
> How can you be sure the include files (especially auto generated list)
> are the same in Linux, FreeBSD and Windows as well as the special
> versions of libc (musl etc).

Yep, there is a risk of that but:

a) that's why we have automated build checks of various types in
test-meson-builds and why we have automated testing of patches in a number
of different labs reported out in patchwork.
b) if we say we can't risk it, we then can't gain much/any real benefit
from tools like iwyu - a tool that you have encouraged use of a number of
times :-).

Overall, I think this is a good thing for DPDK, and I encouraged Sean to
look into it, because it just helps with code quality in DPDK. However,
it's also true that there is risk in this, so I think the approach of only
doing a few libraries initially - so we can see how it works out - is
probably best, rather than trying to do the whole codebase in one go. I
think it's also true to say that the risk is greatest with EAL, so if it
can be cleaned up, cleaning the other libs and maybe then drivers should be
easier. Having the whole processing automated will also help with this, as -
if we want to minimize risk - the work on each driver can be done by the
driver maintainers themselves, who are best placed to know the
platform-specific issues for that code.

Regards,
/Bruce