diff mbox series

[17/20] app/chkincs: add chkincs app to verify headers

Message ID 20210114110606.21142-18-bruce.richardson@intel.com (mailing list archive)
State Superseded, archived
Delegated to: David Marchand
Headers show
Series ensure headers have correct includes | expand

Checks

Context Check Description
ci/checkpatch warning coding style issues

Commit Message

Bruce Richardson Jan. 14, 2021, 11:06 a.m. UTC
To verify that all DPDK headers are ok for inclusion directly in a C
file, and are not missing any other pre-requisite headers, we can
auto-generate for each header an empty C file that includes that header.
Compiling these files will throw errors if any header has unmet
dependencies.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/chkincs/gen_c_file_for_header.py | 49 ++++++++++++++++++++++++++++
 app/chkincs/main.c                   |  4 +++
 app/chkincs/meson.build              | 28 ++++++++++++++++
 app/meson.build                      |  1 +
 lib/meson.build                      |  1 +
 meson.build                          |  1 +
 meson_options.txt                    |  2 ++
 7 files changed, 86 insertions(+)
 create mode 100755 app/chkincs/gen_c_file_for_header.py
 create mode 100644 app/chkincs/main.c
 create mode 100644 app/chkincs/meson.build

--
2.27.0

Comments

David Marchand Jan. 14, 2021, 12:16 p.m. UTC | #1
On Thu, Jan 14, 2021 at 12:09 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> To verify that all DPDK headers are ok for inclusion directly in a C
> file, and are not missing any other pre-requisite headers, we can
> auto-generate for each header an empty C file that includes that header.
> Compiling these files will throw errors if any header has unmet
> dependencies.

Some drivers expose APIs to applications, their headers would need checks too.

>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  app/chkincs/gen_c_file_for_header.py | 49 ++++++++++++++++++++++++++++
>  app/chkincs/main.c                   |  4 +++
>  app/chkincs/meson.build              | 28 ++++++++++++++++
>  app/meson.build                      |  1 +
>  lib/meson.build                      |  1 +
>  meson.build                          |  1 +
>  meson_options.txt                    |  2 ++
>  7 files changed, 86 insertions(+)
>  create mode 100755 app/chkincs/gen_c_file_for_header.py
>  create mode 100644 app/chkincs/main.c
>  create mode 100644 app/chkincs/meson.build
>
> diff --git a/app/chkincs/gen_c_file_for_header.py b/app/chkincs/gen_c_file_for_header.py
> new file mode 100755
> index 0000000000..f92f2b412c
> --- /dev/null
> +++ b/app/chkincs/gen_c_file_for_header.py
> @@ -0,0 +1,49 @@
> +#! /usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2020 Intel Corporation

2021*


> +
> +from sys import argv
> +from os.path import abspath
> +
> +empty_contents = 'static const char *empty __attribute__((unused)) = "empty";'
> +# files which are not used directly, but included via others
> +exceptions = [
> +    'rte_cmp_arm64.h',
> +    'rte_cmp_x86.h',
> +    'rte_crc_arm64.h',
> +    'rte_eal_interrupts.h',
> +    'rte_eth_ctrl.h',
> +    'rte_ethdev_core.h',
> +    'rte_ipsec_group.h',
> +    'rte_lpm_altivec.h',
> +    'rte_lpm_neon.h',
> +    'rte_lpm_sse.h',
> +    'rte_lpm_x86.h',
> +    'rte_lru_arm64.h',
> +    'rte_lru_x86.h',
> +    'rte_regexdev_core.h',
> +    'rte_ring_core.h',
> +    'rte_ring_generic.h',
> +    'rte_ring_hts_c11_mem.h',
> +    'rte_ring_hts.h',
> +    'rte_ring_peek_c11_mem.h',
> +    'rte_ring_peek.h',
> +    'rte_ring_peek_zc.h',
> +    'rte_ring_rts_c11_mem.h',
> +    'rte_ring_rts.h',
> +    'rte_stack_lf_c11.h',
> +    'rte_stack_lf_generic.h',
> +    'rte_stack_lf.h',
> +    'rte_stack_std.h',
> +    'rte_table_hash_func_arm64.h',
> +    ]

Can we instead flag those headers from the libraries themselves?
In addition of the headers current variable, something like a
internal_headers or private_headers variable?


> +
> +(h_file, c_file) = argv[1:]
> +
> +contents = '#include "' + abspath(h_file) + '"'
> +for ex in exceptions:
> +    if h_file.endswith(ex):
> +        contents = empty_contents
> +
> +with open(c_file, 'w') as cf:
> +    cf.write(contents)
Bruce Richardson Jan. 14, 2021, 12:28 p.m. UTC | #2
On Thu, Jan 14, 2021 at 01:16:58PM +0100, David Marchand wrote:
> On Thu, Jan 14, 2021 at 12:09 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > To verify that all DPDK headers are ok for inclusion directly in a C
> > file, and are not missing any other pre-requisite headers, we can
> > auto-generate for each header an empty C file that includes that header.
> > Compiling these files will throw errors if any header has unmet
> > dependencies.
> 
> Some drivers expose APIs to applications, their headers would need checks too.
> 

Yes, that is something that should be checked too, but I've left it as
"further work" for now, and I'm not planning it in this set, as I view it
as less important and this set is big-enough as it is for now. :-)

> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> >  app/chkincs/gen_c_file_for_header.py | 49 ++++++++++++++++++++++++++++
> >  app/chkincs/main.c                   |  4 +++
> >  app/chkincs/meson.build              | 28 ++++++++++++++++
> >  app/meson.build                      |  1 +
> >  lib/meson.build                      |  1 +
> >  meson.build                          |  1 +
> >  meson_options.txt                    |  2 ++
> >  7 files changed, 86 insertions(+)
> >  create mode 100755 app/chkincs/gen_c_file_for_header.py
> >  create mode 100644 app/chkincs/main.c
> >  create mode 100644 app/chkincs/meson.build
> >
> > diff --git a/app/chkincs/gen_c_file_for_header.py b/app/chkincs/gen_c_file_for_header.py
> > new file mode 100755
> > index 0000000000..f92f2b412c
> > --- /dev/null
> > +++ b/app/chkincs/gen_c_file_for_header.py
> > @@ -0,0 +1,49 @@
> > +#! /usr/bin/env python3
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2020 Intel Corporation
> 
> 2021*
> 

Code actually was written in 2020 hence the date there.

> 
> > +
> > +from sys import argv
> > +from os.path import abspath
> > +
> > +empty_contents = 'static const char *empty __attribute__((unused)) = "empty";'
> > +# files which are not used directly, but included via others
> > +exceptions = [
> > +    'rte_cmp_arm64.h',
> > +    'rte_cmp_x86.h',
> > +    'rte_crc_arm64.h',
> > +    'rte_eal_interrupts.h',
> > +    'rte_eth_ctrl.h',
> > +    'rte_ethdev_core.h',
> > +    'rte_ipsec_group.h',
> > +    'rte_lpm_altivec.h',
> > +    'rte_lpm_neon.h',
> > +    'rte_lpm_sse.h',
> > +    'rte_lpm_x86.h',
> > +    'rte_lru_arm64.h',
> > +    'rte_lru_x86.h',
> > +    'rte_regexdev_core.h',
> > +    'rte_ring_core.h',
> > +    'rte_ring_generic.h',
> > +    'rte_ring_hts_c11_mem.h',
> > +    'rte_ring_hts.h',
> > +    'rte_ring_peek_c11_mem.h',
> > +    'rte_ring_peek.h',
> > +    'rte_ring_peek_zc.h',
> > +    'rte_ring_rts_c11_mem.h',
> > +    'rte_ring_rts.h',
> > +    'rte_stack_lf_c11.h',
> > +    'rte_stack_lf_generic.h',
> > +    'rte_stack_lf.h',
> > +    'rte_stack_std.h',
> > +    'rte_table_hash_func_arm64.h',
> > +    ]
> 
> Can we instead flag those headers from the libraries themselves?
> In addition of the headers current variable, something like a
> internal_headers or private_headers variable?
> 

Yes, we could do that, though we need a suitable name, since they aren't
internal only or private. :-) Maybe just have it literally called
"skip_chkincs_headers"? Open to more ideas...

Thanks for the review!

/Bruce
David Marchand Jan. 14, 2021, 12:49 p.m. UTC | #3
On Thu, Jan 14, 2021 at 1:28 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
> > Can we instead flag those headers from the libraries themselves?
> > In addition of the headers current variable, something like a
> > internal_headers or private_headers variable?
> >
>
> Yes, we could do that, though we need a suitable name, since they aren't
> internal only or private. :-) Maybe just have it literally called

Yes.

> "skip_chkincs_headers"? Open to more ideas...
>

People who get reports from this odd tool "chkincs" in the CI on their
new shiny header, might find some info more easily if they grep for
chkincs.
+1.
David Marchand Jan. 14, 2021, 4:04 p.m. UTC | #4
On Thu, Jan 14, 2021 at 12:09 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> To verify that all DPDK headers are ok for inclusion directly in a C
> file, and are not missing any other pre-requisite headers, we can
> auto-generate for each header an empty C file that includes that header.
> Compiling these files will throw errors if any header has unmet
> dependencies.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  app/chkincs/gen_c_file_for_header.py | 49 ++++++++++++++++++++++++++++
>  app/chkincs/main.c                   |  4 +++
>  app/chkincs/meson.build              | 28 ++++++++++++++++

Those are new files, so the patch is missing a MAINTAINERS update.
Bruce Richardson Jan. 14, 2021, 4:18 p.m. UTC | #5
On Thu, Jan 14, 2021 at 05:04:11PM +0100, David Marchand wrote:
> On Thu, Jan 14, 2021 at 12:09 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > To verify that all DPDK headers are ok for inclusion directly in a C
> > file, and are not missing any other pre-requisite headers, we can
> > auto-generate for each header an empty C file that includes that header.
> > Compiling these files will throw errors if any header has unmet
> > dependencies.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> >  app/chkincs/gen_c_file_for_header.py | 49 ++++++++++++++++++++++++++++
> >  app/chkincs/main.c                   |  4 +++
> >  app/chkincs/meson.build              | 28 ++++++++++++++++
> 
> Those are new files, so the patch is missing a MAINTAINERS update.
> 
Yes, forgot that. Will add in V2.
diff mbox series

Patch

diff --git a/app/chkincs/gen_c_file_for_header.py b/app/chkincs/gen_c_file_for_header.py
new file mode 100755
index 0000000000..f92f2b412c
--- /dev/null
+++ b/app/chkincs/gen_c_file_for_header.py
@@ -0,0 +1,49 @@ 
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2020 Intel Corporation
+
+from sys import argv
+from os.path import abspath
+
+empty_contents = 'static const char *empty __attribute__((unused)) = "empty";'
+# files which are not used directly, but included via others
+exceptions = [
+    'rte_cmp_arm64.h',
+    'rte_cmp_x86.h',
+    'rte_crc_arm64.h',
+    'rte_eal_interrupts.h',
+    'rte_eth_ctrl.h',
+    'rte_ethdev_core.h',
+    'rte_ipsec_group.h',
+    'rte_lpm_altivec.h',
+    'rte_lpm_neon.h',
+    'rte_lpm_sse.h',
+    'rte_lpm_x86.h',
+    'rte_lru_arm64.h',
+    'rte_lru_x86.h',
+    'rte_regexdev_core.h',
+    'rte_ring_core.h',
+    'rte_ring_generic.h',
+    'rte_ring_hts_c11_mem.h',
+    'rte_ring_hts.h',
+    'rte_ring_peek_c11_mem.h',
+    'rte_ring_peek.h',
+    'rte_ring_peek_zc.h',
+    'rte_ring_rts_c11_mem.h',
+    'rte_ring_rts.h',
+    'rte_stack_lf_c11.h',
+    'rte_stack_lf_generic.h',
+    'rte_stack_lf.h',
+    'rte_stack_std.h',
+    'rte_table_hash_func_arm64.h',
+    ]
+
+(h_file, c_file) = argv[1:]
+
+contents = '#include "' + abspath(h_file) + '"'
+for ex in exceptions:
+    if h_file.endswith(ex):
+        contents = empty_contents
+
+with open(c_file, 'w') as cf:
+    cf.write(contents)
diff --git a/app/chkincs/main.c b/app/chkincs/main.c
new file mode 100644
index 0000000000..ecdf641954
--- /dev/null
+++ b/app/chkincs/main.c
@@ -0,0 +1,4 @@ 
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2020 Intel Corporation
+ */
+int main(void) { return 0; }
diff --git a/app/chkincs/meson.build b/app/chkincs/meson.build
new file mode 100644
index 0000000000..c16ca1f4fe
--- /dev/null
+++ b/app/chkincs/meson.build
@@ -0,0 +1,28 @@ 
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2020 Intel Corporation
+
+if not get_option('test_includes')
+	build = false
+	subdir_done()
+endif
+
+if is_windows
+	# for windows, the shebang line in the script won't work.
+	error('option "test_includes" is not supported on windows')
+endif
+
+gen_c_file_for_header = find_program('gen_c_file_for_header.py')
+gen_c_files = generator(gen_c_file_for_header,
+	output: '@BASENAME@.c',
+	arguments: ['@INPUT@', '@OUTPUT@'])
+
+cflags += '-Wno-unused-function' # needed if we include generic headers
+
+# some ethdev headers depend on bus headers
+includes += include_directories('../../drivers/bus/pci',
+	'../../drivers/bus/vdev')
+
+sources += files('main.c')
+sources += gen_c_files.process(dpdk_headers)
+
+deps = enabled_libs
diff --git a/app/meson.build b/app/meson.build
index b6f2a6f56e..8832ba8b17 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -6,6 +6,7 @@  if is_windows
 endif

 apps = [
+	'chkincs',
 	'pdump',
 	'proc-info',
 	'test-acl',
diff --git a/lib/meson.build b/lib/meson.build
index ed00f89146..7698e6749c 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -103,6 +103,7 @@  foreach l:libraries
 		dpdk_conf.set('RTE_LIBRTE_' + name.to_upper(), 1) #old macro
 		dpdk_conf.set('RTE_LIB_' + name.to_upper(), 1) # new macro
 		install_headers(headers)
+		dpdk_headers += headers

 		libname = 'rte_' + name
 		includes += include_directories(dir_name)
diff --git a/meson.build b/meson.build
index 45d974cd2c..5c612fe93a 100644
--- a/meson.build
+++ b/meson.build
@@ -16,6 +16,7 @@  cc = meson.get_compiler('c')
 dpdk_conf = configuration_data()
 dpdk_libraries = []
 dpdk_static_libraries = []
+dpdk_headers = []
 dpdk_driver_classes = []
 dpdk_drivers = []
 dpdk_extra_ldflags = []
diff --git a/meson_options.txt b/meson_options.txt
index e384e6dbb2..9214219444 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -32,5 +32,7 @@  option('enable_trace_fp', type: 'boolean', value: false,
 	description: 'enable fast path trace points.')
 option('tests', type: 'boolean', value: true,
 	description: 'build unit tests')
+option('test_includes', type: 'boolean', value: false,
+	description: 'build "chkincs" to verify each header file can compile alone')
 option('use_hpet', type: 'boolean', value: false,
 	description: 'use HPET timer in EAL')