diff mbox series

[v4,5/6] doc/guides: rebuild with meson whenever a file changes

Message ID 20200110215205.513185-6-bruce.richardson@intel.com (mailing list archive)
State Accepted, archived
Delegated to: David Marchand
Headers
Series meson build improvements |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Bruce Richardson Jan. 10, 2020, 9:52 p.m. UTC 
  Add proper support for calling sphinx whenever a file in the doc
directory changes. This is accomplished by using a wrapper script
for sphinx, which runs sphinx but also emits a gcc-format dependency
file listing all the doc files. This is used by ninja so that any
change to the doc files triggers a rebuild of the docs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
---
 MAINTAINERS                     |  1 +
 buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 4 files changed, 44 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

Comments

David Marchand Jan. 14, 2020, 3:59 p.m. UTC | #1 
On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Add proper support for calling sphinx whenever a file in the doc
> directory changes. This is accomplished by using a wrapper script
> for sphinx, which runs sphinx but also emits a gcc-format dependency
> file listing all the doc files. This is used by ninja so that any
> change to the doc files triggers a rebuild of the docs.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Aaron Conole <aconole@redhat.com>
> Acked-by: Luca Boccassi <bluca@debian.org>
> ---
>  MAINTAINERS                     |  1 +
>  buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
>  buildtools/meson.build          |  6 ++++--
>  doc/guides/meson.build          | 22 ++++++++--------------
>  4 files changed, 44 insertions(+), 16 deletions(-)
>  create mode 100755 buildtools/call-sphinx-build.py

Caught an issue on fc30:

[2094/2338] Generating html_guides with a custom command.
FAILED: doc/guides/html
/usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py
/usr/libexec/python2-sphinx/sphinx-build
/home/dmarchan/dpdk/doc/guides
/home/dmarchan/builds/build-x86-default/doc/guides
sphinx-build 1.8.4
Traceback (most recent call last):
  File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
    ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
IndexError: list index out of range

Double checked, the version went to stderr:

$ sphinx-build --version >/dev/null
sphinx-build 1.8.4
$ sphinx-build --version 2>/dev/null
Bruce Richardson Jan. 14, 2020, 4:17 p.m. UTC | #2 
On Tue, Jan 14, 2020 at 04:59:24PM +0100, David Marchand wrote:
> On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Add proper support for calling sphinx whenever a file in the doc
> > directory changes. This is accomplished by using a wrapper script
> > for sphinx, which runs sphinx but also emits a gcc-format dependency
> > file listing all the doc files. This is used by ninja so that any
> > change to the doc files triggers a rebuild of the docs.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Aaron Conole <aconole@redhat.com>
> > Acked-by: Luca Boccassi <bluca@debian.org>
> > ---
> >  MAINTAINERS                     |  1 +
> >  buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
> >  buildtools/meson.build          |  6 ++++--
> >  doc/guides/meson.build          | 22 ++++++++--------------
> >  4 files changed, 44 insertions(+), 16 deletions(-)
> >  create mode 100755 buildtools/call-sphinx-build.py
> 
> Caught an issue on fc30:
> 
> [2094/2338] Generating html_guides with a custom command.
> FAILED: doc/guides/html
> /usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py
> /usr/libexec/python2-sphinx/sphinx-build
> /home/dmarchan/dpdk/doc/guides
> /home/dmarchan/builds/build-x86-default/doc/guides
> sphinx-build 1.8.4
> Traceback (most recent call last):
>   File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
>     ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
> IndexError: list index out of range
> 
> Double checked, the version went to stderr:
> 
> $ sphinx-build --version >/dev/null
> sphinx-build 1.8.4
> $ sphinx-build --version 2>/dev/null
> 
That is strange. Didn't hit any issues on fedora 31 which I am using here.
I'll see if I can make it work with both stderr and stdout reporting.

/Bruce
Bruce Richardson Jan. 17, 2020, 11:15 a.m. UTC | #3 
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, January 14, 2020 3:59 PM
> To: Richardson, Bruce <bruce.richardson@intel.com>
> Cc: Mcnamara, John <john.mcnamara@intel.com>; Luca Boccassi
> <bluca@debian.org>; dev <dev@dpdk.org>; Aaron Conole <aconole@redhat.com>;
> Thomas Monjalon <thomas@monjalon.net>
> Subject: Re: [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file
> changes
> 
> On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Add proper support for calling sphinx whenever a file in the doc
> > directory changes. This is accomplished by using a wrapper script for
> > sphinx, which runs sphinx but also emits a gcc-format dependency file
> > listing all the doc files. This is used by ninja so that any change to
> > the doc files triggers a rebuild of the docs.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Aaron Conole <aconole@redhat.com>
> > Acked-by: Luca Boccassi <bluca@debian.org>
> > ---
> >  MAINTAINERS                     |  1 +
> >  buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
> >  buildtools/meson.build          |  6 ++++--
> >  doc/guides/meson.build          | 22 ++++++++--------------
> >  4 files changed, 44 insertions(+), 16 deletions(-)  create mode
> > 100755 buildtools/call-sphinx-build.py
> 
> Caught an issue on fc30:
> 
> [2094/2338] Generating html_guides with a custom command.
> FAILED: doc/guides/html
> /usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py
> /usr/libexec/python2-sphinx/sphinx-build
> /home/dmarchan/dpdk/doc/guides
> /home/dmarchan/builds/build-x86-default/doc/guides
> sphinx-build 1.8.4
> Traceback (most recent call last):
>   File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
>     ver = run([sphinx, '--version'],
> stdout=PIPE).stdout.decode().split()[-1]
> IndexError: list index out of range
> 
> Double checked, the version went to stderr:
> 
> $ sphinx-build --version >/dev/null
> sphinx-build 1.8.4
> $ sphinx-build --version 2>/dev/null
> 

The series http://patches.dpdk.org/project/dpdk/list/?series=8190 should fix
this issue and also adds support for the "werror" option for building the guides.

/Bruce
diff mbox series

Patch

diff --git a/MAINTAINERS b/MAINTAINERS
index 4395d8df1..aa32bfc59 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -128,6 +128,7 @@  F: meson.build
 F: lib/librte_eal/freebsd/BSDmakefile.meson
 F: meson_options.txt
 F: config/rte_config.h
+F: buildtools/call-sphinx-build.py
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/map_to_def.py
 F: buildtools/symlink-drivers-solibs.sh
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
new file mode 100755
index 000000000..b9a3994e1
--- /dev/null
+++ b/buildtools/call-sphinx-build.py
@@ -0,0 +1,31 @@ 
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+from os.path import join
+from subprocess import run, PIPE
+from distutils.version import StrictVersion
+
+(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
+
+# for sphinx version >= 1.7 add parallelism using "-j auto"
+ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
+sphinx_cmd = [sphinx]
+if StrictVersion(ver) >= StrictVersion('1.7'):
+    sphinx_cmd += ['-j', 'auto']
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    srcfiles.extend([join(root, f) for f in files])
+
+# run sphinx, putting the html output in a "html" directory
+process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')], check=True)
+print(str(process.args) + ' Done OK')
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(join(dst, '.html.d'), 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 6ef2c5721..cd1d05403 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -10,10 +10,12 @@  check_experimental_syms = find_program('check-experimental-syms.sh')
 # set up map-to-def script using python, either built-in or external
 python3 = import('python').find_installation(required: false)
 if python3.found()
-	map_to_def_cmd = [python3, files('map_to_def.py')]
+	py3 = [python3]
 else
-	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+	py3 = ['meson', 'runpython']
 endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # stable ABI always starts with "DPDK_"
 is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index 80c21d168..732e7ad3a 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -7,24 +7,18 @@  if not sphinx.found()
 	subdir_done()
 endif
 
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
-	input: meson.current_source_dir(),
-	output: 'guides',
-	command: [sphinx, '-b', 'html',
-		'-d', meson.current_build_dir() + '/.doctrees',
-		'@INPUT@', meson.current_build_dir() + '/guides'],
+	input: files('index.rst'),
+	output: 'html',
+	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+	depfile: '.html.d',
 	build_by_default: get_option('enable_docs'),
 	install: get_option('enable_docs'),
 	install_dir: htmldir)
 
+install_data(files('custom.css'),
+			install_dir: join_paths(htmldir,'_static', 'css'))
+
 doc_targets += html_guides
 doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
-	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
-	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')