[v2,5/6] doc/guides: rebuild with meson whenever a file changes

  Add proper support for calling sphinx whenever a file in the doc
directory changes. This is accomplished by using a wrapper script
for sphinx, which runs sphinx but also emits a gcc-format dependency
file listing all the doc files. This is used by ninja so that any
change to the doc files triggers a rebuild of the docs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 41 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
  

Bruce Richardson <bruce.richardson@intel.com> writes:

> Add proper support for calling sphinx whenever a file in the doc
> directory changes. This is accomplished by using a wrapper script
> for sphinx, which runs sphinx but also emits a gcc-format dependency
> file listing all the doc files. This is used by ninja so that any
> change to the doc files triggers a rebuild of the docs.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
>  buildtools/meson.build          |  6 ++++--
>  doc/guides/meson.build          | 22 ++++++++--------------
>  3 files changed, 41 insertions(+), 16 deletions(-)
>  create mode 100755 buildtools/call-sphinx-build.py
>

Acked-by: Aaron Conole <aconole@redhat.com>

Some nits follow - it would be good to clean them up before applying but
they are fairly inconsequential.

> diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
> new file mode 100755
> index 000000000..027317b9b
> --- /dev/null
> +++ b/buildtools/call-sphinx-build.py
> @@ -0,0 +1,29 @@
> +#! /usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2019 Intel Corporation
> +#
> +
> +import sys
> +import os
> +import os.path
> +import subprocess
> +
> +sphinx = sys.argv[1]
> +src = sys.argv[2]
> +dst = sys.argv[3]
> +depfile = os.path.join(dst,'.html.d')
                              ^ whitespace here
> +
> +# find all the files sphinx will process so we can write them as dependencies
> +srcfiles = []
> +for root, dirs, files in os.walk(src):
> +    for f in files:
> +        srcfiles.append(os.path.join(root, f))
> +
> +# run sphinx, putting the html output in a "html" directory
> +subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src,
> +                os.path.join(dst, 'html')], check = True)
                                                    ^ no whitespace
                                                      around =
> +
> +# create a gcc format .d file giving all the dependencies of this doc build
> +with open(depfile, 'w') as d:
> +    d.write('html: ' + ' '.join(srcfiles) + '\n')
> +subprocess.run(['cp', '-f', depfile, '/tmp'])
> diff --git a/buildtools/meson.build b/buildtools/meson.build
> index 6ef2c5721..cd1d05403 100644
> --- a/buildtools/meson.build
> +++ b/buildtools/meson.build
> @@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
>  # set up map-to-def script using python, either built-in or external
>  python3 = import('python').find_installation(required: false)
>  if python3.found()
> -	map_to_def_cmd = [python3, files('map_to_def.py')]
> +	py3 = [python3]
>  else
> -	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
> +	py3 = ['meson', 'runpython']
>  endif
> +map_to_def_cmd = py3 + files('map_to_def.py')
> +sphinx_wrapper = py3 + files('call-sphinx-build.py')
>  
>  # stable ABI always starts with "DPDK_"
>  is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
> diff --git a/doc/guides/meson.build b/doc/guides/meson.build
> index 80c21d168..732e7ad3a 100644
> --- a/doc/guides/meson.build
> +++ b/doc/guides/meson.build
> @@ -7,24 +7,18 @@ if not sphinx.found()
>  	subdir_done()
>  endif
>  
> -htmldir = join_paths('share', 'doc', 'dpdk')
> +htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
>  html_guides = custom_target('html_guides',
> -	input: meson.current_source_dir(),
> -	output: 'guides',
> -	command: [sphinx, '-b', 'html',
> -		'-d', meson.current_build_dir() + '/.doctrees',
> -		'@INPUT@', meson.current_build_dir() + '/guides'],
> +	input: files('index.rst'),
> +	output: 'html',
> +	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
> +	depfile: '.html.d',
>  	build_by_default: get_option('enable_docs'),
>  	install: get_option('enable_docs'),
>  	install_dir: htmldir)
>  
> +install_data(files('custom.css'),
> +			install_dir: join_paths(htmldir,'_static', 'css'))
> +
>  doc_targets += html_guides
>  doc_target_names += 'HTML_Guides'
> -
> -# sphinx leaves a .buildinfo in the target directory, which we don't
> -# want to install. Note that sh -c has to be used, otherwise the
> -# env var does not get expanded if calling rm/install directly.
> -meson.add_install_script('sh', '-c',
> -	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
> -meson.add_install_script('sh', '-c',
> -	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
  

On Thu, Jan 09, 2020 at 09:27:42AM -0500, Aaron Conole wrote:
> Bruce Richardson <bruce.richardson@intel.com> writes:
> 
> > Add proper support for calling sphinx whenever a file in the doc
> > directory changes. This is accomplished by using a wrapper script
> > for sphinx, which runs sphinx but also emits a gcc-format dependency
> > file listing all the doc files. This is used by ninja so that any
> > change to the doc files triggers a rebuild of the docs.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> >  buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
> >  buildtools/meson.build          |  6 ++++--
> >  doc/guides/meson.build          | 22 ++++++++--------------
> >  3 files changed, 41 insertions(+), 16 deletions(-)
> >  create mode 100755 buildtools/call-sphinx-build.py
> >
> 
> Acked-by: Aaron Conole <aconole@redhat.com>
> 
> Some nits follow - it would be good to clean them up before applying but
> they are fairly inconsequential.
> 
> > diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
> > new file mode 100755
> > index 000000000..027317b9b
> > --- /dev/null
> > +++ b/buildtools/call-sphinx-build.py
> > @@ -0,0 +1,29 @@
> > +#! /usr/bin/env python3
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2019 Intel Corporation
> > +#
> > +
> > +import sys
> > +import os
> > +import os.path
> > +import subprocess
> > +
> > +sphinx = sys.argv[1]
> > +src = sys.argv[2]
> > +dst = sys.argv[3]
> > +depfile = os.path.join(dst,'.html.d')
>                               ^ whitespace here
> > +
> > +# find all the files sphinx will process so we can write them as dependencies
> > +srcfiles = []
> > +for root, dirs, files in os.walk(src):
> > +    for f in files:
> > +        srcfiles.append(os.path.join(root, f))
> > +
> > +# run sphinx, putting the html output in a "html" directory
> > +subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src,
> > +                os.path.join(dst, 'html')], check = True)
>                                                     ^ no whitespace
>                                                       around =
> > +
> > +# create a gcc format .d file giving all the dependencies of this doc build
> > +with open(depfile, 'w') as d:
> > +    d.write('html: ' + ' '.join(srcfiles) + '\n')
> > +subprocess.run(['cp', '-f', depfile, '/tmp'])

Ok, will fix. I also see I left in an unnecessary subprocess.run at the end
to copy the file to /tmp. This was for debugging only since ninja deletes
the .d files after processing them.
[https://ninja-build.org/manual.html#_deps, see behaviour for deps=gcc]

/Bruce