diff mbox series

[v3] build: fix symlink of drivers for Windows

Message ID 20210426100733.22478-1-nick.connolly@mayadata.io (mailing list archive)
State Accepted, archived
Delegated to: Thomas Monjalon
Headers
Series [v3] build: fix symlink of drivers for Windows |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/Intel-compilation success Compilation OK
ci/intel-Testing success Testing PASS
ci/iol-abi-testing success Testing PASS
ci/iol-testing success Testing PASS
ci/github-robot success github build: passed
ci/iol-mellanox-Performance success Performance Testing PASS

Commit Message

Nick Connolly April 26, 2021, 10:07 a.m. UTC 
  The symlink-drivers-solibs.sh script was disabled as part of 'install'
for Windows because there is no support for shell scripts. However,
this means that driver related DLLs are not present in the installed
'libdir' directory. Add a python script to perform the install and use
it for Windows if the version of meson supports using an external
program with add_install_script (>= 0.55.0).

On Windows, symbolic links are somewhat problematic since the
SeCreateSymbolicLinkPrivilege is required to be able to create them.
In addition, different cross-compilation environments handle symbolic
links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
distinguish these scenarios, the python script will perform a file copy
for any Windows specific names.

On Windows, the shared library outputs have different names depending
upon which toolset has been used to build them. The script currently
handles Clang and GCC.

On Linux the functionality is unchanged, but could be replaced with the
python script once the required minimum version of meson is >= 0.55.0.

Fixes: 5c7d86948764 ("build: fix install on Windows")
Cc: stable@dpdk.org

Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
---
v3:
* rebase

v2:
* 0.55.0 is required to use external program with add_install_script

 buildtools/symlink-drivers-solibs.py | 49 ++++++++++++++++++++++++++++
 config/meson.build                   |  4 +++
 2 files changed, 53 insertions(+)
 create mode 100644 buildtools/symlink-drivers-solibs.py

Comments

Nick Connolly May 27, 2021, 5:37 p.m. UTC | #1 
Hi Bruce,

Would you have some time to take a look at this?  It's a replacement for the
symlink-drivers-solibs.sh script for windows builds.

Thanks,
Nick

On 26/04/2021 11:07, Nick Connolly wrote:
> The symlink-drivers-solibs.sh script was disabled as part of 'install'
> for Windows because there is no support for shell scripts. However,
> this means that driver related DLLs are not present in the installed
> 'libdir' directory. Add a python script to perform the install and use
> it for Windows if the version of meson supports using an external
> program with add_install_script (>= 0.55.0).
>
> On Windows, symbolic links are somewhat problematic since the
> SeCreateSymbolicLinkPrivilege is required to be able to create them.
> In addition, different cross-compilation environments handle symbolic
> links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
> distinguish these scenarios, the python script will perform a file copy
> for any Windows specific names.
>
> On Windows, the shared library outputs have different names depending
> upon which toolset has been used to build them. The script currently
> handles Clang and GCC.
>
> On Linux the functionality is unchanged, but could be replaced with the
> python script once the required minimum version of meson is >= 0.55.0.
>
> Fixes: 5c7d86948764 ("build: fix install on Windows")
> Cc: stable@dpdk.org
>
> Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
> Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
> Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
> ---
> v3:
> * rebase
>
> v2:
> * 0.55.0 is required to use external program with add_install_script
>
>   buildtools/symlink-drivers-solibs.py | 49 ++++++++++++++++++++++++++++
>   config/meson.build                   |  4 +++
>   2 files changed, 53 insertions(+)
>   create mode 100644 buildtools/symlink-drivers-solibs.py
>
> diff --git a/buildtools/symlink-drivers-solibs.py b/buildtools/symlink-drivers-solibs.py
> new file mode 100644
> index 000000000..5627ddd9d
> --- /dev/null
> +++ b/buildtools/symlink-drivers-solibs.py
> @@ -0,0 +1,49 @@
> +#!/usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2021 Intel Corporation
> +
> +import os
> +import sys
> +import glob
> +import shutil
> +
> +# post-install script for meson/ninja builds to symlink the PMDs stored in
> +# $libdir/dpdk/pmds-*/ to $libdir. This is needed as some PMDs depend on
> +# others, e.g. PCI device PMDs depending on the PCI bus driver.
> +
> +# parameters to script are paths relative to install prefix:
> +# 1. directory for installed regular libs e.g. lib64
> +# 2. subdirectory of libdir where the pmds are
> +# 3. directory for installed regular binaries e.g. bin
> +
> +os.chdir(os.environ['MESON_INSTALL_DESTDIR_PREFIX'])
> +
> +lib_dir = sys.argv[1]
> +pmd_subdir = sys.argv[2]
> +bin_dir = sys.argv[3]
> +pmd_dir = os.path.join(lib_dir, pmd_subdir)
> +
> +# copy Windows PMDs to avoid any issues with symlinks since the
> +# build could be a cross-compilation under WSL, Msys or Cygnus.
> +# the filenames are dependent upon the specific toolchain in use.
> +
> +def copy_pmd_files(pattern, to_dir):
> +	for file in glob.glob(os.path.join(pmd_dir, pattern)):
> +		to = os.path.join(to_dir, os.path.basename(file))
> +		shutil.copy2(file, to)
> +		print(to + ' -> ' + file)
> +
> +copy_pmd_files('*rte_*.dll', bin_dir)
> +copy_pmd_files('*rte_*.pdb', bin_dir)
> +copy_pmd_files('*rte_*.lib', lib_dir)
> +copy_pmd_files('*rte_*.dll.a', lib_dir)
> +
> +# symlink shared objects
> +
> +os.chdir(lib_dir)
> +for file in glob.glob(os.path.join(pmd_subdir, 'librte_*.so*')):
> +	to = os.path.basename(file)
> +	if os.path.exists(to):
> +		os.remove(to)
> +	os.symlink(file, to)
> +	print(to + ' -> ' + file)
> diff --git a/config/meson.build b/config/meson.build
> index 017bb2efb..9e04c4c76 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -59,6 +59,10 @@ eal_pmd_path = join_paths(get_option('prefix'), driver_install_path)
>   if not is_windows
>       meson.add_install_script('../buildtools/symlink-drivers-solibs.sh',
>               get_option('libdir'), pmd_subdir_opt)
> +elif meson.version().version_compare('>=0.55.0')
> +    # 0.55.0 is required to use external program with add_install_script
> +    meson.add_install_script(py3, '../buildtools/symlink-drivers-solibs.py',
> +            get_option('libdir'), pmd_subdir_opt, get_option('bindir'))
>   endif
>   
>   # init disable/enable driver lists that will be populated in different places
Bruce Richardson May 28, 2021, 8:19 a.m. UTC | #2 
On Mon, Apr 26, 2021 at 11:07:32AM +0100, Nick Connolly wrote:
> The symlink-drivers-solibs.sh script was disabled as part of 'install'
> for Windows because there is no support for shell scripts. However,
> this means that driver related DLLs are not present in the installed
> 'libdir' directory. Add a python script to perform the install and use
> it for Windows if the version of meson supports using an external
> program with add_install_script (>= 0.55.0).
> 
> On Windows, symbolic links are somewhat problematic since the
> SeCreateSymbolicLinkPrivilege is required to be able to create them.
> In addition, different cross-compilation environments handle symbolic
> links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
> distinguish these scenarios, the python script will perform a file copy
> for any Windows specific names.
> 
> On Windows, the shared library outputs have different names depending
> upon which toolset has been used to build them. The script currently
> handles Clang and GCC.
> 
> On Linux the functionality is unchanged, but could be replaced with the
> python script once the required minimum version of meson is >= 0.55.0.
> 
> Fixes: 5c7d86948764 ("build: fix install on Windows")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
> Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
> Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
> ---
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
Bruce Richardson May 28, 2021, 8:19 a.m. UTC | #3 
On Thu, May 27, 2021 at 06:37:57PM +0100, Nick Connolly wrote:
> Hi Bruce,
> 
> Would you have some time to take a look at this?  It's a replacement for the
> symlink-drivers-solibs.sh script for windows builds.
> 
> Thanks,
> Nick
> 
I've reviewed it now and it looks good. Couldn't even find anything to
nit-pick in it! :-)
Nick Connolly May 28, 2021, 10:50 a.m. UTC | #4 
> I've reviewed it now and it looks good. Couldn't even find anything to
> nit-pick in it! :-)

Thanks Bruce - appreciated  :-)
Nick
Thomas Monjalon July 9, 2021, 3:05 p.m. UTC | #5 
28/05/2021 10:19, Bruce Richardson:
> On Mon, Apr 26, 2021 at 11:07:32AM +0100, Nick Connolly wrote:
> > The symlink-drivers-solibs.sh script was disabled as part of 'install'
> > for Windows because there is no support for shell scripts. However,
> > this means that driver related DLLs are not present in the installed
> > 'libdir' directory. Add a python script to perform the install and use
> > it for Windows if the version of meson supports using an external
> > program with add_install_script (>= 0.55.0).
> > 
> > On Windows, symbolic links are somewhat problematic since the
> > SeCreateSymbolicLinkPrivilege is required to be able to create them.
> > In addition, different cross-compilation environments handle symbolic
> > links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
> > distinguish these scenarios, the python script will perform a file copy
> > for any Windows specific names.
> > 
> > On Windows, the shared library outputs have different names depending
> > upon which toolset has been used to build them. The script currently
> > handles Clang and GCC.
> > 
> > On Linux the functionality is unchanged, but could be replaced with the
> > python script once the required minimum version of meson is >= 0.55.0.
> > 
> > Fixes: 5c7d86948764 ("build: fix install on Windows")
> > Cc: stable@dpdk.org
> > 
> > Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
> > Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
> > Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
> > ---
> Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>

Added the new file in MAINTAINERS.

Applied, thanks.
diff mbox series

Patch

diff --git a/buildtools/symlink-drivers-solibs.py b/buildtools/symlink-drivers-solibs.py
new file mode 100644
index 000000000..5627ddd9d
--- /dev/null
+++ b/buildtools/symlink-drivers-solibs.py
@@ -0,0 +1,49 @@ 
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 Intel Corporation
+
+import os
+import sys
+import glob
+import shutil
+
+# post-install script for meson/ninja builds to symlink the PMDs stored in
+# $libdir/dpdk/pmds-*/ to $libdir. This is needed as some PMDs depend on
+# others, e.g. PCI device PMDs depending on the PCI bus driver.
+
+# parameters to script are paths relative to install prefix:
+# 1. directory for installed regular libs e.g. lib64
+# 2. subdirectory of libdir where the pmds are
+# 3. directory for installed regular binaries e.g. bin
+
+os.chdir(os.environ['MESON_INSTALL_DESTDIR_PREFIX'])
+
+lib_dir = sys.argv[1]
+pmd_subdir = sys.argv[2]
+bin_dir = sys.argv[3]
+pmd_dir = os.path.join(lib_dir, pmd_subdir)
+
+# copy Windows PMDs to avoid any issues with symlinks since the
+# build could be a cross-compilation under WSL, Msys or Cygnus.
+# the filenames are dependent upon the specific toolchain in use.
+
+def copy_pmd_files(pattern, to_dir):
+	for file in glob.glob(os.path.join(pmd_dir, pattern)):
+		to = os.path.join(to_dir, os.path.basename(file))
+		shutil.copy2(file, to)
+		print(to + ' -> ' + file)
+
+copy_pmd_files('*rte_*.dll', bin_dir)
+copy_pmd_files('*rte_*.pdb', bin_dir)
+copy_pmd_files('*rte_*.lib', lib_dir)
+copy_pmd_files('*rte_*.dll.a', lib_dir)
+
+# symlink shared objects
+
+os.chdir(lib_dir)
+for file in glob.glob(os.path.join(pmd_subdir, 'librte_*.so*')):
+	to = os.path.basename(file)
+	if os.path.exists(to):
+		os.remove(to)
+	os.symlink(file, to)
+	print(to + ' -> ' + file)
diff --git a/config/meson.build b/config/meson.build
index 017bb2efb..9e04c4c76 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -59,6 +59,10 @@  eal_pmd_path = join_paths(get_option('prefix'), driver_install_path)
 if not is_windows
     meson.add_install_script('../buildtools/symlink-drivers-solibs.sh',
             get_option('libdir'), pmd_subdir_opt)
+elif meson.version().version_compare('>=0.55.0')
+    # 0.55.0 is required to use external program with add_install_script
+    meson.add_install_script(py3, '../buildtools/symlink-drivers-solibs.py',
+            get_option('libdir'), pmd_subdir_opt, get_option('bindir'))
 endif
 
 # init disable/enable driver lists that will be populated in different places