[dpdk-dev,03/10] mk: install a standard cutomizable tree
Commit Message
The rule "install" follows these conventions:
https://www.gnu.org/prep/standards/html_node/Directory-Variables.html
https://www.gnu.org/prep/standards/html_node/DESTDIR.html
The variable sdkdir has been added to the more standards ones,
to configure the directory used with RTE_SDK when using the DPDK makefiles
to build an application.
The old installed tree was static and always had .config, includes and
libs in a RTE_TARGET subdirectory. There is no such directory anymore in
an installed SDK. So the top directory is checked.
But RTE_TARGET can still be used, especially to build an app with a
compiled but not installed SDK.
That's why both cases are looked for RTE_SDK_BIN.
The default prefix /usr/local is empty in the T= case which is
used only for a local install.
It is still possible to build DPDK with the "install T=" rule without
specifying any DESTDIR. In such case there is no install, as before.
The old usage of an installed SDK is:
make -C examples/helloworld RTE_SDK=$(readlink -m $DESTDIR) \
RTE_TARGET=x86_64-native-linuxapp-gcc
RTE_TARGET can be specified but is useless now with an installed SDK.
The RTE_SDK directory must now point to a different path depending of
the installation.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
doc/build-sdk-quick.txt | 1 +
mk/internal/rte.extvars.mk | 8 +++++++
mk/rte.sdkinstall.mk | 59 ++++++++++++++++++++++++++++++++++++----------
mk/rte.vars.mk | 7 ------
4 files changed, 55 insertions(+), 20 deletions(-)
Comments
On 12/02/2015 05:57 AM, Thomas Monjalon wrote:
> The rule "install" follows these conventions:
> https://www.gnu.org/prep/standards/html_node/Directory-Variables.html
> https://www.gnu.org/prep/standards/html_node/DESTDIR.html
>
> The variable sdkdir has been added to the more standards ones,
> to configure the directory used with RTE_SDK when using the DPDK makefiles
> to build an application.
>
> The old installed tree was static and always had .config, includes and
> libs in a RTE_TARGET subdirectory. There is no such directory anymore in
> an installed SDK. So the top directory is checked.
> But RTE_TARGET can still be used, especially to build an app with a
> compiled but not installed SDK.
> That's why both cases are looked for RTE_SDK_BIN.
>
> The default prefix /usr/local is empty in the T= case which is
> used only for a local install.
> It is still possible to build DPDK with the "install T=" rule without
> specifying any DESTDIR. In such case there is no install, as before.
>
> The old usage of an installed SDK is:
> make -C examples/helloworld RTE_SDK=$(readlink -m $DESTDIR) \
> RTE_TARGET=x86_64-native-linuxapp-gcc
> RTE_TARGET can be specified but is useless now with an installed SDK.
> The RTE_SDK directory must now point to a different path depending of
> the installation.
>
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> ---
[...]
> @@ -32,10 +33,30 @@
> # Build directory is given with O=
> O ?= .
>
> +prefix ?= /usr/local
> +exec_prefix ?= $(prefix)
> +bindir ?= $(exec_prefix)/bin
> +libdir ?= $(exec_prefix)/lib
> +includedir ?= $(prefix)/include/dpdk
> +datarootdir ?= $(prefix)/share
> +datadir ?= $(datarootdir)/dpdk
> +sdkdir ?= $(datadir)
> +
> +# The install directories may be staged in DESTDIR
[...]
> + @echo ================== Installing $(DESTDIR)$(prefix)/
> + $(Q)$(call rte_mkdir, $(DESTDIR)$(libdir))
> + $(Q)cp -a $(BUILD_DIR)/lib/* $(DESTDIR)$(libdir)
> + $(Q)$(call rte_mkdir, $(DESTDIR)$(bindir))
> + $(Q)tar -cf - -C $(BUILD_DIR) app --exclude 'app/*.map' \
> + --exclude 'app/cmdline*' --exclude app/test \
> + --exclude app/testacl --exclude app/testpipeline | \
> + tar -xf - -C $(DESTDIR)$(bindir) --strip-components=1 \
> + --keep-newer-files --warning=no-ignore-newer
> + $(Q)$(call rte_mkdir, $(DESTDIR)$(datadir))
> + $(Q)cp -a $(RTE_SDK)/tools $(DESTDIR)$(datadir)
> + $(Q)$(call rte_mkdir, $(DESTDIR)$(includedir))
> + $(Q)tar -chf - -C $(BUILD_DIR) include | \
> + tar -xf - -C $(DESTDIR)$(includedir) --strip-components=1 \
> + --keep-newer-files --warning=no-ignore-newer
> + $(Q)$(call rte_mkdir, $(DESTDIR)$(sdkdir))
> + $(Q)cp -a $(BUILD_DIR)/.config $(DESTDIR)$(sdkdir)
> + $(Q)cp -a $(RTE_SDK)/{mk,scripts} $(DESTDIR)$(sdkdir)
> + $(Q)$(call rte_symlink, $(DESTDIR)$(includedir), $(DESTDIR)$(sdkdir)/include)
> + $(Q)$(call rte_symlink, $(DESTDIR)$(libdir), $(DESTDIR)$(sdkdir)/lib)
> + @echo Installation in $(DESTDIR)$(prefix)/ complete
> +endif
$(prefix)/share is supposed to be shareable across different
architectures. Most of the content here is, but at least the lib symlink
and .config file are not.
One option is to install .config and the symlinks within $(sdkdir)/$(T)
directories, then it can be shared across architectures because each
lives in their own directory. Another possibility is moving the whole
sdk directory into a subdir in $(libdir), but that misses the
opportunity to share across architectures (whether anybody actually
cares is a whole other question :)
$(sdkdir)/lib -> $(libdir) symlink seems reasonable when installing to
an empty staging root, but on a real-world installation it'd point to
/usr/lib(something) which has hundreds or thousands of other unrelated
libraries. My memory is hazy on details but I think this caused an
actual problem with something because I ended up $(sdkdir)/lib an actual
directory populated with symlinks to the individual DPDK libraries.
- Panu -
2015-12-02 12:27, Panu Matilainen:
> On 12/02/2015 05:57 AM, Thomas Monjalon wrote:
> > The old installed tree was static and always had .config, includes and
> > libs in a RTE_TARGET subdirectory. There is no such directory anymore in
> > an installed SDK. So the top directory is checked.
> > But RTE_TARGET can still be used, especially to build an app with a
> > compiled but not installed SDK.
> > That's why both cases are looked for RTE_SDK_BIN.
[...]
> > The old usage of an installed SDK is:
> > make -C examples/helloworld RTE_SDK=$(readlink -m $DESTDIR) \
> > RTE_TARGET=x86_64-native-linuxapp-gcc
> > RTE_TARGET can be specified but is useless now with an installed SDK.
> > The RTE_SDK directory must now point to a different path depending of
> > the installation.
[...]
> > + $(Q)$(call rte_mkdir, $(DESTDIR)$(sdkdir))
> > + $(Q)cp -a $(BUILD_DIR)/.config $(DESTDIR)$(sdkdir)
> > + $(Q)cp -a $(RTE_SDK)/{mk,scripts} $(DESTDIR)$(sdkdir)
> > + $(Q)$(call rte_symlink, $(DESTDIR)$(includedir), $(DESTDIR)$(sdkdir)/include)
> > + $(Q)$(call rte_symlink, $(DESTDIR)$(libdir), $(DESTDIR)$(sdkdir)/lib)
>
> $(prefix)/share is supposed to be shareable across different
> architectures. Most of the content here is, but at least the lib symlink
> and .config file are not.
The case you want to address is multilib 32/x32/64, right?
> One option is to install .config and the symlinks within $(sdkdir)/$(T)
> directories, then it can be shared across architectures because each
> lives in their own directory. Another possibility is moving the whole
> sdk directory into a subdir in $(libdir), but that misses the
> opportunity to share across architectures (whether anybody actually
> cares is a whole other question :)
Yes, I tried to remove the use of RTE_TARGET when building an example.
But we can keep it with a subdirectory in $(sdkdir).
> $(sdkdir)/lib -> $(libdir) symlink seems reasonable when installing to
> an empty staging root, but on a real-world installation it'd point to
> /usr/lib(something) which has hundreds or thousands of other unrelated
> libraries. My memory is hazy on details but I think this caused an
> actual problem with something because I ended up $(sdkdir)/lib an actual
> directory populated with symlinks to the individual DPDK libraries.
I don't see the problem.
I suggest to keep it and see how to fix it if an issue is raised.
On 12/02/2015 01:25 PM, Thomas Monjalon wrote:
> 2015-12-02 12:27, Panu Matilainen:
>> On 12/02/2015 05:57 AM, Thomas Monjalon wrote:
>>> The old installed tree was static and always had .config, includes and
>>> libs in a RTE_TARGET subdirectory. There is no such directory anymore in
>>> an installed SDK. So the top directory is checked.
>>> But RTE_TARGET can still be used, especially to build an app with a
>>> compiled but not installed SDK.
>>> That's why both cases are looked for RTE_SDK_BIN.
> [...]
>>> The old usage of an installed SDK is:
>>> make -C examples/helloworld RTE_SDK=$(readlink -m $DESTDIR) \
>>> RTE_TARGET=x86_64-native-linuxapp-gcc
>>> RTE_TARGET can be specified but is useless now with an installed SDK.
>>> The RTE_SDK directory must now point to a different path depending of
>>> the installation.
> [...]
>>> + $(Q)$(call rte_mkdir, $(DESTDIR)$(sdkdir))
>>> + $(Q)cp -a $(BUILD_DIR)/.config $(DESTDIR)$(sdkdir)
>>> + $(Q)cp -a $(RTE_SDK)/{mk,scripts} $(DESTDIR)$(sdkdir)
>>> + $(Q)$(call rte_symlink, $(DESTDIR)$(includedir), $(DESTDIR)$(sdkdir)/include)
>>> + $(Q)$(call rte_symlink, $(DESTDIR)$(libdir), $(DESTDIR)$(sdkdir)/lib)
>>
>> $(prefix)/share is supposed to be shareable across different
>> architectures. Most of the content here is, but at least the lib symlink
>> and .config file are not.
>
> The case you want to address is multilib 32/x32/64, right?
That, plus modern Debian/Ubuntu supports multiarch, not just -lib.
And then there's the pedantic side, ie to be in line with the FHS
definition:
http://www.pathname.com/fhs/pub/fhs-2.3.html#USRSHAREARCHITECTUREINDEPENDENTDATA
>
>> One option is to install .config and the symlinks within $(sdkdir)/$(T)
>> directories, then it can be shared across architectures because each
>> lives in their own directory. Another possibility is moving the whole
>> sdk directory into a subdir in $(libdir), but that misses the
>> opportunity to share across architectures (whether anybody actually
>> cares is a whole other question :)
>
> Yes, I tried to remove the use of RTE_TARGET when building an example.
> But we can keep it with a subdirectory in $(sdkdir).
Just realized my suggestion $(sdkdir)/$(T) would not cut it because if
T= is specified then this installation method wont be invoked at all :D
So yeah, RTE_TARGET. Or perhaps just RTE_ARCH. Dunno if there's actual
added value to having the whole target string there, but I wont mind either.
>
>> $(sdkdir)/lib -> $(libdir) symlink seems reasonable when installing to
>> an empty staging root, but on a real-world installation it'd point to
>> /usr/lib(something) which has hundreds or thousands of other unrelated
>> libraries. My memory is hazy on details but I think this caused an
>> actual problem with something because I ended up $(sdkdir)/lib an actual
>> directory populated with symlinks to the individual DPDK libraries.
>
> I don't see the problem.
> I suggest to keep it and see how to fix it if an issue is raised.
The problem probably had to do with something external, like compiling
OVS or pktgen, but ... this is too hand-wavy to worry about right now.
Just wanted to mention it because I dont think I added the extra
complexity in packaging just for fun.
- Panu -
2015-12-02 14:54, Panu Matilainen:
> On 12/02/2015 01:25 PM, Thomas Monjalon wrote:
> > 2015-12-02 12:27, Panu Matilainen:
> >> $(prefix)/share is supposed to be shareable across different
> >> architectures. Most of the content here is, but at least the lib symlink
> >> and .config file are not.
> >
> > The case you want to address is multilib 32/x32/64, right?
>
> That, plus modern Debian/Ubuntu supports multiarch, not just -lib.
We do not support completely different platforms (e.g. ARM and x86)
with only one include directory. At the moment, only variants (32/64)
live together.
> >> One option is to install .config and the symlinks within $(sdkdir)/$(T)
> >> directories, then it can be shared across architectures because each
> >> lives in their own directory. Another possibility is moving the whole
> >> sdk directory into a subdir in $(libdir), but that misses the
> >> opportunity to share across architectures (whether anybody actually
> >> cares is a whole other question :)
> >
> > Yes, I tried to remove the use of RTE_TARGET when building an example.
> > But we can keep it with a subdirectory in $(sdkdir).
>
> Just realized my suggestion $(sdkdir)/$(T) would not cut it because if
> T= is specified then this installation method wont be invoked at all :D
I don't understand what you mean.
In my patchset, the installation is the same (except some default values)
with and without T=.
> So yeah, RTE_TARGET. Or perhaps just RTE_ARCH. Dunno if there's actual
> added value to having the whole target string there, but I wont mind either.
RTE_TARGET is a safe choice for future.
On 12/02/2015 03:05 PM, Thomas Monjalon wrote:
> 2015-12-02 14:54, Panu Matilainen:
>> On 12/02/2015 01:25 PM, Thomas Monjalon wrote:
>>> 2015-12-02 12:27, Panu Matilainen:
>>>> $(prefix)/share is supposed to be shareable across different
>>>> architectures. Most of the content here is, but at least the lib symlink
>>>> and .config file are not.
>>>
>>> The case you want to address is multilib 32/x32/64, right?
>>
>> That, plus modern Debian/Ubuntu supports multiarch, not just -lib.
>
> We do not support completely different platforms (e.g. ARM and x86)
> with only one include directory. At the moment, only variants (32/64)
> live together.
Actually even the variants will run into problems because eg
rte_config.h will differ between 32- and 64-bit. But that's a problem
for another day, this is hardly the most pressing of issues :)
>
>>>> One option is to install .config and the symlinks within $(sdkdir)/$(T)
>>>> directories, then it can be shared across architectures because each
>>>> lives in their own directory. Another possibility is moving the whole
>>>> sdk directory into a subdir in $(libdir), but that misses the
>>>> opportunity to share across architectures (whether anybody actually
>>>> cares is a whole other question :)
>>>
>>> Yes, I tried to remove the use of RTE_TARGET when building an example.
>>> But we can keep it with a subdirectory in $(sdkdir).
>>
>> Just realized my suggestion $(sdkdir)/$(T) would not cut it because if
>> T= is specified then this installation method wont be invoked at all :D
>
> I don't understand what you mean.
> In my patchset, the installation is the same (except some default values)
> with and without T=.
Hmm, must've misuderstood/mixed up with something Marios patches do.
Never mind, I was just mumbling out loud anyhow.
>
>> So yeah, RTE_TARGET. Or perhaps just RTE_ARCH. Dunno if there's actual
>> added value to having the whole target string there, but I wont mind either.
>
> RTE_TARGET is a safe choice for future.
>
Nod.
- Panu -
@@ -19,6 +19,7 @@ Build variables
D debug dependencies
O build directory (default: build/ - install default: ./)
DESTDIR second-stage install directory
+ prefix root install directory
T target template - used with config or install
format: <arch-machine-execenv-toolchain>
templates in config/defconfig_*
@@ -51,7 +51,15 @@ endif
RTE_EXTMK ?= $(RTE_SRCDIR)/Makefile
export RTE_EXTMK
+# RTE_SDK_BIN must point to .config, include/ and lib/.
RTE_SDK_BIN := $(RTE_SDK)/$(RTE_TARGET)
+ifeq ($(wildcard $(RTE_SDK_BIN)/.config),)
+# RTE_TARGET is empty when using a default SDK installation.
+RTE_SDK_BIN := $(RTE_SDK)
+ifeq ($(wildcard $(RTE_SDK_BIN)/.config),)
+$(error Cannot find .config in $(RTE_SDK))
+endif
+endif
#
# Output files wil go in a separate directory: default output is
@@ -1,6 +1,7 @@
# BSD LICENSE
#
# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# Copyright 2015 6WIND S.A.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@@ -32,10 +33,30 @@
# Build directory is given with O=
O ?= .
+prefix ?= /usr/local
+exec_prefix ?= $(prefix)
+bindir ?= $(exec_prefix)/bin
+libdir ?= $(exec_prefix)/lib
+includedir ?= $(prefix)/include/dpdk
+datarootdir ?= $(prefix)/share
+datadir ?= $(datarootdir)/dpdk
+sdkdir ?= $(datadir)
+
+# The install directories may be staged in DESTDIR
+
# Configuration, compilation and installation can be done at once
# with make install T=<config>
# The build directory is T and may be prepended with O
BUILD_DIR := $O/$T
+# Default usage of T= will install an almost flat staging tree
+export prefix =
+
+# Create the directory $1 if not exists
+rte_mkdir = test -d $1 || mkdir -p $1
+
+# Create the relative symbolic link $2 -> $1
+# May be replaced with --relative option of ln from coreutils-8.16
+rte_symlink = ln -snf $$($(RTE_SDK)/scripts/relpath.sh $1 $(dir $2)) $2
.PHONY: pre_install
pre_install:
@@ -58,16 +79,28 @@ pre_install:
.PHONY: install
install: pre_install
- @echo ================== Installing $(DESTDIR)
- $(Q)mkdir -p $(DESTDIR)
- $(Q)tar -C $(RTE_SDK) -cf - mk scripts/*.sh | tar -C $(DESTDIR) -x \
- --keep-newer-files --warning=no-ignore-newer -f -
- $(Q)mkdir -p $(DESTDIR)/$T
- $(Q)tar -C $(BUILD_DIR) -chf - \
- --exclude app --exclude hostapp --exclude build \
- --exclude Makefile --exclude .depdirs . | \
- tar -C $(DESTDIR)/$T -x --keep-newer-files \
- --warning=no-ignore-newer -f -
- $(Q)install -D $(BUILD_DIR)/app/testpmd \
- $(DESTDIR)/$T/app/testpmd
- @echo Installation in $(DESTDIR) complete
+ifeq '$(DESTDIR)$(if $T,,+)' ''
+ @echo Installation cannot run with T defined and DESTDIR undefined
+else
+ @echo ================== Installing $(DESTDIR)$(prefix)/
+ $(Q)$(call rte_mkdir, $(DESTDIR)$(libdir))
+ $(Q)cp -a $(BUILD_DIR)/lib/* $(DESTDIR)$(libdir)
+ $(Q)$(call rte_mkdir, $(DESTDIR)$(bindir))
+ $(Q)tar -cf - -C $(BUILD_DIR) app --exclude 'app/*.map' \
+ --exclude 'app/cmdline*' --exclude app/test \
+ --exclude app/testacl --exclude app/testpipeline | \
+ tar -xf - -C $(DESTDIR)$(bindir) --strip-components=1 \
+ --keep-newer-files --warning=no-ignore-newer
+ $(Q)$(call rte_mkdir, $(DESTDIR)$(datadir))
+ $(Q)cp -a $(RTE_SDK)/tools $(DESTDIR)$(datadir)
+ $(Q)$(call rte_mkdir, $(DESTDIR)$(includedir))
+ $(Q)tar -chf - -C $(BUILD_DIR) include | \
+ tar -xf - -C $(DESTDIR)$(includedir) --strip-components=1 \
+ --keep-newer-files --warning=no-ignore-newer
+ $(Q)$(call rte_mkdir, $(DESTDIR)$(sdkdir))
+ $(Q)cp -a $(BUILD_DIR)/.config $(DESTDIR)$(sdkdir)
+ $(Q)cp -a $(RTE_SDK)/{mk,scripts} $(DESTDIR)$(sdkdir)
+ $(Q)$(call rte_symlink, $(DESTDIR)$(includedir), $(DESTDIR)$(sdkdir)/include)
+ $(Q)$(call rte_symlink, $(DESTDIR)$(libdir), $(DESTDIR)$(sdkdir)/lib)
+ @echo Installation in $(DESTDIR)$(prefix)/ complete
+endif
@@ -67,13 +67,6 @@ endif
RTE_LIBNAME := dpdk
-# RTE_TARGET is deducted from config when we are building the SDK.
-# Else, when building an external app, RTE_TARGET must be specified
-# by the user.
-ifeq ($(RTE_TARGET),)
-$(error RTE_TARGET is not defined)
-endif
-
ifeq ($(BUILDING_RTE_SDK),)
# if we are building an external app/lib, include internal/rte.extvars.mk that will
# define RTE_OUTPUT, RTE_SRCDIR, RTE_EXTMK, RTE_SDK_BIN, (etc ...)