diff mbox series

[v2] devtools: fix 32-bits build

Message ID 20201109145506.1758827-1-ferruh.yigit@intel.com (mailing list archive)
State Rejected, archived
Delegated to: Thomas Monjalon
Headers
Series [v2] devtools: fix 32-bits build |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/iol-broadcom-Functional success Functional Testing PASS
ci/iol-broadcom-Performance success Performance Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-testing success Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/travis-robot success Travis build: passed
ci/iol-mellanox-Performance success Performance Testing PASS
ci/Intel-compilation success Compilation OK

Commit Message

Ferruh Yigit Nov. 9, 2020, 2:55 p.m. UTC 
  If the 'PKG_CONFIG_PATH' is set in the environment before script run,
32 bit still uses that value for 64-bits libraries.

From the build log:

"
Using DPDK_TARGET i386-pc-linux-gnu
meson  -Dexamples=l3fwd --buildtype=debugoptimized --werror
-Dc_args=-m32 -Dc_link_args=-m32
/tmp/dpdk_maintain/self/dpdk/devtools/.. ./build-32b
...
Using 'PKG_CONFIG_PATH' from environment with value:
'/usr/local/lib64/pkgconfig/
"

This causes build error when linking with the found libraries.

Reproduced with 'librte_bpf' which only has 64 bit installed but still
enables building 'af_xdp' and link fails.

To fix this, using default 'PKG_CONFIG_PATH' variable unless
'CUSTOM_PKG_CONFIG_PATH' set, and set the 'CUSTOM_PKG_CONFIG_PATH' for
32 bit build.

Fixes: 9b83106d8784 ("devtools: test 32-bit build")

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
Cc: Bruce Richardson <bruce.richardson@intel.com>
Cc: Thomas Monjalon <thomas@monjalon.net>

v2:
* Enable overwriting default 'PKG_CONFIG_PATH' value
---
 devtools/test-meson-builds.sh | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

Comments

Thomas Monjalon Nov. 9, 2020, 3:28 p.m. UTC | #1 
09/11/2020 15:55, Ferruh Yigit:
> If the 'PKG_CONFIG_PATH' is set in the environment before script run,
> 32 bit still uses that value for 64-bits libraries.
> 
> From the build log:
> 
> "
> Using DPDK_TARGET i386-pc-linux-gnu
> meson  -Dexamples=l3fwd --buildtype=debugoptimized --werror
> -Dc_args=-m32 -Dc_link_args=-m32
> /tmp/dpdk_maintain/self/dpdk/devtools/.. ./build-32b
> ...
> Using 'PKG_CONFIG_PATH' from environment with value:
> '/usr/local/lib64/pkgconfig/
> "
> 
> This causes build error when linking with the found libraries.
> 
> Reproduced with 'librte_bpf' which only has 64 bit installed but still
> enables building 'af_xdp' and link fails.
> 
> To fix this, using default 'PKG_CONFIG_PATH' variable unless
> 'CUSTOM_PKG_CONFIG_PATH' set, and set the 'CUSTOM_PKG_CONFIG_PATH' for
> 32 bit build.
> 
> Fixes: 9b83106d8784 ("devtools: test 32-bit build")
> 
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> ---
> Cc: Bruce Richardson <bruce.richardson@intel.com>
> Cc: Thomas Monjalon <thomas@monjalon.net>
> 
> v2:
> * Enable overwriting default 'PKG_CONFIG_PATH' value

It was not my conclusion.
I think we can just reset all env vars.
Bruce Richardson Nov. 9, 2020, 3:44 p.m. UTC | #2 
On Mon, Nov 09, 2020 at 04:28:16PM +0100, Thomas Monjalon wrote:
> 09/11/2020 15:55, Ferruh Yigit:
> > If the 'PKG_CONFIG_PATH' is set in the environment before script run,
> > 32 bit still uses that value for 64-bits libraries.
> > 
> > From the build log:
> > 
> > "
> > Using DPDK_TARGET i386-pc-linux-gnu
> > meson  -Dexamples=l3fwd --buildtype=debugoptimized --werror
> > -Dc_args=-m32 -Dc_link_args=-m32
> > /tmp/dpdk_maintain/self/dpdk/devtools/.. ./build-32b
> > ...
> > Using 'PKG_CONFIG_PATH' from environment with value:
> > '/usr/local/lib64/pkgconfig/
> > "
> > 
> > This causes build error when linking with the found libraries.
> > 
> > Reproduced with 'librte_bpf' which only has 64 bit installed but still
> > enables building 'af_xdp' and link fails.
> > 
> > To fix this, using default 'PKG_CONFIG_PATH' variable unless
> > 'CUSTOM_PKG_CONFIG_PATH' set, and set the 'CUSTOM_PKG_CONFIG_PATH' for
> > 32 bit build.
> > 
> > Fixes: 9b83106d8784 ("devtools: test 32-bit build")
> > 
> > Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > ---
> > Cc: Bruce Richardson <bruce.richardson@intel.com>
> > Cc: Thomas Monjalon <thomas@monjalon.net>
> > 
> > v2:
> > * Enable overwriting default 'PKG_CONFIG_PATH' value
> 
> It was not my conclusion.
> I think we can just reset all env vars.
> 

Perhaps, but I think that may cause more problems for people who want
custom CFLAGS and LDFLAGS for building with extra 3rd-party packages.
While this can be done using the config script that is sourced in, I
suspect most of us do not use such a script.

 Therefore I'd suggest rather than clearing the env vars globally, we just
override them temporarily for the 32-bit, using CFLAGS32, LDFLAGS32,
PKG_CONFIG_PATH32 instead. That allows someone to have a 32-bit and 64-bit
version of e.g. libbpf installed, with a PKG_CONFIG_PATH pointing to each.

/Bruce
Thomas Monjalon Nov. 9, 2020, 4:14 p.m. UTC | #3 
09/11/2020 16:44, Bruce Richardson:
> On Mon, Nov 09, 2020 at 04:28:16PM +0100, Thomas Monjalon wrote:
> > 09/11/2020 15:55, Ferruh Yigit:
> > > If the 'PKG_CONFIG_PATH' is set in the environment before script run,
> > > 32 bit still uses that value for 64-bits libraries.
> > > 
> > > From the build log:
> > > 
> > > "
> > > Using DPDK_TARGET i386-pc-linux-gnu
> > > meson  -Dexamples=l3fwd --buildtype=debugoptimized --werror
> > > -Dc_args=-m32 -Dc_link_args=-m32
> > > /tmp/dpdk_maintain/self/dpdk/devtools/.. ./build-32b
> > > ...
> > > Using 'PKG_CONFIG_PATH' from environment with value:
> > > '/usr/local/lib64/pkgconfig/
> > > "
> > > 
> > > This causes build error when linking with the found libraries.
> > > 
> > > Reproduced with 'librte_bpf' which only has 64 bit installed but still
> > > enables building 'af_xdp' and link fails.
> > > 
> > > To fix this, using default 'PKG_CONFIG_PATH' variable unless
> > > 'CUSTOM_PKG_CONFIG_PATH' set, and set the 'CUSTOM_PKG_CONFIG_PATH' for
> > > 32 bit build.
> > > 
> > > Fixes: 9b83106d8784 ("devtools: test 32-bit build")
> > > 
> > > Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > > ---
> > > Cc: Bruce Richardson <bruce.richardson@intel.com>
> > > Cc: Thomas Monjalon <thomas@monjalon.net>
> > > 
> > > v2:
> > > * Enable overwriting default 'PKG_CONFIG_PATH' value
> > 
> > It was not my conclusion.
> > I think we can just reset all env vars.
> > 
> 
> Perhaps, but I think that may cause more problems for people who want
> custom CFLAGS and LDFLAGS for building with extra 3rd-party packages.
> While this can be done using the config script that is sourced in, I
> suspect most of us do not use such a script.
> 
>  Therefore I'd suggest rather than clearing the env vars globally, we just
> override them temporarily for the 32-bit, using CFLAGS32, LDFLAGS32,
> PKG_CONFIG_PATH32 instead. That allows someone to have a 32-bit and 64-bit
> version of e.g. libbpf installed, with a PKG_CONFIG_PATH pointing to each.

So you want to duplicate all vars just to avoid writing them cleanly in a file?
And what happens for other targets?

I'm tending to nack this approach.
There is no problem in using a clean config file.
Bruce Richardson Nov. 9, 2020, 4:19 p.m. UTC | #4 
On Mon, Nov 09, 2020 at 05:14:24PM +0100, Thomas Monjalon wrote:
> 09/11/2020 16:44, Bruce Richardson:
> > On Mon, Nov 09, 2020 at 04:28:16PM +0100, Thomas Monjalon wrote:
> > > 09/11/2020 15:55, Ferruh Yigit:
> > > > If the 'PKG_CONFIG_PATH' is set in the environment before script run,
> > > > 32 bit still uses that value for 64-bits libraries.
> > > > 
> > > > From the build log:
> > > > 
> > > > "
> > > > Using DPDK_TARGET i386-pc-linux-gnu
> > > > meson  -Dexamples=l3fwd --buildtype=debugoptimized --werror
> > > > -Dc_args=-m32 -Dc_link_args=-m32
> > > > /tmp/dpdk_maintain/self/dpdk/devtools/.. ./build-32b
> > > > ...
> > > > Using 'PKG_CONFIG_PATH' from environment with value:
> > > > '/usr/local/lib64/pkgconfig/
> > > > "
> > > > 
> > > > This causes build error when linking with the found libraries.
> > > > 
> > > > Reproduced with 'librte_bpf' which only has 64 bit installed but still
> > > > enables building 'af_xdp' and link fails.
> > > > 
> > > > To fix this, using default 'PKG_CONFIG_PATH' variable unless
> > > > 'CUSTOM_PKG_CONFIG_PATH' set, and set the 'CUSTOM_PKG_CONFIG_PATH' for
> > > > 32 bit build.
> > > > 
> > > > Fixes: 9b83106d8784 ("devtools: test 32-bit build")
> > > > 
> > > > Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > > > ---
> > > > Cc: Bruce Richardson <bruce.richardson@intel.com>
> > > > Cc: Thomas Monjalon <thomas@monjalon.net>
> > > > 
> > > > v2:
> > > > * Enable overwriting default 'PKG_CONFIG_PATH' value
> > > 
> > > It was not my conclusion.
> > > I think we can just reset all env vars.
> > > 
> > 
> > Perhaps, but I think that may cause more problems for people who want
> > custom CFLAGS and LDFLAGS for building with extra 3rd-party packages.
> > While this can be done using the config script that is sourced in, I
> > suspect most of us do not use such a script.
> > 
> >  Therefore I'd suggest rather than clearing the env vars globally, we just
> > override them temporarily for the 32-bit, using CFLAGS32, LDFLAGS32,
> > PKG_CONFIG_PATH32 instead. That allows someone to have a 32-bit and 64-bit
> > version of e.g. libbpf installed, with a PKG_CONFIG_PATH pointing to each.
> 
> So you want to duplicate all vars just to avoid writing them cleanly in a file?
> And what happens for other targets?
> 
> I'm tending to nack this approach.
> There is no problem in using a clean config file.
> 
Well, when you put it that way :-)
However, I was only thinking of the 3 variables above, though really it's
only the PKG_CONFIG_PATH that is going to be the most likely culprit, so
supporting just one extra var "PKG_CONFIG_PATH_32" seems reasonable.

I'm also ok with your solution of just clearing the environment and relying
on a config file too, though.

/Bruce
Ferruh Yigit Nov. 9, 2020, 4:48 p.m. UTC | #5 
On 11/9/2020 4:19 PM, Bruce Richardson wrote:
> On Mon, Nov 09, 2020 at 05:14:24PM +0100, Thomas Monjalon wrote:
>> 09/11/2020 16:44, Bruce Richardson:
>>> On Mon, Nov 09, 2020 at 04:28:16PM +0100, Thomas Monjalon wrote:
>>>> 09/11/2020 15:55, Ferruh Yigit:
>>>>> If the 'PKG_CONFIG_PATH' is set in the environment before script run,
>>>>> 32 bit still uses that value for 64-bits libraries.
>>>>>
>>>>>  From the build log:
>>>>>
>>>>> "
>>>>> Using DPDK_TARGET i386-pc-linux-gnu
>>>>> meson  -Dexamples=l3fwd --buildtype=debugoptimized --werror
>>>>> -Dc_args=-m32 -Dc_link_args=-m32
>>>>> /tmp/dpdk_maintain/self/dpdk/devtools/.. ./build-32b
>>>>> ...
>>>>> Using 'PKG_CONFIG_PATH' from environment with value:
>>>>> '/usr/local/lib64/pkgconfig/
>>>>> "
>>>>>
>>>>> This causes build error when linking with the found libraries.
>>>>>
>>>>> Reproduced with 'librte_bpf' which only has 64 bit installed but still
>>>>> enables building 'af_xdp' and link fails.
>>>>>
>>>>> To fix this, using default 'PKG_CONFIG_PATH' variable unless
>>>>> 'CUSTOM_PKG_CONFIG_PATH' set, and set the 'CUSTOM_PKG_CONFIG_PATH' for
>>>>> 32 bit build.
>>>>>
>>>>> Fixes: 9b83106d8784 ("devtools: test 32-bit build")
>>>>>
>>>>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>> ---
>>>>> Cc: Bruce Richardson <bruce.richardson@intel.com>
>>>>> Cc: Thomas Monjalon <thomas@monjalon.net>
>>>>>
>>>>> v2:
>>>>> * Enable overwriting default 'PKG_CONFIG_PATH' value
>>>>
>>>> It was not my conclusion.
>>>> I think we can just reset all env vars.
>>>>
>>>
>>> Perhaps, but I think that may cause more problems for people who want
>>> custom CFLAGS and LDFLAGS for building with extra 3rd-party packages.
>>> While this can be done using the config script that is sourced in, I
>>> suspect most of us do not use such a script.
>>>
>>>   Therefore I'd suggest rather than clearing the env vars globally, we just
>>> override them temporarily for the 32-bit, using CFLAGS32, LDFLAGS32,
>>> PKG_CONFIG_PATH32 instead. That allows someone to have a 32-bit and 64-bit
>>> version of e.g. libbpf installed, with a PKG_CONFIG_PATH pointing to each.
>>
>> So you want to duplicate all vars just to avoid writing them cleanly in a file?
>> And what happens for other targets?
>>
>> I'm tending to nack this approach.
>> There is no problem in using a clean config file.
>>
> Well, when you put it that way :-)
> However, I was only thinking of the 3 variables above, though really it's
> only the PKG_CONFIG_PATH that is going to be the most likely culprit, so
> supporting just one extra var "PKG_CONFIG_PATH_32" seems reasonable.
> 
> I'm also ok with your solution of just clearing the environment and relying
> on a config file too, though.
> 

I was worried for any side affects to reset the var, this is smaller change to 
change only 'CUSTOM_PKG_CONFIG_PATH' for 'build-32b'

I can send a new version to reset 'CUSTOM_PKG_CONFIG_PATH',
what is the intention/plan with the config file?
Thomas Monjalon Nov. 9, 2020, 5:01 p.m. UTC | #6 
09/11/2020 17:48, Ferruh Yigit:
> On 11/9/2020 4:19 PM, Bruce Richardson wrote:
> > On Mon, Nov 09, 2020 at 05:14:24PM +0100, Thomas Monjalon wrote:
> >> 09/11/2020 16:44, Bruce Richardson:
> >>> On Mon, Nov 09, 2020 at 04:28:16PM +0100, Thomas Monjalon wrote:
> >>>> 09/11/2020 15:55, Ferruh Yigit:
> >>>>> If the 'PKG_CONFIG_PATH' is set in the environment before script run,
> >>>>> 32 bit still uses that value for 64-bits libraries.
> >>>>>
> >>>>>  From the build log:
> >>>>>
> >>>>> "
> >>>>> Using DPDK_TARGET i386-pc-linux-gnu
> >>>>> meson  -Dexamples=l3fwd --buildtype=debugoptimized --werror
> >>>>> -Dc_args=-m32 -Dc_link_args=-m32
> >>>>> /tmp/dpdk_maintain/self/dpdk/devtools/.. ./build-32b
> >>>>> ...
> >>>>> Using 'PKG_CONFIG_PATH' from environment with value:
> >>>>> '/usr/local/lib64/pkgconfig/
> >>>>> "
> >>>>>
> >>>>> This causes build error when linking with the found libraries.
> >>>>>
> >>>>> Reproduced with 'librte_bpf' which only has 64 bit installed but still
> >>>>> enables building 'af_xdp' and link fails.
> >>>>>
> >>>>> To fix this, using default 'PKG_CONFIG_PATH' variable unless
> >>>>> 'CUSTOM_PKG_CONFIG_PATH' set, and set the 'CUSTOM_PKG_CONFIG_PATH' for
> >>>>> 32 bit build.
> >>>>>
> >>>>> Fixes: 9b83106d8784 ("devtools: test 32-bit build")
> >>>>>
> >>>>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>>> ---
> >>>>> Cc: Bruce Richardson <bruce.richardson@intel.com>
> >>>>> Cc: Thomas Monjalon <thomas@monjalon.net>
> >>>>>
> >>>>> v2:
> >>>>> * Enable overwriting default 'PKG_CONFIG_PATH' value
> >>>>
> >>>> It was not my conclusion.
> >>>> I think we can just reset all env vars.
> >>>>
> >>>
> >>> Perhaps, but I think that may cause more problems for people who want
> >>> custom CFLAGS and LDFLAGS for building with extra 3rd-party packages.
> >>> While this can be done using the config script that is sourced in, I
> >>> suspect most of us do not use such a script.
> >>>
> >>>   Therefore I'd suggest rather than clearing the env vars globally, we just
> >>> override them temporarily for the 32-bit, using CFLAGS32, LDFLAGS32,
> >>> PKG_CONFIG_PATH32 instead. That allows someone to have a 32-bit and 64-bit
> >>> version of e.g. libbpf installed, with a PKG_CONFIG_PATH pointing to each.
> >>
> >> So you want to duplicate all vars just to avoid writing them cleanly in a file?
> >> And what happens for other targets?
> >>
> >> I'm tending to nack this approach.
> >> There is no problem in using a clean config file.
> >>
> > Well, when you put it that way :-)
> > However, I was only thinking of the 3 variables above, though really it's
> > only the PKG_CONFIG_PATH that is going to be the most likely culprit, so
> > supporting just one extra var "PKG_CONFIG_PATH_32" seems reasonable.
> > 
> > I'm also ok with your solution of just clearing the environment and relying
> > on a config file too, though.
> > 
> 
> I was worried for any side affects to reset the var, this is smaller change to 
> change only 'CUSTOM_PKG_CONFIG_PATH' for 'build-32b'
> 
> I can send a new version to reset 'CUSTOM_PKG_CONFIG_PATH',
> what is the intention/plan with the config file?

You should use a config file to set PKG_CONFIG_PATH, PATH,
DPDK_MESON_OPTIONS, etc accordingly to build with all
your non-installed dependencies.
CUSTOM_PKG_CONFIG_PATH is a way to avoid using the config file
in a single case.

Don't be afraid breaking the assumption of theoritical users
of this script. If we want to avoid PATH reset,
we can add such line at the beginning of the script:
default_path=${PATH:-/bin:/usr/bin}
Ferruh Yigit Nov. 9, 2020, 5:15 p.m. UTC | #7 
On 11/9/2020 5:01 PM, Thomas Monjalon wrote:
> 09/11/2020 17:48, Ferruh Yigit:
>> On 11/9/2020 4:19 PM, Bruce Richardson wrote:
>>> On Mon, Nov 09, 2020 at 05:14:24PM +0100, Thomas Monjalon wrote:
>>>> 09/11/2020 16:44, Bruce Richardson:
>>>>> On Mon, Nov 09, 2020 at 04:28:16PM +0100, Thomas Monjalon wrote:
>>>>>> 09/11/2020 15:55, Ferruh Yigit:
>>>>>>> If the 'PKG_CONFIG_PATH' is set in the environment before script run,
>>>>>>> 32 bit still uses that value for 64-bits libraries.
>>>>>>>
>>>>>>>   From the build log:
>>>>>>>
>>>>>>> "
>>>>>>> Using DPDK_TARGET i386-pc-linux-gnu
>>>>>>> meson  -Dexamples=l3fwd --buildtype=debugoptimized --werror
>>>>>>> -Dc_args=-m32 -Dc_link_args=-m32
>>>>>>> /tmp/dpdk_maintain/self/dpdk/devtools/.. ./build-32b
>>>>>>> ...
>>>>>>> Using 'PKG_CONFIG_PATH' from environment with value:
>>>>>>> '/usr/local/lib64/pkgconfig/
>>>>>>> "
>>>>>>>
>>>>>>> This causes build error when linking with the found libraries.
>>>>>>>
>>>>>>> Reproduced with 'librte_bpf' which only has 64 bit installed but still
>>>>>>> enables building 'af_xdp' and link fails.
>>>>>>>
>>>>>>> To fix this, using default 'PKG_CONFIG_PATH' variable unless
>>>>>>> 'CUSTOM_PKG_CONFIG_PATH' set, and set the 'CUSTOM_PKG_CONFIG_PATH' for
>>>>>>> 32 bit build.
>>>>>>>
>>>>>>> Fixes: 9b83106d8784 ("devtools: test 32-bit build")
>>>>>>>
>>>>>>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>>>> ---
>>>>>>> Cc: Bruce Richardson <bruce.richardson@intel.com>
>>>>>>> Cc: Thomas Monjalon <thomas@monjalon.net>
>>>>>>>
>>>>>>> v2:
>>>>>>> * Enable overwriting default 'PKG_CONFIG_PATH' value
>>>>>>
>>>>>> It was not my conclusion.
>>>>>> I think we can just reset all env vars.
>>>>>>
>>>>>
>>>>> Perhaps, but I think that may cause more problems for people who want
>>>>> custom CFLAGS and LDFLAGS for building with extra 3rd-party packages.
>>>>> While this can be done using the config script that is sourced in, I
>>>>> suspect most of us do not use such a script.
>>>>>
>>>>>    Therefore I'd suggest rather than clearing the env vars globally, we just
>>>>> override them temporarily for the 32-bit, using CFLAGS32, LDFLAGS32,
>>>>> PKG_CONFIG_PATH32 instead. That allows someone to have a 32-bit and 64-bit
>>>>> version of e.g. libbpf installed, with a PKG_CONFIG_PATH pointing to each.
>>>>
>>>> So you want to duplicate all vars just to avoid writing them cleanly in a file?
>>>> And what happens for other targets?
>>>>
>>>> I'm tending to nack this approach.
>>>> There is no problem in using a clean config file.
>>>>
>>> Well, when you put it that way :-)
>>> However, I was only thinking of the 3 variables above, though really it's
>>> only the PKG_CONFIG_PATH that is going to be the most likely culprit, so
>>> supporting just one extra var "PKG_CONFIG_PATH_32" seems reasonable.
>>>
>>> I'm also ok with your solution of just clearing the environment and relying
>>> on a config file too, though.
>>>
>>
>> I was worried for any side affects to reset the var, this is smaller change to
>> change only 'CUSTOM_PKG_CONFIG_PATH' for 'build-32b'
>>
>> I can send a new version to reset 'CUSTOM_PKG_CONFIG_PATH',
>> what is the intention/plan with the config file?
> 
> You should use a config file to set PKG_CONFIG_PATH, PATH,
> DPDK_MESON_OPTIONS, etc accordingly to build with all
> your non-installed dependencies.
> CUSTOM_PKG_CONFIG_PATH is a way to avoid using the config file
> in a single case.
> 

Either use 'PKG_CONFIG_PATH' environment variable or a config file, both has the 
same problem that we need two different pkg config path, one for 64bit and one 
for 32bit.

v2 of this patch provides a way to use correct one when needed.

Moving the default values from system environment or script hardcoded values to 
a config file is something else and can be done later, what do you think?


> Don't be afraid breaking the assumption of theoritical users
> of this script. If we want to avoid PATH reset,
> we can add such line at the beginning of the script:
> default_path=${PATH:-/bin:/usr/bin}
> 
>
Thomas Monjalon Nov. 9, 2020, 5:20 p.m. UTC | #8 
09/11/2020 18:15, Ferruh Yigit:
> On 11/9/2020 5:01 PM, Thomas Monjalon wrote:
> > 09/11/2020 17:48, Ferruh Yigit:
> >> On 11/9/2020 4:19 PM, Bruce Richardson wrote:
> >>> On Mon, Nov 09, 2020 at 05:14:24PM +0100, Thomas Monjalon wrote:
> >>>> 09/11/2020 16:44, Bruce Richardson:
> >>>>> On Mon, Nov 09, 2020 at 04:28:16PM +0100, Thomas Monjalon wrote:
> >>>>>> 09/11/2020 15:55, Ferruh Yigit:
> >>>>>>> If the 'PKG_CONFIG_PATH' is set in the environment before script run,
> >>>>>>> 32 bit still uses that value for 64-bits libraries.
> >>>>>>>
> >>>>>>>   From the build log:
> >>>>>>>
> >>>>>>> "
> >>>>>>> Using DPDK_TARGET i386-pc-linux-gnu
> >>>>>>> meson  -Dexamples=l3fwd --buildtype=debugoptimized --werror
> >>>>>>> -Dc_args=-m32 -Dc_link_args=-m32
> >>>>>>> /tmp/dpdk_maintain/self/dpdk/devtools/.. ./build-32b
> >>>>>>> ...
> >>>>>>> Using 'PKG_CONFIG_PATH' from environment with value:
> >>>>>>> '/usr/local/lib64/pkgconfig/
> >>>>>>> "
> >>>>>>>
> >>>>>>> This causes build error when linking with the found libraries.
> >>>>>>>
> >>>>>>> Reproduced with 'librte_bpf' which only has 64 bit installed but still
> >>>>>>> enables building 'af_xdp' and link fails.
> >>>>>>>
> >>>>>>> To fix this, using default 'PKG_CONFIG_PATH' variable unless
> >>>>>>> 'CUSTOM_PKG_CONFIG_PATH' set, and set the 'CUSTOM_PKG_CONFIG_PATH' for
> >>>>>>> 32 bit build.
> >>>>>>>
> >>>>>>> Fixes: 9b83106d8784 ("devtools: test 32-bit build")
> >>>>>>>
> >>>>>>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>>>>> ---
> >>>>>>> Cc: Bruce Richardson <bruce.richardson@intel.com>
> >>>>>>> Cc: Thomas Monjalon <thomas@monjalon.net>
> >>>>>>>
> >>>>>>> v2:
> >>>>>>> * Enable overwriting default 'PKG_CONFIG_PATH' value
> >>>>>>
> >>>>>> It was not my conclusion.
> >>>>>> I think we can just reset all env vars.
> >>>>>>
> >>>>>
> >>>>> Perhaps, but I think that may cause more problems for people who want
> >>>>> custom CFLAGS and LDFLAGS for building with extra 3rd-party packages.
> >>>>> While this can be done using the config script that is sourced in, I
> >>>>> suspect most of us do not use such a script.
> >>>>>
> >>>>>    Therefore I'd suggest rather than clearing the env vars globally, we just
> >>>>> override them temporarily for the 32-bit, using CFLAGS32, LDFLAGS32,
> >>>>> PKG_CONFIG_PATH32 instead. That allows someone to have a 32-bit and 64-bit
> >>>>> version of e.g. libbpf installed, with a PKG_CONFIG_PATH pointing to each.
> >>>>
> >>>> So you want to duplicate all vars just to avoid writing them cleanly in a file?
> >>>> And what happens for other targets?
> >>>>
> >>>> I'm tending to nack this approach.
> >>>> There is no problem in using a clean config file.
> >>>>
> >>> Well, when you put it that way :-)
> >>> However, I was only thinking of the 3 variables above, though really it's
> >>> only the PKG_CONFIG_PATH that is going to be the most likely culprit, so
> >>> supporting just one extra var "PKG_CONFIG_PATH_32" seems reasonable.
> >>>
> >>> I'm also ok with your solution of just clearing the environment and relying
> >>> on a config file too, though.
> >>>
> >>
> >> I was worried for any side affects to reset the var, this is smaller change to
> >> change only 'CUSTOM_PKG_CONFIG_PATH' for 'build-32b'
> >>
> >> I can send a new version to reset 'CUSTOM_PKG_CONFIG_PATH',
> >> what is the intention/plan with the config file?
> > 
> > You should use a config file to set PKG_CONFIG_PATH, PATH,
> > DPDK_MESON_OPTIONS, etc accordingly to build with all
> > your non-installed dependencies.
> > CUSTOM_PKG_CONFIG_PATH is a way to avoid using the config file
> > in a single case.
> > 
> 
> Either use 'PKG_CONFIG_PATH' environment variable or a config file, both has the 
> same problem that we need two different pkg config path, one for 64bit and one 
> for 32bit.

No. In the config file, you can set the right value by testing $DPDK_TARGET.
Note: the config file is a shell script.

> v2 of this patch provides a way to use correct one when needed.

This is not a global fix.

> Moving the default values from system environment or script hardcoded values to 
> a config file is something else and can be done later, what do you think?

The root cause of your issue is using variables from the environment.
The global fix is then to reset them all,
while keeping a default value for PATH which is not only for compilation config.


> > Don't be afraid breaking the assumption of theoritical users
> > of this script. If we want to avoid PATH reset,
> > we can add such line at the beginning of the script:
> > default_path=${PATH:-/bin:/usr/bin}
Thomas Monjalon Nov. 9, 2020, 5:44 p.m. UTC | #9 
09/11/2020 18:20, Thomas Monjalon:
> 09/11/2020 18:15, Ferruh Yigit:
> > On 11/9/2020 5:01 PM, Thomas Monjalon wrote:
> > > 09/11/2020 17:48, Ferruh Yigit:
> > >> I can send a new version to reset 'CUSTOM_PKG_CONFIG_PATH',
> > >> what is the intention/plan with the config file?
> > > 
> > > You should use a config file to set PKG_CONFIG_PATH, PATH,
> > > DPDK_MESON_OPTIONS, etc accordingly to build with all
> > > your non-installed dependencies.
> > > CUSTOM_PKG_CONFIG_PATH is a way to avoid using the config file
> > > in a single case.
> > > 
> > 
> > Either use 'PKG_CONFIG_PATH' environment variable or a config file, both has the 
> > same problem that we need two different pkg config path, one for 64bit and one 
> > for 32bit.
> 
> No. In the config file, you can set the right value by testing $DPDK_TARGET.
> Note: the config file is a shell script.
> 
> > v2 of this patch provides a way to use correct one when needed.
> 
> This is not a global fix.
> 
> > Moving the default values from system environment or script hardcoded values to 
> > a config file is something else and can be done later, what do you think?
> 
> The root cause of your issue is using variables from the environment.
> The global fix is then to reset them all,
> while keeping a default value for PATH which is not only for compilation config.

This is what we have currently:

    export PATH=$default_path
    export PKG_CONFIG_PATH=$default_pkgpath
    export CPPFLAGS=$default_cppflags
    export CFLAGS=$default_cflags 
    export LDFLAGS=$default_ldflags
    unset DPDK_MESON_OPTIONS

The PATH needs to be restored at each run to avoid using a wrong toolchain.
The default value is best taken from the start environment,
at the condition no conflicting toolchain is already set.

PKG_CONFIG_PATH is specific to each target.
I believe it can be reset at each run and configured
only from the config file.
A default PKG_CONFIG_PATH for all targets does not make any sense.

CPPFLAGS, CFLAGS and LDFLAGS could be used to customize some options
applying to all targets in some build tests without touching the config file.

DPDK_MESON_OPTIONS could be the same as CFLAGS: take a global default from env
and restore the value at each run, so each target can have additional options
from the config file.

Reminder about what can be a config file loaded by devtools/load-devel-config:
Note: the complex regexps below are for compatibility with the Make build tests.

if echo $DPDK_TARGET | grep -q '^i[3-6]86' ; then
	export PKG_CONFIG_PATH=${mlxdep}32/lib/pkgconfig
elif echo $DPDK_TARGET | grep -q '^x86_64-w' ; then # MinGW
	:
elif echo $DPDK_TARGET | grep -q '^x86_64' ; then
	export DPDK_MESON_OPTIONS="enable_kmods=true"
	export CFLAGS="-I$aesnidep -I$root/libsso/libsso-snow3g-0.3.1/include -I$root/libsso/libsso-kasumi-0.3.1/include -I$root/libsso/libsso-zuc-0.1.1-182/include"
	export LDFLAGS="-L$aesnidep -L$root/libsso/libsso-snow3g-0.3.1/build -L$root/libsso/libsso-kasumi-0.3.1/build -L$root/libsso/libsso-zuc-0.1.1-182/build"
	export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$mlxdep/lib/pkgconfig:$nfbdep/lib64/pkgconfig:$szedep/lib64/pkgconfig:$isaldep/..
elif echo $DPDK_TARGET | grep -q '^arm-' ; then
	export PATH=$PATH:$root/../pkg/gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf/bin
elif echo $DPDK_TARGET | grep -q '^a.*64-' ; then
	export DPDK_MESON_OPTIONS="lib_musdk_dir=$LIBMUSDK_PATH"
elif echo $DPDK_TARGET | grep -q '^p.*pc.*64' ; then
	export PATH=$PATH:$root/../pkg/powerpc64le-power8--glibc--stable-2018.11-1/bin
fi

export DPDK_GETMAINTAINER_PATH=$root/linux/scripts/get_maintainer.pl
export DPDK_CHECKPATCH_PATH=$root/linux/scripts/checkpatch.pl
export DPDK_CHECKPATCH_LINE_LENGTH=100
export DPDK_CHECKPATCH_CODESPELL=$root/codespell/dictionary.txt

export DPDK_BUILD_TEST_DIR=$root/dpdk-build
export DPDK_ABI_REF_DIR=$DPDK_BUILD_TEST_DIR/abiref
#export DPDK_ABI_REF_VERSION=v20.11
Thomas Monjalon Nov. 9, 2020, 9:02 p.m. UTC | #10 
09/11/2020 18:44, Thomas Monjalon:
> 09/11/2020 18:20, Thomas Monjalon:
> > 09/11/2020 18:15, Ferruh Yigit:
> > > On 11/9/2020 5:01 PM, Thomas Monjalon wrote:
> > > > 09/11/2020 17:48, Ferruh Yigit:
> > > >> I can send a new version to reset 'CUSTOM_PKG_CONFIG_PATH',
> > > >> what is the intention/plan with the config file?
> > > > 
> > > > You should use a config file to set PKG_CONFIG_PATH, PATH,
> > > > DPDK_MESON_OPTIONS, etc accordingly to build with all
> > > > your non-installed dependencies.
> > > > CUSTOM_PKG_CONFIG_PATH is a way to avoid using the config file
> > > > in a single case.
> > > > 
> > > 
> > > Either use 'PKG_CONFIG_PATH' environment variable or a config file, both has the 
> > > same problem that we need two different pkg config path, one for 64bit and one 
> > > for 32bit.
> > 
> > No. In the config file, you can set the right value by testing $DPDK_TARGET.
> > Note: the config file is a shell script.
> > 
> > > v2 of this patch provides a way to use correct one when needed.
> > 
> > This is not a global fix.
> > 
> > > Moving the default values from system environment or script hardcoded values to 
> > > a config file is something else and can be done later, what do you think?
> > 
> > The root cause of your issue is using variables from the environment.
> > The global fix is then to reset them all,
> > while keeping a default value for PATH which is not only for compilation config.
> 
> This is what we have currently:
> 
>     export PATH=$default_path
>     export PKG_CONFIG_PATH=$default_pkgpath
>     export CPPFLAGS=$default_cppflags
>     export CFLAGS=$default_cflags 
>     export LDFLAGS=$default_ldflags
>     unset DPDK_MESON_OPTIONS
> 
> The PATH needs to be restored at each run to avoid using a wrong toolchain.
> The default value is best taken from the start environment,
> at the condition no conflicting toolchain is already set.
> 
> PKG_CONFIG_PATH is specific to each target.
> I believe it can be reset at each run and configured
> only from the config file.
> A default PKG_CONFIG_PATH for all targets does not make any sense.
> 
> CPPFLAGS, CFLAGS and LDFLAGS could be used to customize some options
> applying to all targets in some build tests without touching the config file.
> 
> DPDK_MESON_OPTIONS could be the same as CFLAGS: take a global default from env
> and restore the value at each run, so each target can have additional options
> from the config file.

The implementation of the above is done in this patch:
	https://patches.dpdk.org/patch/83861/
Ferruh Yigit Nov. 11, 2020, 12:34 p.m. UTC | #11 
On 11/9/2020 9:02 PM, Thomas Monjalon wrote:
> 09/11/2020 18:44, Thomas Monjalon:
>> 09/11/2020 18:20, Thomas Monjalon:
>>> 09/11/2020 18:15, Ferruh Yigit:
>>>> On 11/9/2020 5:01 PM, Thomas Monjalon wrote:
>>>>> 09/11/2020 17:48, Ferruh Yigit:
>>>>>> I can send a new version to reset 'CUSTOM_PKG_CONFIG_PATH',
>>>>>> what is the intention/plan with the config file?
>>>>>
>>>>> You should use a config file to set PKG_CONFIG_PATH, PATH,
>>>>> DPDK_MESON_OPTIONS, etc accordingly to build with all
>>>>> your non-installed dependencies.
>>>>> CUSTOM_PKG_CONFIG_PATH is a way to avoid using the config file
>>>>> in a single case.
>>>>>
>>>>
>>>> Either use 'PKG_CONFIG_PATH' environment variable or a config file, both has the
>>>> same problem that we need two different pkg config path, one for 64bit and one
>>>> for 32bit.
>>>
>>> No. In the config file, you can set the right value by testing $DPDK_TARGET.
>>> Note: the config file is a shell script.
>>>
>>>> v2 of this patch provides a way to use correct one when needed.
>>>
>>> This is not a global fix.
>>>
>>>> Moving the default values from system environment or script hardcoded values to
>>>> a config file is something else and can be done later, what do you think?
>>>
>>> The root cause of your issue is using variables from the environment.
>>> The global fix is then to reset them all,
>>> while keeping a default value for PATH which is not only for compilation config.
>>
>> This is what we have currently:
>>
>>      export PATH=$default_path
>>      export PKG_CONFIG_PATH=$default_pkgpath
>>      export CPPFLAGS=$default_cppflags
>>      export CFLAGS=$default_cflags
>>      export LDFLAGS=$default_ldflags
>>      unset DPDK_MESON_OPTIONS
>>
>> The PATH needs to be restored at each run to avoid using a wrong toolchain.
>> The default value is best taken from the start environment,
>> at the condition no conflicting toolchain is already set.
>>
>> PKG_CONFIG_PATH is specific to each target.
>> I believe it can be reset at each run and configured
>> only from the config file.
>> A default PKG_CONFIG_PATH for all targets does not make any sense.
>>
>> CPPFLAGS, CFLAGS and LDFLAGS could be used to customize some options
>> applying to all targets in some build tests without touching the config file.
>>
>> DPDK_MESON_OPTIONS could be the same as CFLAGS: take a global default from env
>> and restore the value at each run, so each target can have additional options
>> from the config file.
> 
> The implementation of the above is done in this patch:
> 	https://patches.dpdk.org/patch/83861/
> 
> 

self nack this patch on behalf of above one.
diff mbox series

Patch

diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index 0c95d1cc98..29aed9ac84 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -53,7 +53,11 @@  load_env () # <target compiler>
 {
 	targetcc=$1
 	export PATH=$default_path
-	export PKG_CONFIG_PATH=$default_pkgpath
+	if [ -n "$CUSTOM_PKG_CONFIG_PATH" ]; then
+		export PKG_CONFIG_PATH=$CUSTOM_PKG_CONFIG_PATH
+	else
+		export PKG_CONFIG_PATH=$default_pkgpath
+	fi
 	export CPPFLAGS=$default_cppflags
 	export CFLAGS=$default_cflags
 	export LDFLAGS=$default_ldflags
@@ -226,10 +230,12 @@  if check_cc_flags '-m32' ; then
 		# 32-bit pkgconfig on RHEL/Fedora (lib vs lib64)
 		export PKG_CONFIG_LIBDIR='/usr/lib/pkgconfig'
 	fi
+	export CUSTOM_PKG_CONFIG_PATH=$PKG_CONFIG_LIBDIR
 	target_override='i386-pc-linux-gnu'
 	build build-32b cc -Dc_args='-m32' -Dc_link_args='-m32'
 	target_override=
 	unset PKG_CONFIG_LIBDIR
+	unset CUSTOM_PKG_CONFIG_PATH
 fi
 
 # x86 MinGW