diff mbox series

[2/2] build: find max lcore programmatically

Message ID	20200825211317.8358-2-dharmik.thakkar@arm.com (mailing list archive)
State	Rejected, archived
Delegated to:	Thomas Monjalon
Headers	From: Dharmik Thakkar <dharmik.thakkar@arm.com> To: Thomas Monjalon <thomas@monjalon.net> Cc: dev@dpdk.org, nd@arm.com, Dharmik Thakkar <dharmik.thakkar@arm.com> Date: Tue, 25 Aug 2020 16:13:17 -0500 Message-Id: <20200825211317.8358-2-dharmik.thakkar@arm.com> In-Reply-To: <20200825211317.8358-1-dharmik.thakkar@arm.com> References: <20200825211317.8358-1-dharmik.thakkar@arm.com> Subject: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically Precedence: list Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org>
Series	[1/2] config/arm: avoid variable reuse \| [1/2] config/arm: avoid variable reuse [2/2] build: find max lcore programmatically

Checks

Context	Check	Description
ci/Intel-compilation	fail	Compilation issues
ci/iol-testing	fail	Testing issues
ci/iol-mellanox-Performance	success	Performance Testing PASS
ci/travis-robot	warning	Travis build: failed
ci/checkpatch	success	coding style OK

Commit Message

Dharmik Thakkar Aug. 25, 2020, 9:13 p.m. UTC

  For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to incorrect
RTE_MAX_LCORE when machines have same Implemener and part number
but different number of CPUs.
For x86, RTE_MAX_LCORE is always set to 128 (using the value
set in meson_options.txt)

Use python script to find max lcore when using native build to
correctly set RTE_MAX_LCORE.

Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
---
 config/get_max_lcores.py | 13 +++++++++++++
 config/meson.build       | 13 ++++++++++++-
 2 files changed, 25 insertions(+), 1 deletion(-)
 create mode 100755 config/get_max_lcores.py

Comments

Jerin Jacob Aug. 26, 2020, 4:47 a.m. UTC | #1

On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar <dharmik.thakkar@arm.com> wrote:
>
> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to incorrect
> RTE_MAX_LCORE when machines have same Implemener and part number
> but different number of CPUs.
> For x86, RTE_MAX_LCORE is always set to 128 (using the value
> set in meson_options.txt)
>
> Use python script to find max lcore when using native build to
> correctly set RTE_MAX_LCORE.

We may need to build on the native arm64 machine and use it on another
arm64 machine(Just like x86).
So I think, at least for default config(which will be used by
distribution) to support max
lcores as fixed. I am not sure this patch changes those aspects or
not? Please check.

>
> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> ---
>  config/get_max_lcores.py | 13 +++++++++++++
>  config/meson.build       | 13 ++++++++++++-
>  2 files changed, 25 insertions(+), 1 deletion(-)
>  create mode 100755 config/get_max_lcores.py
>
> diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py
> new file mode 100755
> index 000000000000..ebf1c7efdadd
> --- /dev/null
> +++ b/config/get_max_lcores.py
> @@ -0,0 +1,13 @@
> +#!/usr/bin/python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2020 Arm Limited
> +
> +import os
> +
> +max_lcores = []
> +
> +nCPU = os.cpu_count()
> +
> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
> +
> +print(' '.join(max_lcores))
> diff --git a/config/meson.build b/config/meson.build
> index 6996e5cbeaa5..80c05bc15d2f 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller reserved memory areas
>         dpdk_conf.set('RTE_MAX_MEM_MB', 2048)
>  endif
>
> -
>  compile_time_cpuflags = []
>  subdir(arch_subdir)
>  dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags))
>
> +# set max lcores
> +if machine != 'default' and not meson.is_cross_build()
> +       # The script returns max lcores
> +       params = files('get_max_lcores.py')
> +       cmd_out = run_command(params)
> +       if cmd_out.returncode() == 0
> +               cmd_lcore = cmd_out.stdout().to_lower().strip().split(' ')
> +       endif
> +       max_lcore = cmd_lcore[0].to_int()
> +       dpdk_conf.set('RTE_MAX_LCORE', max_lcore)
> +endif
> +
>  # set the install path for the drivers
>  dpdk_conf.set_quoted('RTE_EAL_PMD_PATH', eal_pmd_path)
>
> --
> 2.17.1
>

Dharmik Thakkar Aug. 26, 2020, 4:55 a.m. UTC | #2

> On Aug 25, 2020, at 11:47 PM, Jerin Jacob <jerinjacobk@gmail.com> wrote:
> 
> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar <dharmik.thakkar@arm.com> wrote:
>> 
>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to incorrect
>> RTE_MAX_LCORE when machines have same Implemener and part number
>> but different number of CPUs.
>> For x86, RTE_MAX_LCORE is always set to 128 (using the value
>> set in meson_options.txt)
>> 
>> Use python script to find max lcore when using native build to
>> correctly set RTE_MAX_LCORE.
> 
> We may need to build on the native arm64 machine and use it on another
> arm64 machine(Just like x86).
> So I think, at least for default config(which will be used by
> distribution) to support max
> lcores as fixed. I am not sure this patch changes those aspects or
> not? Please check.

This patch does *not* affect ‘default’ build type and cross-compilation.

> 
>> 
>> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
>> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> ---
>> config/get_max_lcores.py | 13 +++++++++++++
>> config/meson.build       | 13 ++++++++++++-
>> 2 files changed, 25 insertions(+), 1 deletion(-)
>> create mode 100755 config/get_max_lcores.py
>> 
>> diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py
>> new file mode 100755
>> index 000000000000..ebf1c7efdadd
>> --- /dev/null
>> +++ b/config/get_max_lcores.py
>> @@ -0,0 +1,13 @@
>> +#!/usr/bin/python3
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2020 Arm Limited
>> +
>> +import os
>> +
>> +max_lcores = []
>> +
>> +nCPU = os.cpu_count()
>> +
>> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
>> +
>> +print(' '.join(max_lcores))
>> diff --git a/config/meson.build b/config/meson.build
>> index 6996e5cbeaa5..80c05bc15d2f 100644
>> --- a/config/meson.build
>> +++ b/config/meson.build
>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller reserved memory areas
>>        dpdk_conf.set('RTE_MAX_MEM_MB', 2048)
>> endif
>> 
>> -
>> compile_time_cpuflags = []
>> subdir(arch_subdir)
>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags))
>> 
>> +# set max lcores
>> +if machine != 'default' and not meson.is_cross_build()
>> +       # The script returns max lcores
>> +       params = files('get_max_lcores.py')
>> +       cmd_out = run_command(params)
>> +       if cmd_out.returncode() == 0
>> +               cmd_lcore = cmd_out.stdout().to_lower().strip().split(' ')
>> +       endif
>> +       max_lcore = cmd_lcore[0].to_int()
>> +       dpdk_conf.set('RTE_MAX_LCORE', max_lcore)
>> +endif
>> +
>> # set the install path for the drivers
>> dpdk_conf.set_quoted('RTE_EAL_PMD_PATH', eal_pmd_path)
>> 
>> --
>> 2.17.1
>>

Juraj Linkeš Sept. 3, 2020, 6:20 a.m. UTC | #3

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
> Sent: Wednesday, August 26, 2020 6:56 AM
> To: Jerin Jacob <jerinjacobk@gmail.com>
> Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
> 
> 
> 
> > On Aug 25, 2020, at 11:47 PM, Jerin Jacob <jerinjacobk@gmail.com> wrote:
> >
> > On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar
> <dharmik.thakkar@arm.com> wrote:
> >>
> >> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to
> >> incorrect RTE_MAX_LCORE when machines have same Implemener and part
> >> number but different number of CPUs.
> >> For x86, RTE_MAX_LCORE is always set to 128 (using the value set in
> >> meson_options.txt)
> >>
> >> Use python script to find max lcore when using native build to
> >> correctly set RTE_MAX_LCORE.
> >
> > We may need to build on the native arm64 machine and use it on another
> > arm64 machine(Just like x86).
> > So I think, at least for default config(which will be used by
> > distribution) to support max
> > lcores as fixed. I am not sure this patch changes those aspects or
> > not? Please check.
> 
> This patch does *not* affect ‘default’ build type and cross-compilation.
> 
> >
> >>
> >> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
> >> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> ---
> >> config/get_max_lcores.py | 13 +++++++++++++
> >> config/meson.build       | 13 ++++++++++++-
> >> 2 files changed, 25 insertions(+), 1 deletion(-) create mode 100755
> >> config/get_max_lcores.py
> >>
> >> diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py new
> >> file mode 100755 index 000000000000..ebf1c7efdadd
> >> --- /dev/null
> >> +++ b/config/get_max_lcores.py
> >> @@ -0,0 +1,13 @@
> >> +#!/usr/bin/python3
> >> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020 Arm
> >> +Limited
> >> +
> >> +import os
> >> +
> >> +max_lcores = []
> >> +
> >> +nCPU = os.cpu_count()
> >> +
> >> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
> >> +
> >> +print(' '.join(max_lcores))
> >> diff --git a/config/meson.build b/config/meson.build index
> >> 6996e5cbeaa5..80c05bc15d2f 100644
> >> --- a/config/meson.build
> >> +++ b/config/meson.build
> >> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller reserved memory
> areas
> >>        dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
> >>
> >> -
> >> compile_time_cpuflags = []
> >> subdir(arch_subdir)
> >> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
> >> ','.join(compile_time_cpuflags))
> >>
> >> +# set max lcores
> >> +if machine != 'default' and not meson.is_cross_build()
> >> +       # The script returns max lcores
> >> +       params = files('get_max_lcores.py')
> >> +       cmd_out = run_command(params)

Have you considered running just a shell command, such as "nproc --all"?

> >> +       if cmd_out.returncode() == 0
> >> +               cmd_lcore = cmd_out.stdout().to_lower().strip().split(' ')
> >> +       endif
> >> +       max_lcore = cmd_lcore[0].to_int()
> >> +       dpdk_conf.set('RTE_MAX_LCORE', max_lcore) endif
> >> +
> >> # set the install path for the drivers
> >> dpdk_conf.set_quoted('RTE_EAL_PMD_PATH', eal_pmd_path)
> >>
> >> --
> >> 2.17.1
> >>

Stephen Hemminger Sept. 3, 2020, 10:52 p.m. UTC | #4

On Thu, 3 Sep 2020 06:20:17 +0000
Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:

> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
> > Sent: Wednesday, August 26, 2020 6:56 AM
> > To: Jerin Jacob <jerinjacobk@gmail.com>
> > Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd <nd@arm.com>
> > Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
> > 
> > 
> >   
> > > On Aug 25, 2020, at 11:47 PM, Jerin Jacob <jerinjacobk@gmail.com> wrote:
> > >
> > > On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar  
> > <dharmik.thakkar@arm.com> wrote:  
> > >>
> > >> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to
> > >> incorrect RTE_MAX_LCORE when machines have same Implemener and part
> > >> number but different number of CPUs.
> > >> For x86, RTE_MAX_LCORE is always set to 128 (using the value set in
> > >> meson_options.txt)
> > >>
> > >> Use python script to find max lcore when using native build to
> > >> correctly set RTE_MAX_LCORE.  
> > >
> > > We may need to build on the native arm64 machine and use it on another
> > > arm64 machine(Just like x86).
> > > So I think, at least for default config(which will be used by
> > > distribution) to support max
> > > lcores as fixed. I am not sure this patch changes those aspects or
> > > not? Please check.  
> > 
> > This patch does *not* affect ‘default’ build type and cross-compilation.
> >   
> > >  
> > >>
> > >> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
> > >> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >> ---
> > >> config/get_max_lcores.py | 13 +++++++++++++
> > >> config/meson.build       | 13 ++++++++++++-
> > >> 2 files changed, 25 insertions(+), 1 deletion(-) create mode 100755
> > >> config/get_max_lcores.py
> > >>
> > >> diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py new
> > >> file mode 100755 index 000000000000..ebf1c7efdadd
> > >> --- /dev/null
> > >> +++ b/config/get_max_lcores.py
> > >> @@ -0,0 +1,13 @@
> > >> +#!/usr/bin/python3
> > >> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020 Arm
> > >> +Limited
> > >> +
> > >> +import os
> > >> +
> > >> +max_lcores = []
> > >> +
> > >> +nCPU = os.cpu_count()
> > >> +
> > >> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
> > >> +
> > >> +print(' '.join(max_lcores))
> > >> diff --git a/config/meson.build b/config/meson.build index
> > >> 6996e5cbeaa5..80c05bc15d2f 100644
> > >> --- a/config/meson.build
> > >> +++ b/config/meson.build
> > >> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller reserved memory  
> > areas  
> > >>        dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
> > >>
> > >> -
> > >> compile_time_cpuflags = []
> > >> subdir(arch_subdir)
> > >> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
> > >> ','.join(compile_time_cpuflags))
> > >>
> > >> +# set max lcores
> > >> +if machine != 'default' and not meson.is_cross_build()
> > >> +       # The script returns max lcores
> > >> +       params = files('get_max_lcores.py')
> > >> +       cmd_out = run_command(params)  
> 
> Have you considered running just a shell command, such as "nproc --all"?

Is this really a good idea?
For real distributions and NFV products, the build and runtime environment will usually be
different even if on same CPU architecture.

In many cases there maybe a huge build machine (128 CPU) or in a container (reported as single cpu)
even if not doing cross build.

Dharmik Thakkar Sept. 4, 2020, 5:26 a.m. UTC | #5

> On Sep 3, 2020, at 1:20 AM, Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
>> Sent: Wednesday, August 26, 2020 6:56 AM
>> To: Jerin Jacob <jerinjacobk@gmail.com>
>> Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd <nd@arm.com>
>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
>> 
>> 
>> 
>>> On Aug 25, 2020, at 11:47 PM, Jerin Jacob <jerinjacobk@gmail.com> wrote:
>>> 
>>> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar
>> <dharmik.thakkar@arm.com> wrote:
>>>> 
>>>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to
>>>> incorrect RTE_MAX_LCORE when machines have same Implemener and part
>>>> number but different number of CPUs.
>>>> For x86, RTE_MAX_LCORE is always set to 128 (using the value set in
>>>> meson_options.txt)
>>>> 
>>>> Use python script to find max lcore when using native build to
>>>> correctly set RTE_MAX_LCORE.
>>> 
>>> We may need to build on the native arm64 machine and use it on another
>>> arm64 machine(Just like x86).
>>> So I think, at least for default config(which will be used by
>>> distribution) to support max
>>> lcores as fixed. I am not sure this patch changes those aspects or
>>> not? Please check.
>> 
>> This patch does *not* affect ‘default’ build type and cross-compilation.
>> 
>>> 
>>>> 
>>>> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
>>>> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
>>>> ---
>>>> config/get_max_lcores.py | 13 +++++++++++++
>>>> config/meson.build       | 13 ++++++++++++-
>>>> 2 files changed, 25 insertions(+), 1 deletion(-) create mode 100755
>>>> config/get_max_lcores.py
>>>> 
>>>> diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py new
>>>> file mode 100755 index 000000000000..ebf1c7efdadd
>>>> --- /dev/null
>>>> +++ b/config/get_max_lcores.py
>>>> @@ -0,0 +1,13 @@
>>>> +#!/usr/bin/python3
>>>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020 Arm
>>>> +Limited
>>>> +
>>>> +import os
>>>> +
>>>> +max_lcores = []
>>>> +
>>>> +nCPU = os.cpu_count()
>>>> +
>>>> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
>>>> +
>>>> +print(' '.join(max_lcores))
>>>> diff --git a/config/meson.build b/config/meson.build index
>>>> 6996e5cbeaa5..80c05bc15d2f 100644
>>>> --- a/config/meson.build
>>>> +++ b/config/meson.build
>>>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller reserved memory
>> areas
>>>>       dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
>>>> 
>>>> -
>>>> compile_time_cpuflags = []
>>>> subdir(arch_subdir)
>>>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
>>>> ','.join(compile_time_cpuflags))
>>>> 
>>>> +# set max lcores
>>>> +if machine != 'default' and not meson.is_cross_build()
>>>> +       # The script returns max lcores
>>>> +       params = files('get_max_lcores.py')
>>>> +       cmd_out = run_command(params)
> 
> Have you considered running just a shell command, such as "nproc --all"?

Shell command based solution such as “nproc —all” is not OS-agnostic, it doesn’t work on Windows.

> 
>>>> +       if cmd_out.returncode() == 0
>>>> +               cmd_lcore = cmd_out.stdout().to_lower().strip().split(' ')
>>>> +       endif
>>>> +       max_lcore = cmd_lcore[0].to_int()
>>>> +       dpdk_conf.set('RTE_MAX_LCORE', max_lcore) endif
>>>> +
>>>> # set the install path for the drivers
>>>> dpdk_conf.set_quoted('RTE_EAL_PMD_PATH', eal_pmd_path)
>>>> 
>>>> --
>>>> 2.17.1

Dharmik Thakkar Sept. 4, 2020, 5:43 a.m. UTC | #6

> On Sep 3, 2020, at 5:52 PM, Stephen Hemminger <stephen@networkplumber.org> wrote:
> 
> On Thu, 3 Sep 2020 06:20:17 +0000
> Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> 
>>> -----Original Message-----
>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
>>> Sent: Wednesday, August 26, 2020 6:56 AM
>>> To: Jerin Jacob <jerinjacobk@gmail.com>
>>> Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd <nd@arm.com>
>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
>>> 
>>> 
>>> 
>>>> On Aug 25, 2020, at 11:47 PM, Jerin Jacob <jerinjacobk@gmail.com> wrote:
>>>> 
>>>> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar  
>>> <dharmik.thakkar@arm.com> wrote:  
>>>>> 
>>>>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to
>>>>> incorrect RTE_MAX_LCORE when machines have same Implemener and part
>>>>> number but different number of CPUs.
>>>>> For x86, RTE_MAX_LCORE is always set to 128 (using the value set in
>>>>> meson_options.txt)
>>>>> 
>>>>> Use python script to find max lcore when using native build to
>>>>> correctly set RTE_MAX_LCORE.  
>>>> 
>>>> We may need to build on the native arm64 machine and use it on another
>>>> arm64 machine(Just like x86).
>>>> So I think, at least for default config(which will be used by
>>>> distribution) to support max
>>>> lcores as fixed. I am not sure this patch changes those aspects or
>>>> not? Please check.  
>>> 
>>> This patch does *not* affect ‘default’ build type and cross-compilation.
>>> 
>>>> 
>>>>> 
>>>>> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
>>>>> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
>>>>> ---
>>>>> config/get_max_lcores.py | 13 +++++++++++++
>>>>> config/meson.build       | 13 ++++++++++++-
>>>>> 2 files changed, 25 insertions(+), 1 deletion(-) create mode 100755
>>>>> config/get_max_lcores.py
>>>>> 
>>>>> diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py new
>>>>> file mode 100755 index 000000000000..ebf1c7efdadd
>>>>> --- /dev/null
>>>>> +++ b/config/get_max_lcores.py
>>>>> @@ -0,0 +1,13 @@
>>>>> +#!/usr/bin/python3
>>>>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020 Arm
>>>>> +Limited
>>>>> +
>>>>> +import os
>>>>> +
>>>>> +max_lcores = []
>>>>> +
>>>>> +nCPU = os.cpu_count()
>>>>> +
>>>>> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
>>>>> +
>>>>> +print(' '.join(max_lcores))
>>>>> diff --git a/config/meson.build b/config/meson.build index
>>>>> 6996e5cbeaa5..80c05bc15d2f 100644
>>>>> --- a/config/meson.build
>>>>> +++ b/config/meson.build
>>>>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller reserved memory  
>>> areas  
>>>>>       dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
>>>>> 
>>>>> -
>>>>> compile_time_cpuflags = []
>>>>> subdir(arch_subdir)
>>>>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
>>>>> ','.join(compile_time_cpuflags))
>>>>> 
>>>>> +# set max lcores
>>>>> +if machine != 'default' and not meson.is_cross_build()
>>>>> +       # The script returns max lcores
>>>>> +       params = files('get_max_lcores.py')
>>>>> +       cmd_out = run_command(params)  
>> 
>> Have you considered running just a shell command, such as "nproc --all"?
> 
> Is this really a good idea?
> For real distributions and NFV products, the build and runtime environment will usually be
> different even if on same CPU architecture.
> 
> In many cases there maybe a huge build machine (128 CPU) or in a container (reported as single cpu)
> even if not doing cross build.

That’s a great point, Stephen. IMO, this patch is useful when building and running natively.
For all other purposes (like the ones you mentioned), do you think it is a good idea to set RTE_MAX_LCORE using -Dmax_lcores?

Juraj Linkeš Sept. 17, 2020, 9:56 a.m. UTC | #7

> -----Original Message-----
> From: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
> Sent: Friday, September 4, 2020 7:44 AM
> To: Stephen Hemminger <stephen@networkplumber.org>
> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; Jerin Jacob
> <jerinjacobk@gmail.com>; thomas@monjalon.net; dpdk-dev <dev@dpdk.org>;
> nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
> 
> 
> 
> > On Sep 3, 2020, at 5:52 PM, Stephen Hemminger
> <stephen@networkplumber.org> wrote:
> >
> > On Thu, 3 Sep 2020 06:20:17 +0000
> > Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> >
> >>> -----Original Message-----
> >>> From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
> >>> Sent: Wednesday, August 26, 2020 6:56 AM
> >>> To: Jerin Jacob <jerinjacobk@gmail.com>
> >>> Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd <nd@arm.com>
> >>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
> >>> programmatically
> >>>
> >>>
> >>>
> >>>> On Aug 25, 2020, at 11:47 PM, Jerin Jacob <jerinjacobk@gmail.com>
> wrote:
> >>>>
> >>>> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar
> >>> <dharmik.thakkar@arm.com> wrote:
> >>>>>
> >>>>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to
> >>>>> incorrect RTE_MAX_LCORE when machines have same Implemener and
> >>>>> part number but different number of CPUs.
> >>>>> For x86, RTE_MAX_LCORE is always set to 128 (using the value set
> >>>>> in
> >>>>> meson_options.txt)
> >>>>>
> >>>>> Use python script to find max lcore when using native build to
> >>>>> correctly set RTE_MAX_LCORE.
> >>>>
> >>>> We may need to build on the native arm64 machine and use it on
> >>>> another
> >>>> arm64 machine(Just like x86).
> >>>> So I think, at least for default config(which will be used by
> >>>> distribution) to support max
> >>>> lcores as fixed. I am not sure this patch changes those aspects or
> >>>> not? Please check.
> >>>
> >>> This patch does *not* affect ‘default’ build type and cross-compilation.
> >>>
> >>>>
> >>>>>
> >>>>> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
> >>>>> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >>>>> ---
> >>>>> config/get_max_lcores.py | 13 +++++++++++++
> >>>>> config/meson.build       | 13 ++++++++++++-
> >>>>> 2 files changed, 25 insertions(+), 1 deletion(-) create mode
> >>>>> 100755 config/get_max_lcores.py
> >>>>>
> >>>>> diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py
> >>>>> new file mode 100755 index 000000000000..ebf1c7efdadd
> >>>>> --- /dev/null
> >>>>> +++ b/config/get_max_lcores.py
> >>>>> @@ -0,0 +1,13 @@
> >>>>> +#!/usr/bin/python3
> >>>>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020 Arm
> >>>>> +Limited
> >>>>> +
> >>>>> +import os
> >>>>> +
> >>>>> +max_lcores = []
> >>>>> +
> >>>>> +nCPU = os.cpu_count()
> >>>>> +
> >>>>> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
> >>>>> +
> >>>>> +print(' '.join(max_lcores))
> >>>>> diff --git a/config/meson.build b/config/meson.build index
> >>>>> 6996e5cbeaa5..80c05bc15d2f 100644
> >>>>> --- a/config/meson.build
> >>>>> +++ b/config/meson.build
> >>>>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller reserved
> >>>>> memory
> >>> areas
> >>>>>       dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
> >>>>>
> >>>>> -
> >>>>> compile_time_cpuflags = []
> >>>>> subdir(arch_subdir)
> >>>>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
> >>>>> ','.join(compile_time_cpuflags))
> >>>>>
> >>>>> +# set max lcores
> >>>>> +if machine != 'default' and not meson.is_cross_build()
> >>>>> +       # The script returns max lcores
> >>>>> +       params = files('get_max_lcores.py')
> >>>>> +       cmd_out = run_command(params)
> >>
> >> Have you considered running just a shell command, such as "nproc --all"?
> >
> > Is this really a good idea?
> > For real distributions and NFV products, the build and runtime
> > environment will usually be different even if on same CPU architecture.
> >
> > In many cases there maybe a huge build machine (128 CPU) or in a
> > container (reported as single cpu) even if not doing cross build.
> 
> That’s a great point, Stephen. IMO, this patch is useful when building and
> running natively.
> For all other purposes (like the ones you mentioned), do you think it is a good
> idea to set RTE_MAX_LCORE using -Dmax_lcores?

We should only use this native builds, as that would be consistent with the current meson build philosophy of "meson figuring as much as possible on its own". Any build other than native implies the user wants to deviate from the build machine.

One way to do this automatic core count is when max_lcores=0 (0 would have the special meaning of "figure core count automatically"). We can set that as default in meson_option.txt and then users will have the ability to set it to whatever they want, even for native builds. What do you think?

Currently the -Dmax_lcores option doesn't work for arm builds (the value of RTE_MAX_LCORE is overwritten in config/arm/meson.build). I believe the patch tries to address this, but still, we need to be mindful of that.

Juraj

Dharmik Thakkar Sept. 18, 2020, 5:47 a.m. UTC | #8

> On Sep 17, 2020, at 4:56 AM, Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
>> Sent: Friday, September 4, 2020 7:44 AM
>> To: Stephen Hemminger <stephen@networkplumber.org>
>> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; Jerin Jacob
>> <jerinjacobk@gmail.com>; thomas@monjalon.net; dpdk-dev <dev@dpdk.org>;
>> nd <nd@arm.com>
>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
>> 
>> 
>> 
>>> On Sep 3, 2020, at 5:52 PM, Stephen Hemminger
>> <stephen@networkplumber.org> wrote:
>>> 
>>> On Thu, 3 Sep 2020 06:20:17 +0000
>>> Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>>> 
>>>>> -----Original Message-----
>>>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
>>>>> Sent: Wednesday, August 26, 2020 6:56 AM
>>>>> To: Jerin Jacob <jerinjacobk@gmail.com>
>>>>> Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd <nd@arm.com>
>>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
>>>>> programmatically
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Aug 25, 2020, at 11:47 PM, Jerin Jacob <jerinjacobk@gmail.com>
>> wrote:
>>>>>> 
>>>>>> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar
>>>>> <dharmik.thakkar@arm.com> wrote:
>>>>>>> 
>>>>>>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to
>>>>>>> incorrect RTE_MAX_LCORE when machines have same Implemener and
>>>>>>> part number but different number of CPUs.
>>>>>>> For x86, RTE_MAX_LCORE is always set to 128 (using the value set
>>>>>>> in
>>>>>>> meson_options.txt)
>>>>>>> 
>>>>>>> Use python script to find max lcore when using native build to
>>>>>>> correctly set RTE_MAX_LCORE.
>>>>>> 
>>>>>> We may need to build on the native arm64 machine and use it on
>>>>>> another
>>>>>> arm64 machine(Just like x86).
>>>>>> So I think, at least for default config(which will be used by
>>>>>> distribution) to support max
>>>>>> lcores as fixed. I am not sure this patch changes those aspects or
>>>>>> not? Please check.
>>>>> 
>>>>> This patch does *not* affect ‘default’ build type and cross-compilation.
>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
>>>>>>> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
>>>>>>> ---
>>>>>>> config/get_max_lcores.py | 13 +++++++++++++
>>>>>>> config/meson.build       | 13 ++++++++++++-
>>>>>>> 2 files changed, 25 insertions(+), 1 deletion(-) create mode
>>>>>>> 100755 config/get_max_lcores.py
>>>>>>> 
>>>>>>> diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py
>>>>>>> new file mode 100755 index 000000000000..ebf1c7efdadd
>>>>>>> --- /dev/null
>>>>>>> +++ b/config/get_max_lcores.py
>>>>>>> @@ -0,0 +1,13 @@
>>>>>>> +#!/usr/bin/python3
>>>>>>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020 Arm
>>>>>>> +Limited
>>>>>>> +
>>>>>>> +import os
>>>>>>> +
>>>>>>> +max_lcores = []
>>>>>>> +
>>>>>>> +nCPU = os.cpu_count()
>>>>>>> +
>>>>>>> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
>>>>>>> +
>>>>>>> +print(' '.join(max_lcores))
>>>>>>> diff --git a/config/meson.build b/config/meson.build index
>>>>>>> 6996e5cbeaa5..80c05bc15d2f 100644
>>>>>>> --- a/config/meson.build
>>>>>>> +++ b/config/meson.build
>>>>>>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller reserved
>>>>>>> memory
>>>>> areas
>>>>>>>      dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
>>>>>>> 
>>>>>>> -
>>>>>>> compile_time_cpuflags = []
>>>>>>> subdir(arch_subdir)
>>>>>>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
>>>>>>> ','.join(compile_time_cpuflags))
>>>>>>> 
>>>>>>> +# set max lcores
>>>>>>> +if machine != 'default' and not meson.is_cross_build()
>>>>>>> +       # The script returns max lcores
>>>>>>> +       params = files('get_max_lcores.py')
>>>>>>> +       cmd_out = run_command(params)
>>>> 
>>>> Have you considered running just a shell command, such as "nproc --all"?
>>> 
>>> Is this really a good idea?
>>> For real distributions and NFV products, the build and runtime
>>> environment will usually be different even if on same CPU architecture.
>>> 
>>> In many cases there maybe a huge build machine (128 CPU) or in a
>>> container (reported as single cpu) even if not doing cross build.
>> 
>> That’s a great point, Stephen. IMO, this patch is useful when building and
>> running natively.
>> For all other purposes (like the ones you mentioned), do you think it is a good
>> idea to set RTE_MAX_LCORE using -Dmax_lcores?
> 
> We should only use this native builds, as that would be consistent with the current meson build philosophy of "meson figuring as much as possible on its own". Any build other than native implies the user wants to deviate from the build machine.
> 

The MIDR value-based probing doesn’t quite work well for Arm IP (currently being discussed on this patch: https://patches.dpdk.org/patch/76981/).

> One way to do this automatic core count is when max_lcores=0 (0 would have the special meaning of "figure core count automatically"). We can set that as default in meson_option.txt and then users will have the ability to set it to whatever they want, even for native builds. What do you think?
> 

Yes, agreed.

> Currently the -Dmax_lcores option doesn't work for arm builds (the value of RTE_MAX_LCORE is overwritten in config/arm/meson.build). I believe the patch tries to address this, but still, we need to be mindful of that.
> 
> Juraj

Thomas Monjalon Oct. 13, 2020, 2:31 p.m. UTC | #9

Please, what is the conclusion here?


18/09/2020 07:47, Dharmik Thakkar:
> 
> > On Sep 17, 2020, at 4:56 AM, Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> > 
> > 
> > 
> >> -----Original Message-----
> >> From: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
> >> Sent: Friday, September 4, 2020 7:44 AM
> >> To: Stephen Hemminger <stephen@networkplumber.org>
> >> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; Jerin Jacob
> >> <jerinjacobk@gmail.com>; thomas@monjalon.net; dpdk-dev <dev@dpdk.org>;
> >> nd <nd@arm.com>
> >> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
> >> 
> >> 
> >> 
> >>> On Sep 3, 2020, at 5:52 PM, Stephen Hemminger
> >> <stephen@networkplumber.org> wrote:
> >>> 
> >>> On Thu, 3 Sep 2020 06:20:17 +0000
> >>> Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> >>> 
> >>>>> -----Original Message-----
> >>>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
> >>>>> Sent: Wednesday, August 26, 2020 6:56 AM
> >>>>> To: Jerin Jacob <jerinjacobk@gmail.com>
> >>>>> Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd <nd@arm.com>
> >>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
> >>>>> programmatically
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>>> On Aug 25, 2020, at 11:47 PM, Jerin Jacob <jerinjacobk@gmail.com>
> >> wrote:
> >>>>>> 
> >>>>>> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar
> >>>>> <dharmik.thakkar@arm.com> wrote:
> >>>>>>> 
> >>>>>>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to
> >>>>>>> incorrect RTE_MAX_LCORE when machines have same Implemener and
> >>>>>>> part number but different number of CPUs.
> >>>>>>> For x86, RTE_MAX_LCORE is always set to 128 (using the value set
> >>>>>>> in
> >>>>>>> meson_options.txt)
> >>>>>>> 
> >>>>>>> Use python script to find max lcore when using native build to
> >>>>>>> correctly set RTE_MAX_LCORE.
> >>>>>> 
> >>>>>> We may need to build on the native arm64 machine and use it on
> >>>>>> another
> >>>>>> arm64 machine(Just like x86).
> >>>>>> So I think, at least for default config(which will be used by
> >>>>>> distribution) to support max
> >>>>>> lcores as fixed. I am not sure this patch changes those aspects or
> >>>>>> not? Please check.
> >>>>> 
> >>>>> This patch does *not* affect ‘default’ build type and cross-compilation.
> >>>>> 
> >>>>>> 
> >>>>>>> 
> >>>>>>> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
> >>>>>>> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >>>>>>> ---
> >>>>>>> config/get_max_lcores.py | 13 +++++++++++++
> >>>>>>> config/meson.build       | 13 ++++++++++++-
> >>>>>>> 2 files changed, 25 insertions(+), 1 deletion(-) create mode
> >>>>>>> 100755 config/get_max_lcores.py
> >>>>>>> 
> >>>>>>> diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py
> >>>>>>> new file mode 100755 index 000000000000..ebf1c7efdadd
> >>>>>>> --- /dev/null
> >>>>>>> +++ b/config/get_max_lcores.py
> >>>>>>> @@ -0,0 +1,13 @@
> >>>>>>> +#!/usr/bin/python3
> >>>>>>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020 Arm
> >>>>>>> +Limited
> >>>>>>> +
> >>>>>>> +import os
> >>>>>>> +
> >>>>>>> +max_lcores = []
> >>>>>>> +
> >>>>>>> +nCPU = os.cpu_count()
> >>>>>>> +
> >>>>>>> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
> >>>>>>> +
> >>>>>>> +print(' '.join(max_lcores))
> >>>>>>> diff --git a/config/meson.build b/config/meson.build index
> >>>>>>> 6996e5cbeaa5..80c05bc15d2f 100644
> >>>>>>> --- a/config/meson.build
> >>>>>>> +++ b/config/meson.build
> >>>>>>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller reserved
> >>>>>>> memory
> >>>>> areas
> >>>>>>>      dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
> >>>>>>> 
> >>>>>>> -
> >>>>>>> compile_time_cpuflags = []
> >>>>>>> subdir(arch_subdir)
> >>>>>>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
> >>>>>>> ','.join(compile_time_cpuflags))
> >>>>>>> 
> >>>>>>> +# set max lcores
> >>>>>>> +if machine != 'default' and not meson.is_cross_build()
> >>>>>>> +       # The script returns max lcores
> >>>>>>> +       params = files('get_max_lcores.py')
> >>>>>>> +       cmd_out = run_command(params)
> >>>> 
> >>>> Have you considered running just a shell command, such as "nproc --all"?
> >>> 
> >>> Is this really a good idea?
> >>> For real distributions and NFV products, the build and runtime
> >>> environment will usually be different even if on same CPU architecture.
> >>> 
> >>> In many cases there maybe a huge build machine (128 CPU) or in a
> >>> container (reported as single cpu) even if not doing cross build.
> >> 
> >> That’s a great point, Stephen. IMO, this patch is useful when building and
> >> running natively.
> >> For all other purposes (like the ones you mentioned), do you think it is a good
> >> idea to set RTE_MAX_LCORE using -Dmax_lcores?
> > 
> > We should only use this native builds, as that would be consistent with the current meson build philosophy of "meson figuring as much as possible on its own". Any build other than native implies the user wants to deviate from the build machine.
> > 
> 
> The MIDR value-based probing doesn’t quite work well for Arm IP (currently being discussed on this patch: https://patches.dpdk.org/patch/76981/).
> 
> > One way to do this automatic core count is when max_lcores=0 (0 would have the special meaning of "figure core count automatically"). We can set that as default in meson_option.txt and then users will have the ability to set it to whatever they want, even for native builds. What do you think?
> > 
> 
> Yes, agreed.
> 
> > Currently the -Dmax_lcores option doesn't work for arm builds (the value of RTE_MAX_LCORE is overwritten in config/arm/meson.build). I believe the patch tries to address this, but still, we need to be mindful of that.
> > 
> > Juraj

Juraj Linkeš Oct. 13, 2020, 2:58 p.m. UTC | #10

I believe we're going to drop this patch series in favor of http://patches.dpdk.org/project/dpdk/list/?series=12923. 

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, October 13, 2020 4:32 PM
> To: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; dev@dpdk.org; Stephen
> Hemminger <stephen@networkplumber.org>; Jerin Jacob
> <jerinjacobk@gmail.com>; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
> 
> Please, what is the conclusion here?
> 
> 
> 18/09/2020 07:47, Dharmik Thakkar:
> >
> > > On Sep 17, 2020, at 4:56 AM, Juraj Linkeš <juraj.linkes@pantheon.tech>
> wrote:
> > >
> > >
> > >
> > >> -----Original Message-----
> > >> From: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
> > >> Sent: Friday, September 4, 2020 7:44 AM
> > >> To: Stephen Hemminger <stephen@networkplumber.org>
> > >> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; Jerin Jacob
> > >> <jerinjacobk@gmail.com>; thomas@monjalon.net; dpdk-dev
> > >> <dev@dpdk.org>; nd <nd@arm.com>
> > >> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
> > >> programmatically
> > >>
> > >>
> > >>
> > >>> On Sep 3, 2020, at 5:52 PM, Stephen Hemminger
> > >> <stephen@networkplumber.org> wrote:
> > >>>
> > >>> On Thu, 3 Sep 2020 06:20:17 +0000
> > >>> Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> > >>>
> > >>>>> -----Original Message-----
> > >>>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
> > >>>>> Sent: Wednesday, August 26, 2020 6:56 AM
> > >>>>> To: Jerin Jacob <jerinjacobk@gmail.com>
> > >>>>> Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd
> > >>>>> <nd@arm.com>
> > >>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
> > >>>>> programmatically
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>> On Aug 25, 2020, at 11:47 PM, Jerin Jacob
> > >>>>>> <jerinjacobk@gmail.com>
> > >> wrote:
> > >>>>>>
> > >>>>>> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar
> > >>>>> <dharmik.thakkar@arm.com> wrote:
> > >>>>>>>
> > >>>>>>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads
> > >>>>>>> to incorrect RTE_MAX_LCORE when machines have same
> Implemener
> > >>>>>>> and part number but different number of CPUs.
> > >>>>>>> For x86, RTE_MAX_LCORE is always set to 128 (using the value
> > >>>>>>> set in
> > >>>>>>> meson_options.txt)
> > >>>>>>>
> > >>>>>>> Use python script to find max lcore when using native build to
> > >>>>>>> correctly set RTE_MAX_LCORE.
> > >>>>>>
> > >>>>>> We may need to build on the native arm64 machine and use it on
> > >>>>>> another
> > >>>>>> arm64 machine(Just like x86).
> > >>>>>> So I think, at least for default config(which will be used by
> > >>>>>> distribution) to support max
> > >>>>>> lcores as fixed. I am not sure this patch changes those aspects
> > >>>>>> or not? Please check.
> > >>>>>
> > >>>>> This patch does *not* affect ‘default’ build type and cross-compilation.
> > >>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
> > >>>>>>> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >>>>>>> ---
> > >>>>>>> config/get_max_lcores.py | 13 +++++++++++++
> > >>>>>>> config/meson.build       | 13 ++++++++++++-
> > >>>>>>> 2 files changed, 25 insertions(+), 1 deletion(-) create mode
> > >>>>>>> 100755 config/get_max_lcores.py
> > >>>>>>>
> > >>>>>>> diff --git a/config/get_max_lcores.py
> > >>>>>>> b/config/get_max_lcores.py new file mode 100755 index
> > >>>>>>> 000000000000..ebf1c7efdadd
> > >>>>>>> --- /dev/null
> > >>>>>>> +++ b/config/get_max_lcores.py
> > >>>>>>> @@ -0,0 +1,13 @@
> > >>>>>>> +#!/usr/bin/python3
> > >>>>>>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020
> > >>>>>>> +Arm Limited
> > >>>>>>> +
> > >>>>>>> +import os
> > >>>>>>> +
> > >>>>>>> +max_lcores = []
> > >>>>>>> +
> > >>>>>>> +nCPU = os.cpu_count()
> > >>>>>>> +
> > >>>>>>> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
> > >>>>>>> +
> > >>>>>>> +print(' '.join(max_lcores))
> > >>>>>>> diff --git a/config/meson.build b/config/meson.build index
> > >>>>>>> 6996e5cbeaa5..80c05bc15d2f 100644
> > >>>>>>> --- a/config/meson.build
> > >>>>>>> +++ b/config/meson.build
> > >>>>>>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller
> > >>>>>>> reserved memory
> > >>>>> areas
> > >>>>>>>      dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
> > >>>>>>>
> > >>>>>>> -
> > >>>>>>> compile_time_cpuflags = []
> > >>>>>>> subdir(arch_subdir)
> > >>>>>>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
> > >>>>>>> ','.join(compile_time_cpuflags))
> > >>>>>>>
> > >>>>>>> +# set max lcores
> > >>>>>>> +if machine != 'default' and not meson.is_cross_build()
> > >>>>>>> +       # The script returns max lcores
> > >>>>>>> +       params = files('get_max_lcores.py')
> > >>>>>>> +       cmd_out = run_command(params)
> > >>>>
> > >>>> Have you considered running just a shell command, such as "nproc --all"?
> > >>>
> > >>> Is this really a good idea?
> > >>> For real distributions and NFV products, the build and runtime
> > >>> environment will usually be different even if on same CPU architecture.
> > >>>
> > >>> In many cases there maybe a huge build machine (128 CPU) or in a
> > >>> container (reported as single cpu) even if not doing cross build.
> > >>
> > >> That’s a great point, Stephen. IMO, this patch is useful when
> > >> building and running natively.
> > >> For all other purposes (like the ones you mentioned), do you think
> > >> it is a good idea to set RTE_MAX_LCORE using -Dmax_lcores?
> > >
> > > We should only use this native builds, as that would be consistent with the
> current meson build philosophy of "meson figuring as much as possible on its
> own". Any build other than native implies the user wants to deviate from the
> build machine.
> > >
> >
> > The MIDR value-based probing doesn’t quite work well for Arm IP (currently
> being discussed on this patch: https://patches.dpdk.org/patch/76981/).
> >
> > > One way to do this automatic core count is when max_lcores=0 (0 would
> have the special meaning of "figure core count automatically"). We can set that
> as default in meson_option.txt and then users will have the ability to set it to
> whatever they want, even for native builds. What do you think?
> > >
> >
> > Yes, agreed.
> >
> > > Currently the -Dmax_lcores option doesn't work for arm builds (the value of
> RTE_MAX_LCORE is overwritten in config/arm/meson.build). I believe the patch
> tries to address this, but still, we need to be mindful of that.
> > >
> > > Juraj
> 
>

Dharmik Thakkar Oct. 13, 2020, 3:14 p.m. UTC | #11

Hi Juraj,

> On Oct 13, 2020, at 9:58 AM, Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> 
> I believe we're going to drop this patch series in favor of http://patches.dpdk.org/project/dpdk/list/?series=12923.

I can see you have included this feature in your series. Thank you!
What are your thoughts on the other patch [1]? Do you plan on including that as well in your series?

[1] 	[1/2] config/arm: avoid variable reuse
https://patches.dpdk.org/patch/75946/

> 
>> -----Original Message-----
>> From: Thomas Monjalon <thomas@monjalon.net>
>> Sent: Tuesday, October 13, 2020 4:32 PM
>> To: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
>> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; dev@dpdk.org; Stephen
>> Hemminger <stephen@networkplumber.org>; Jerin Jacob
>> <jerinjacobk@gmail.com>; nd <nd@arm.com>
>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
>> 
>> Please, what is the conclusion here?
>> 
>> 
>> 18/09/2020 07:47, Dharmik Thakkar:
>>> 
>>>> On Sep 17, 2020, at 4:56 AM, Juraj Linkeš <juraj.linkes@pantheon.tech>
>> wrote:
>>>> 
>>>> 
>>>> 
>>>>> -----Original Message-----
>>>>> From: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
>>>>> Sent: Friday, September 4, 2020 7:44 AM
>>>>> To: Stephen Hemminger <stephen@networkplumber.org>
>>>>> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; Jerin Jacob
>>>>> <jerinjacobk@gmail.com>; thomas@monjalon.net; dpdk-dev
>>>>> <dev@dpdk.org>; nd <nd@arm.com>
>>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
>>>>> programmatically
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Sep 3, 2020, at 5:52 PM, Stephen Hemminger
>>>>> <stephen@networkplumber.org> wrote:
>>>>>> 
>>>>>> On Thu, 3 Sep 2020 06:20:17 +0000
>>>>>> Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>>>>>> 
>>>>>>>> -----Original Message-----
>>>>>>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
>>>>>>>> Sent: Wednesday, August 26, 2020 6:56 AM
>>>>>>>> To: Jerin Jacob <jerinjacobk@gmail.com>
>>>>>>>> Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd
>>>>>>>> <nd@arm.com>
>>>>>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
>>>>>>>> programmatically
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On Aug 25, 2020, at 11:47 PM, Jerin Jacob
>>>>>>>>> <jerinjacobk@gmail.com>
>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar
>>>>>>>> <dharmik.thakkar@arm.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads
>>>>>>>>>> to incorrect RTE_MAX_LCORE when machines have same
>> Implemener
>>>>>>>>>> and part number but different number of CPUs.
>>>>>>>>>> For x86, RTE_MAX_LCORE is always set to 128 (using the value
>>>>>>>>>> set in
>>>>>>>>>> meson_options.txt)
>>>>>>>>>> 
>>>>>>>>>> Use python script to find max lcore when using native build to
>>>>>>>>>> correctly set RTE_MAX_LCORE.
>>>>>>>>> 
>>>>>>>>> We may need to build on the native arm64 machine and use it on
>>>>>>>>> another
>>>>>>>>> arm64 machine(Just like x86).
>>>>>>>>> So I think, at least for default config(which will be used by
>>>>>>>>> distribution) to support max
>>>>>>>>> lcores as fixed. I am not sure this patch changes those aspects
>>>>>>>>> or not? Please check.
>>>>>>>> 
>>>>>>>> This patch does *not* affect ‘default’ build type and cross-compilation.
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
>>>>>>>>>> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
>>>>>>>>>> ---
>>>>>>>>>> config/get_max_lcores.py | 13 +++++++++++++
>>>>>>>>>> config/meson.build       | 13 ++++++++++++-
>>>>>>>>>> 2 files changed, 25 insertions(+), 1 deletion(-) create mode
>>>>>>>>>> 100755 config/get_max_lcores.py
>>>>>>>>>> 
>>>>>>>>>> diff --git a/config/get_max_lcores.py
>>>>>>>>>> b/config/get_max_lcores.py new file mode 100755 index
>>>>>>>>>> 000000000000..ebf1c7efdadd
>>>>>>>>>> --- /dev/null
>>>>>>>>>> +++ b/config/get_max_lcores.py
>>>>>>>>>> @@ -0,0 +1,13 @@
>>>>>>>>>> +#!/usr/bin/python3
>>>>>>>>>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020
>>>>>>>>>> +Arm Limited
>>>>>>>>>> +
>>>>>>>>>> +import os
>>>>>>>>>> +
>>>>>>>>>> +max_lcores = []
>>>>>>>>>> +
>>>>>>>>>> +nCPU = os.cpu_count()
>>>>>>>>>> +
>>>>>>>>>> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
>>>>>>>>>> +
>>>>>>>>>> +print(' '.join(max_lcores))
>>>>>>>>>> diff --git a/config/meson.build b/config/meson.build index
>>>>>>>>>> 6996e5cbeaa5..80c05bc15d2f 100644
>>>>>>>>>> --- a/config/meson.build
>>>>>>>>>> +++ b/config/meson.build
>>>>>>>>>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller
>>>>>>>>>> reserved memory
>>>>>>>> areas
>>>>>>>>>>     dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
>>>>>>>>>> 
>>>>>>>>>> -
>>>>>>>>>> compile_time_cpuflags = []
>>>>>>>>>> subdir(arch_subdir)
>>>>>>>>>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
>>>>>>>>>> ','.join(compile_time_cpuflags))
>>>>>>>>>> 
>>>>>>>>>> +# set max lcores
>>>>>>>>>> +if machine != 'default' and not meson.is_cross_build()
>>>>>>>>>> +       # The script returns max lcores
>>>>>>>>>> +       params = files('get_max_lcores.py')
>>>>>>>>>> +       cmd_out = run_command(params)
>>>>>>> 
>>>>>>> Have you considered running just a shell command, such as "nproc --all"?
>>>>>> 
>>>>>> Is this really a good idea?
>>>>>> For real distributions and NFV products, the build and runtime
>>>>>> environment will usually be different even if on same CPU architecture.
>>>>>> 
>>>>>> In many cases there maybe a huge build machine (128 CPU) or in a
>>>>>> container (reported as single cpu) even if not doing cross build.
>>>>> 
>>>>> That’s a great point, Stephen. IMO, this patch is useful when
>>>>> building and running natively.
>>>>> For all other purposes (like the ones you mentioned), do you think
>>>>> it is a good idea to set RTE_MAX_LCORE using -Dmax_lcores?
>>>> 
>>>> We should only use this native builds, as that would be consistent with the
>> current meson build philosophy of "meson figuring as much as possible on its
>> own". Any build other than native implies the user wants to deviate from the
>> build machine.
>>>> 
>>> 
>>> The MIDR value-based probing doesn’t quite work well for Arm IP (currently
>> being discussed on this patch: https://patches.dpdk.org/patch/76981/).
>>> 
>>>> One way to do this automatic core count is when max_lcores=0 (0 would
>> have the special meaning of "figure core count automatically"). We can set that
>> as default in meson_option.txt and then users will have the ability to set it to
>> whatever they want, even for native builds. What do you think?
>>>> 
>>> 
>>> Yes, agreed.
>>> 
>>>> Currently the -Dmax_lcores option doesn't work for arm builds (the value of
>> RTE_MAX_LCORE is overwritten in config/arm/meson.build). I believe the patch
>> tries to address this, but still, we need to be mindful of that.
>>>> 
>>>> Juraj
>> 
>> 
>

Juraj Linkeš Oct. 14, 2020, 6:53 a.m. UTC | #12

Hi Dharmik,

> -----Original Message-----
> From: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
> Sent: Tuesday, October 13, 2020 5:15 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: thomas@monjalon.net; dev@dpdk.org; Stephen Hemminger
> <stephen@networkplumber.org>; Jerin Jacob <jerinjacobk@gmail.com>; nd
> <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
> 
> Hi Juraj,
> 
> > On Oct 13, 2020, at 9:58 AM, Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> >
> > I believe we're going to drop this patch series in favor of
> http://patches.dpdk.org/project/dpdk/list/?series=12923.
> 
> I can see you have included this feature in your series. Thank you!
> What are your thoughts on the other patch [1]? Do you plan on including that as
> well in your series?
> 
> [1] 	[1/2] config/arm: avoid variable reuse
> https://patches.dpdk.org/patch/75946/
> 

I believe the general idea of your patch is alredy part of my patch series in this patch: http://patches.dpdk.org/patch/80572/

> >
> >> -----Original Message-----
> >> From: Thomas Monjalon <thomas@monjalon.net>
> >> Sent: Tuesday, October 13, 2020 4:32 PM
> >> To: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
> >> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; dev@dpdk.org; Stephen
> >> Hemminger <stephen@networkplumber.org>; Jerin Jacob
> >> <jerinjacobk@gmail.com>; nd <nd@arm.com>
> >> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
> >> programmatically
> >>
> >> Please, what is the conclusion here?
> >>
> >>
> >> 18/09/2020 07:47, Dharmik Thakkar:
> >>>
> >>>> On Sep 17, 2020, at 4:56 AM, Juraj Linkeš
> >>>> <juraj.linkes@pantheon.tech>
> >> wrote:
> >>>>
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
> >>>>> Sent: Friday, September 4, 2020 7:44 AM
> >>>>> To: Stephen Hemminger <stephen@networkplumber.org>
> >>>>> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; Jerin Jacob
> >>>>> <jerinjacobk@gmail.com>; thomas@monjalon.net; dpdk-dev
> >>>>> <dev@dpdk.org>; nd <nd@arm.com>
> >>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
> >>>>> programmatically
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On Sep 3, 2020, at 5:52 PM, Stephen Hemminger
> >>>>> <stephen@networkplumber.org> wrote:
> >>>>>>
> >>>>>> On Thu, 3 Sep 2020 06:20:17 +0000 Juraj Linkeš
> >>>>>> <juraj.linkes@pantheon.tech> wrote:
> >>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
> >>>>>>>> Sent: Wednesday, August 26, 2020 6:56 AM
> >>>>>>>> To: Jerin Jacob <jerinjacobk@gmail.com>
> >>>>>>>> Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd
> >>>>>>>> <nd@arm.com>
> >>>>>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
> >>>>>>>> programmatically
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Aug 25, 2020, at 11:47 PM, Jerin Jacob
> >>>>>>>>> <jerinjacobk@gmail.com>
> >>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar
> >>>>>>>> <dharmik.thakkar@arm.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It
> >>>>>>>>>> leads to incorrect RTE_MAX_LCORE when machines have same
> >> Implemener
> >>>>>>>>>> and part number but different number of CPUs.
> >>>>>>>>>> For x86, RTE_MAX_LCORE is always set to 128 (using the value
> >>>>>>>>>> set in
> >>>>>>>>>> meson_options.txt)
> >>>>>>>>>>
> >>>>>>>>>> Use python script to find max lcore when using native build
> >>>>>>>>>> to correctly set RTE_MAX_LCORE.
> >>>>>>>>>
> >>>>>>>>> We may need to build on the native arm64 machine and use it on
> >>>>>>>>> another
> >>>>>>>>> arm64 machine(Just like x86).
> >>>>>>>>> So I think, at least for default config(which will be used by
> >>>>>>>>> distribution) to support max
> >>>>>>>>> lcores as fixed. I am not sure this patch changes those
> >>>>>>>>> aspects or not? Please check.
> >>>>>>>>
> >>>>>>>> This patch does *not* affect ‘default’ build type and cross-
> compilation.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
> >>>>>>>>>> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >>>>>>>>>> ---
> >>>>>>>>>> config/get_max_lcores.py | 13 +++++++++++++
> >>>>>>>>>> config/meson.build       | 13 ++++++++++++-
> >>>>>>>>>> 2 files changed, 25 insertions(+), 1 deletion(-) create mode
> >>>>>>>>>> 100755 config/get_max_lcores.py
> >>>>>>>>>>
> >>>>>>>>>> diff --git a/config/get_max_lcores.py
> >>>>>>>>>> b/config/get_max_lcores.py new file mode 100755 index
> >>>>>>>>>> 000000000000..ebf1c7efdadd
> >>>>>>>>>> --- /dev/null
> >>>>>>>>>> +++ b/config/get_max_lcores.py
> >>>>>>>>>> @@ -0,0 +1,13 @@
> >>>>>>>>>> +#!/usr/bin/python3
> >>>>>>>>>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020
> >>>>>>>>>> +Arm Limited
> >>>>>>>>>> +
> >>>>>>>>>> +import os
> >>>>>>>>>> +
> >>>>>>>>>> +max_lcores = []
> >>>>>>>>>> +
> >>>>>>>>>> +nCPU = os.cpu_count()
> >>>>>>>>>> +
> >>>>>>>>>> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
> >>>>>>>>>> +
> >>>>>>>>>> +print(' '.join(max_lcores))
> >>>>>>>>>> diff --git a/config/meson.build b/config/meson.build index
> >>>>>>>>>> 6996e5cbeaa5..80c05bc15d2f 100644
> >>>>>>>>>> --- a/config/meson.build
> >>>>>>>>>> +++ b/config/meson.build
> >>>>>>>>>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller
> >>>>>>>>>> reserved memory
> >>>>>>>> areas
> >>>>>>>>>>     dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
> >>>>>>>>>>
> >>>>>>>>>> -
> >>>>>>>>>> compile_time_cpuflags = []
> >>>>>>>>>> subdir(arch_subdir)
> >>>>>>>>>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
> >>>>>>>>>> ','.join(compile_time_cpuflags))
> >>>>>>>>>>
> >>>>>>>>>> +# set max lcores
> >>>>>>>>>> +if machine != 'default' and not meson.is_cross_build()
> >>>>>>>>>> +       # The script returns max lcores
> >>>>>>>>>> +       params = files('get_max_lcores.py')
> >>>>>>>>>> +       cmd_out = run_command(params)
> >>>>>>>
> >>>>>>> Have you considered running just a shell command, such as "nproc --
> all"?
> >>>>>>
> >>>>>> Is this really a good idea?
> >>>>>> For real distributions and NFV products, the build and runtime
> >>>>>> environment will usually be different even if on same CPU architecture.
> >>>>>>
> >>>>>> In many cases there maybe a huge build machine (128 CPU) or in a
> >>>>>> container (reported as single cpu) even if not doing cross build.
> >>>>>
> >>>>> That’s a great point, Stephen. IMO, this patch is useful when
> >>>>> building and running natively.
> >>>>> For all other purposes (like the ones you mentioned), do you think
> >>>>> it is a good idea to set RTE_MAX_LCORE using -Dmax_lcores?
> >>>>
> >>>> We should only use this native builds, as that would be consistent
> >>>> with the
> >> current meson build philosophy of "meson figuring as much as possible
> >> on its own". Any build other than native implies the user wants to
> >> deviate from the build machine.
> >>>>
> >>>
> >>> The MIDR value-based probing doesn’t quite work well for Arm IP
> >>> (currently
> >> being discussed on this patch: https://patches.dpdk.org/patch/76981/).
> >>>
> >>>> One way to do this automatic core count is when max_lcores=0 (0
> >>>> would
> >> have the special meaning of "figure core count automatically"). We
> >> can set that as default in meson_option.txt and then users will have
> >> the ability to set it to whatever they want, even for native builds. What do
> you think?
> >>>>
> >>>
> >>> Yes, agreed.
> >>>
> >>>> Currently the -Dmax_lcores option doesn't work for arm builds (the
> >>>> value of
> >> RTE_MAX_LCORE is overwritten in config/arm/meson.build). I believe
> >> the patch tries to address this, but still, we need to be mindful of that.
> >>>>
> >>>> Juraj
> >>
> >>
> >

Dharmik Thakkar Oct. 14, 2020, 1:28 p.m. UTC | #13

> On Oct 14, 2020, at 1:53 AM, Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> 
> Hi Dharmik,
> 
>> -----Original Message-----
>> From: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
>> Sent: Tuesday, October 13, 2020 5:15 PM
>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
>> Cc: thomas@monjalon.net; dev@dpdk.org; Stephen Hemminger
>> <stephen@networkplumber.org>; Jerin Jacob <jerinjacobk@gmail.com>; nd
>> <nd@arm.com>
>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically
>> 
>> Hi Juraj,
>> 
>>> On Oct 13, 2020, at 9:58 AM, Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>>> 
>>> I believe we're going to drop this patch series in favor of
>> http://patches.dpdk.org/project/dpdk/list/?series=12923.
>> 
>> I can see you have included this feature in your series. Thank you!
>> What are your thoughts on the other patch [1]? Do you plan on including that as
>> well in your series?
>> 
>> [1] 	[1/2] config/arm: avoid variable reuse
>> https://patches.dpdk.org/patch/75946/
>> 
> 
> I believe the general idea of your patch is alredy part of my patch series in this patch: http://patches.dpdk.org/patch/80572/

Great, thank you! I will drop these patches.

> 
>>> 
>>>> -----Original Message-----
>>>> From: Thomas Monjalon <thomas@monjalon.net>
>>>> Sent: Tuesday, October 13, 2020 4:32 PM
>>>> To: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
>>>> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; dev@dpdk.org; Stephen
>>>> Hemminger <stephen@networkplumber.org>; Jerin Jacob
>>>> <jerinjacobk@gmail.com>; nd <nd@arm.com>
>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
>>>> programmatically
>>>> 
>>>> Please, what is the conclusion here?
>>>> 
>>>> 
>>>> 18/09/2020 07:47, Dharmik Thakkar:
>>>>> 
>>>>>> On Sep 17, 2020, at 4:56 AM, Juraj Linkeš
>>>>>> <juraj.linkes@pantheon.tech>
>>>> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Dharmik Thakkar <Dharmik.Thakkar@arm.com>
>>>>>>> Sent: Friday, September 4, 2020 7:44 AM
>>>>>>> To: Stephen Hemminger <stephen@networkplumber.org>
>>>>>>> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; Jerin Jacob
>>>>>>> <jerinjacobk@gmail.com>; thomas@monjalon.net; dpdk-dev
>>>>>>> <dev@dpdk.org>; nd <nd@arm.com>
>>>>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
>>>>>>> programmatically
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> On Sep 3, 2020, at 5:52 PM, Stephen Hemminger
>>>>>>> <stephen@networkplumber.org> wrote:
>>>>>>>> 
>>>>>>>> On Thu, 3 Sep 2020 06:20:17 +0000 Juraj Linkeš
>>>>>>>> <juraj.linkes@pantheon.tech> wrote:
>>>>>>>> 
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Dharmik Thakkar
>>>>>>>>>> Sent: Wednesday, August 26, 2020 6:56 AM
>>>>>>>>>> To: Jerin Jacob <jerinjacobk@gmail.com>
>>>>>>>>>> Cc: thomas@monjalon.net; dpdk-dev <dev@dpdk.org>; nd
>>>>>>>>>> <nd@arm.com>
>>>>>>>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore
>>>>>>>>>> programmatically
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> On Aug 25, 2020, at 11:47 PM, Jerin Jacob
>>>>>>>>>>> <jerinjacobk@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar
>>>>>>>>>> <dharmik.thakkar@arm.com> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It
>>>>>>>>>>>> leads to incorrect RTE_MAX_LCORE when machines have same
>>>> Implemener
>>>>>>>>>>>> and part number but different number of CPUs.
>>>>>>>>>>>> For x86, RTE_MAX_LCORE is always set to 128 (using the value
>>>>>>>>>>>> set in
>>>>>>>>>>>> meson_options.txt)
>>>>>>>>>>>> 
>>>>>>>>>>>> Use python script to find max lcore when using native build
>>>>>>>>>>>> to correctly set RTE_MAX_LCORE.
>>>>>>>>>>> 
>>>>>>>>>>> We may need to build on the native arm64 machine and use it on
>>>>>>>>>>> another
>>>>>>>>>>> arm64 machine(Just like x86).
>>>>>>>>>>> So I think, at least for default config(which will be used by
>>>>>>>>>>> distribution) to support max
>>>>>>>>>>> lcores as fixed. I am not sure this patch changes those
>>>>>>>>>>> aspects or not? Please check.
>>>>>>>>>> 
>>>>>>>>>> This patch does *not* affect ‘default’ build type and cross-
>> compilation.
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
>>>>>>>>>>>> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
>>>>>>>>>>>> ---
>>>>>>>>>>>> config/get_max_lcores.py | 13 +++++++++++++
>>>>>>>>>>>> config/meson.build       | 13 ++++++++++++-
>>>>>>>>>>>> 2 files changed, 25 insertions(+), 1 deletion(-) create mode
>>>>>>>>>>>> 100755 config/get_max_lcores.py
>>>>>>>>>>>> 
>>>>>>>>>>>> diff --git a/config/get_max_lcores.py
>>>>>>>>>>>> b/config/get_max_lcores.py new file mode 100755 index
>>>>>>>>>>>> 000000000000..ebf1c7efdadd
>>>>>>>>>>>> --- /dev/null
>>>>>>>>>>>> +++ b/config/get_max_lcores.py
>>>>>>>>>>>> @@ -0,0 +1,13 @@
>>>>>>>>>>>> +#!/usr/bin/python3
>>>>>>>>>>>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020
>>>>>>>>>>>> +Arm Limited
>>>>>>>>>>>> +
>>>>>>>>>>>> +import os
>>>>>>>>>>>> +
>>>>>>>>>>>> +max_lcores = []
>>>>>>>>>>>> +
>>>>>>>>>>>> +nCPU = os.cpu_count()
>>>>>>>>>>>> +
>>>>>>>>>>>> +max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
>>>>>>>>>>>> +
>>>>>>>>>>>> +print(' '.join(max_lcores))
>>>>>>>>>>>> diff --git a/config/meson.build b/config/meson.build index
>>>>>>>>>>>> 6996e5cbeaa5..80c05bc15d2f 100644
>>>>>>>>>>>> --- a/config/meson.build
>>>>>>>>>>>> +++ b/config/meson.build
>>>>>>>>>>>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller
>>>>>>>>>>>> reserved memory
>>>>>>>>>> areas
>>>>>>>>>>>>    dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif
>>>>>>>>>>>> 
>>>>>>>>>>>> -
>>>>>>>>>>>> compile_time_cpuflags = []
>>>>>>>>>>>> subdir(arch_subdir)
>>>>>>>>>>>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS',
>>>>>>>>>>>> ','.join(compile_time_cpuflags))
>>>>>>>>>>>> 
>>>>>>>>>>>> +# set max lcores
>>>>>>>>>>>> +if machine != 'default' and not meson.is_cross_build()
>>>>>>>>>>>> +       # The script returns max lcores
>>>>>>>>>>>> +       params = files('get_max_lcores.py')
>>>>>>>>>>>> +       cmd_out = run_command(params)
>>>>>>>>> 
>>>>>>>>> Have you considered running just a shell command, such as "nproc --
>> all"?
>>>>>>>> 
>>>>>>>> Is this really a good idea?
>>>>>>>> For real distributions and NFV products, the build and runtime
>>>>>>>> environment will usually be different even if on same CPU architecture.
>>>>>>>> 
>>>>>>>> In many cases there maybe a huge build machine (128 CPU) or in a
>>>>>>>> container (reported as single cpu) even if not doing cross build.
>>>>>>> 
>>>>>>> That’s a great point, Stephen. IMO, this patch is useful when
>>>>>>> building and running natively.
>>>>>>> For all other purposes (like the ones you mentioned), do you think
>>>>>>> it is a good idea to set RTE_MAX_LCORE using -Dmax_lcores?
>>>>>> 
>>>>>> We should only use this native builds, as that would be consistent
>>>>>> with the
>>>> current meson build philosophy of "meson figuring as much as possible
>>>> on its own". Any build other than native implies the user wants to
>>>> deviate from the build machine.
>>>>>> 
>>>>> 
>>>>> The MIDR value-based probing doesn’t quite work well for Arm IP
>>>>> (currently
>>>> being discussed on this patch: https://patches.dpdk.org/patch/76981/).
>>>>> 
>>>>>> One way to do this automatic core count is when max_lcores=0 (0
>>>>>> would
>>>> have the special meaning of "figure core count automatically"). We
>>>> can set that as default in meson_option.txt and then users will have
>>>> the ability to set it to whatever they want, even for native builds. What do
>> you think?
>>>>>> 
>>>>> 
>>>>> Yes, agreed.
>>>>> 
>>>>>> Currently the -Dmax_lcores option doesn't work for arm builds (the
>>>>>> value of
>>>> RTE_MAX_LCORE is overwritten in config/arm/meson.build). I believe
>>>> the patch tries to address this, but still, we need to be mindful of that.
>>>>>> 
>>>>>> Juraj

diff mbox series

Patch

diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py
new file mode 100755
index 000000000000..ebf1c7efdadd
--- /dev/null
+++ b/config/get_max_lcores.py
@@ -0,0 +1,13 @@ 
+#!/usr/bin/python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2020 Arm Limited
+
+import os
+
+max_lcores = []
+
+nCPU = os.cpu_count()
+
+max_lcores.append(str(nCPU & 0xFFF))             # Number of CPUs
+
+print(' '.join(max_lcores))
diff --git a/config/meson.build b/config/meson.build
index 6996e5cbeaa5..80c05bc15d2f 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -237,11 +237,22 @@  else # for 32-bit we need smaller reserved memory areas
 	dpdk_conf.set('RTE_MAX_MEM_MB', 2048)
 endif
 
-
 compile_time_cpuflags = []
 subdir(arch_subdir)
 dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags))
 
+# set max lcores
+if machine != 'default' and not meson.is_cross_build()
+	# The script returns max lcores
+	params = files('get_max_lcores.py')
+	cmd_out = run_command(params)
+	if cmd_out.returncode() == 0
+		cmd_lcore = cmd_out.stdout().to_lower().strip().split(' ')
+	endif
+	max_lcore = cmd_lcore[0].to_int()
+	dpdk_conf.set('RTE_MAX_LCORE', max_lcore)
+endif
+
 # set the install path for the drivers
 dpdk_conf.set_quoted('RTE_EAL_PMD_PATH', eal_pmd_path)