[20.11,00/12] add max SIMD bitwidth to EAL

  A number of components in DPDK have optional AVX-512 or other vector
code paths which can be selected at runtime. Rather than having each
component provide its own mechanism to select a code path, this patchset
adds support for a single setting to control what code paths are used.
This can be used to enable some non-default code paths e.g. ones using
AVX-512, but also to limit the code paths to certain vector widths, or
to scalar code only, which is useful for testing.

The max SIMD bitwidth setting can be set by the app itself through use of
the available API, or can be overriden by a commandline argument passed by
the user.

Ciara Power (12):
  eal: add max SIMD bitwidth
  eal: add default SIMD bitwidth values
  net/i40e: add checks for max SIMD bitwidth
  net/axgbe: add checks for max SIMD bitwidth
  net/bnxt: add checks for max SIMD bitwidth
  net/enic: add checks for max SIMD bitwidth
  net/fm10k: add checks for max SIMD bitwidth
  net/iavf: add checks for max SIMD bitwidth
  net/ice: add checks for max SIMD bitwidth
  net/ixgbe: add checks for max SIMD bitwidth
  net/mlx5: add checks for max SIMD bitwidth
  net/virtio: add checks for max SIMD bitwidth

 drivers/net/axgbe/axgbe_rxtx.c             |  3 +-
 drivers/net/bnxt/bnxt_ethdev.c             |  6 ++-
 drivers/net/enic/enic_rxtx_vec_avx2.c      |  3 +-
 drivers/net/fm10k/fm10k_ethdev.c           | 11 ++--
 drivers/net/i40e/i40e_rxtx.c               | 19 ++++---
 drivers/net/iavf/iavf_rxtx.c               | 16 +++---
 drivers/net/ice/ice_rxtx.c                 | 20 ++++---
 drivers/net/ixgbe/ixgbe_rxtx.c             |  7 ++-
 drivers/net/mlx5/mlx5_ethdev.c             |  3 +-
 drivers/net/virtio/virtio_ethdev.c         | 12 +++--
 lib/librte_eal/arm/include/rte_vect.h      |  2 +
 lib/librte_eal/common/eal_common_options.c | 63 ++++++++++++++++++++++
 lib/librte_eal/common/eal_internal_cfg.h   |  8 +++
 lib/librte_eal/common/eal_options.h        |  2 +
 lib/librte_eal/include/generic/rte_vect.h  |  2 +
 lib/librte_eal/include/rte_eal.h           | 31 +++++++++++
 lib/librte_eal/ppc/include/rte_vect.h      |  2 +
 lib/librte_eal/rte_eal_version.map         |  4 ++
 lib/librte_eal/x86/include/rte_vect.h      |  2 +
 19 files changed, 184 insertions(+), 32 deletions(-)
  

On Fri,  7 Aug 2020 16:58:47 +0100
Ciara Power <ciara.power@intel.com> wrote:

> A number of components in DPDK have optional AVX-512 or other vector
> code paths which can be selected at runtime. Rather than having each
> component provide its own mechanism to select a code path, this patchset
> adds support for a single setting to control what code paths are used.
> This can be used to enable some non-default code paths e.g. ones using
> AVX-512, but also to limit the code paths to certain vector widths, or
> to scalar code only, which is useful for testing.
> 
> The max SIMD bitwidth setting can be set by the app itself through use of
> the available API, or can be overriden by a commandline argument passed by
> the user.
> 
> Ciara Power (12):
>   eal: add max SIMD bitwidth
>   eal: add default SIMD bitwidth values
>   net/i40e: add checks for max SIMD bitwidth
>   net/axgbe: add checks for max SIMD bitwidth
>   net/bnxt: add checks for max SIMD bitwidth
>   net/enic: add checks for max SIMD bitwidth
>   net/fm10k: add checks for max SIMD bitwidth
>   net/iavf: add checks for max SIMD bitwidth
>   net/ice: add checks for max SIMD bitwidth
>   net/ixgbe: add checks for max SIMD bitwidth
>   net/mlx5: add checks for max SIMD bitwidth
>   net/virtio: add checks for max SIMD bitwidth
> 
>  drivers/net/axgbe/axgbe_rxtx.c             |  3 +-
>  drivers/net/bnxt/bnxt_ethdev.c             |  6 ++-
>  drivers/net/enic/enic_rxtx_vec_avx2.c      |  3 +-
>  drivers/net/fm10k/fm10k_ethdev.c           | 11 ++--
>  drivers/net/i40e/i40e_rxtx.c               | 19 ++++---
>  drivers/net/iavf/iavf_rxtx.c               | 16 +++---
>  drivers/net/ice/ice_rxtx.c                 | 20 ++++---
>  drivers/net/ixgbe/ixgbe_rxtx.c             |  7 ++-
>  drivers/net/mlx5/mlx5_ethdev.c             |  3 +-
>  drivers/net/virtio/virtio_ethdev.c         | 12 +++--
>  lib/librte_eal/arm/include/rte_vect.h      |  2 +
>  lib/librte_eal/common/eal_common_options.c | 63 ++++++++++++++++++++++
>  lib/librte_eal/common/eal_internal_cfg.h   |  8 +++
>  lib/librte_eal/common/eal_options.h        |  2 +
>  lib/librte_eal/include/generic/rte_vect.h  |  2 +
>  lib/librte_eal/include/rte_eal.h           | 31 +++++++++++
>  lib/librte_eal/ppc/include/rte_vect.h      |  2 +
>  lib/librte_eal/rte_eal_version.map         |  4 ++
>  lib/librte_eal/x86/include/rte_vect.h      |  2 +
>  19 files changed, 184 insertions(+), 32 deletions(-)
> 

This looks useful, could you add some documentation on rationale
and how you expect application to set it.
  

Hi Stephen,

To give an overview of the rationale behind the patchset:
-  It allows other apps such as OVS and VPP which already make use of
   AVX-512 to indicate that they are happy for DPDK to use AVX-512 too.
-  It allows the end-user to override those settings if so desired.
-  It allows an easy way for the user to test with different vector paths by
   limiting bitwidths.

I can add some documentation for this in a v2, thanks for the suggestion.

- Ciara


>-----Original Message-----
>From: Stephen Hemminger <stephen@networkplumber.org>
>Sent: Friday 7 August 2020 17:19
>To: Power, Ciara <ciara.power@intel.com>
>Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
>Subject: Re: [dpdk-dev] [PATCH 20.11 00/12] add max SIMD bitwidth to EAL
>
>On Fri,  7 Aug 2020 16:58:47 +0100
>Ciara Power <ciara.power@intel.com> wrote:
>
>> A number of components in DPDK have optional AVX-512 or other vector
>> code paths which can be selected at runtime. Rather than having each
>> component provide its own mechanism to select a code path, this
>> patchset adds support for a single setting to control what code paths are
>used.
>> This can be used to enable some non-default code paths e.g. ones using
>> AVX-512, but also to limit the code paths to certain vector widths, or
>> to scalar code only, which is useful for testing.
>>
>> The max SIMD bitwidth setting can be set by the app itself through use
>> of the available API, or can be overriden by a commandline argument
>> passed by the user.
>>
>> Ciara Power (12):
>>   eal: add max SIMD bitwidth
>>   eal: add default SIMD bitwidth values
>>   net/i40e: add checks for max SIMD bitwidth
>>   net/axgbe: add checks for max SIMD bitwidth
>>   net/bnxt: add checks for max SIMD bitwidth
>>   net/enic: add checks for max SIMD bitwidth
>>   net/fm10k: add checks for max SIMD bitwidth
>>   net/iavf: add checks for max SIMD bitwidth
>>   net/ice: add checks for max SIMD bitwidth
>>   net/ixgbe: add checks for max SIMD bitwidth
>>   net/mlx5: add checks for max SIMD bitwidth
>>   net/virtio: add checks for max SIMD bitwidth
>>
>>  drivers/net/axgbe/axgbe_rxtx.c             |  3 +-
>>  drivers/net/bnxt/bnxt_ethdev.c             |  6 ++-
>>  drivers/net/enic/enic_rxtx_vec_avx2.c      |  3 +-
>>  drivers/net/fm10k/fm10k_ethdev.c           | 11 ++--
>>  drivers/net/i40e/i40e_rxtx.c               | 19 ++++---
>>  drivers/net/iavf/iavf_rxtx.c               | 16 +++---
>>  drivers/net/ice/ice_rxtx.c                 | 20 ++++---
>>  drivers/net/ixgbe/ixgbe_rxtx.c             |  7 ++-
>>  drivers/net/mlx5/mlx5_ethdev.c             |  3 +-
>>  drivers/net/virtio/virtio_ethdev.c         | 12 +++--
>>  lib/librte_eal/arm/include/rte_vect.h      |  2 +
>>  lib/librte_eal/common/eal_common_options.c | 63
>++++++++++++++++++++++
>>  lib/librte_eal/common/eal_internal_cfg.h   |  8 +++
>>  lib/librte_eal/common/eal_options.h        |  2 +
>>  lib/librte_eal/include/generic/rte_vect.h  |  2 +
>>  lib/librte_eal/include/rte_eal.h           | 31 +++++++++++
>>  lib/librte_eal/ppc/include/rte_vect.h      |  2 +
>>  lib/librte_eal/rte_eal_version.map         |  4 ++
>>  lib/librte_eal/x86/include/rte_vect.h      |  2 +
>>  19 files changed, 184 insertions(+), 32 deletions(-)
>>
>
>This looks useful, could you add some documentation on rationale and how
>you expect application to set it.
  

Hi Ciara,
	I have not reviewed other patches in this series yet. Few questions inline.

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Ciara Power
> Sent: Friday, August 7, 2020 10:59 AM
> To: dev@dpdk.org
> Cc: bruce.richardson@intel.com; Ciara Power <ciara.power@intel.com>
> Subject: [dpdk-dev] [PATCH 20.11 00/12] add max SIMD bitwidth to EAL
> 
> A number of components in DPDK have optional AVX-512 or other vector code
> paths which can be selected at runtime. Rather than having each component
> provide its own mechanism to select a code path, this patchset adds support
> for a single setting to control what code paths are used.
Do you mean that all the components will have to use AVX-512?
IMO, different libraries might behave differently to the use of different vector sizes. Are we taking away the ability to use different vector sizes for different components.

> This can be used to enable some non-default code paths e.g. ones using AVX-
> 512, but also to limit the code paths to certain vector widths, or to scalar
> code only, which is useful for testing.
> 
> The max SIMD bitwidth setting can be set by the app itself through use of the
> available API, or can be overriden by a commandline argument passed by the
> user.
Arm platforms support SVE (scalable vector extensions) feature. With this feature, the code is agnostic to the vector size. i.e. same code can run on various vector sizes. There is no code yet in DPDK with this feature. But, it will be added in the near future. It would be good to handle this now so that we do not have issues in the future..

> 
> Ciara Power (12):
>   eal: add max SIMD bitwidth
>   eal: add default SIMD bitwidth values
>   net/i40e: add checks for max SIMD bitwidth
>   net/axgbe: add checks for max SIMD bitwidth
>   net/bnxt: add checks for max SIMD bitwidth
>   net/enic: add checks for max SIMD bitwidth
>   net/fm10k: add checks for max SIMD bitwidth
>   net/iavf: add checks for max SIMD bitwidth
>   net/ice: add checks for max SIMD bitwidth
>   net/ixgbe: add checks for max SIMD bitwidth
>   net/mlx5: add checks for max SIMD bitwidth
>   net/virtio: add checks for max SIMD bitwidth
> 
>  drivers/net/axgbe/axgbe_rxtx.c             |  3 +-
>  drivers/net/bnxt/bnxt_ethdev.c             |  6 ++-
>  drivers/net/enic/enic_rxtx_vec_avx2.c      |  3 +-
>  drivers/net/fm10k/fm10k_ethdev.c           | 11 ++--
>  drivers/net/i40e/i40e_rxtx.c               | 19 ++++---
>  drivers/net/iavf/iavf_rxtx.c               | 16 +++---
>  drivers/net/ice/ice_rxtx.c                 | 20 ++++---
>  drivers/net/ixgbe/ixgbe_rxtx.c             |  7 ++-
>  drivers/net/mlx5/mlx5_ethdev.c             |  3 +-
>  drivers/net/virtio/virtio_ethdev.c         | 12 +++--
>  lib/librte_eal/arm/include/rte_vect.h      |  2 +
>  lib/librte_eal/common/eal_common_options.c | 63
> ++++++++++++++++++++++
>  lib/librte_eal/common/eal_internal_cfg.h   |  8 +++
>  lib/librte_eal/common/eal_options.h        |  2 +
>  lib/librte_eal/include/generic/rte_vect.h  |  2 +
>  lib/librte_eal/include/rte_eal.h           | 31 +++++++++++
>  lib/librte_eal/ppc/include/rte_vect.h      |  2 +
>  lib/librte_eal/rte_eal_version.map         |  4 ++
>  lib/librte_eal/x86/include/rte_vect.h      |  2 +
>  19 files changed, 184 insertions(+), 32 deletions(-)
> 
> --
> 2.17.1
  

Hi Honnappa,

 
>-----Original Message-----
>From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
>Sent: Tuesday 11 August 2020 06:37
>To: Power, Ciara <ciara.power@intel.com>; dev@dpdk.org
>Cc: Richardson, Bruce <bruce.richardson@intel.com>; nd <nd@arm.com>;
>Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd
><nd@arm.com>
>Subject: RE: [dpdk-dev] [PATCH 20.11 00/12] add max SIMD bitwidth to EAL
>
>Hi Ciara,
>	I have not reviewed other patches in this series yet. Few questions
>inline.
>
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Ciara Power
>> Sent: Friday, August 7, 2020 10:59 AM
>> To: dev@dpdk.org
>> Cc: bruce.richardson@intel.com; Ciara Power <ciara.power@intel.com>
>> Subject: [dpdk-dev] [PATCH 20.11 00/12] add max SIMD bitwidth to EAL
>>
>> A number of components in DPDK have optional AVX-512 or other vector
>> code paths which can be selected at runtime. Rather than having each
>> component provide its own mechanism to select a code path, this
>> patchset adds support for a single setting to control what code paths are
>used.
>Do you mean that all the components will have to use AVX-512?
>IMO, different libraries might behave differently to the use of different vector
>sizes. Are we taking away the ability to use different vector sizes for different
>components.
>

No, this setting is a max bitwidth which can be set by apps using the EAL API 
based on what is best for the usage in their apps, or by the user with an EAL flag,
but each library is still free to choose it's own best path, subject to it not being
longer than the specified max. 
For example, if the max bitwidth is set to 512, a library can still choose to use a 256-bit
path over a 512 one if its advantageous.


>> This can be used to enable some non-default code paths e.g. ones using
>> AVX- 512, but also to limit the code paths to certain vector widths,
>> or to scalar code only, which is useful for testing.
>>
>> The max SIMD bitwidth setting can be set by the app itself through use
>> of the available API, or can be overriden by a commandline argument
>> passed by the user.
>Arm platforms support SVE (scalable vector extensions) feature. With this
>feature, the code is agnostic to the vector size. i.e. same code can run on
>various vector sizes. There is no code yet in DPDK with this feature. But, it will
>be added in the near future. It would be good to handle this now so that we
>do not have issues in the future..

Do you have any suggestions how this could be handled?


<snip>

Thanks,
Ciara