From patchwork Mon Oct 19 13:48:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81363 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0F730A04DC; Mon, 19 Oct 2020 15:49:41 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E804ACABE; Mon, 19 Oct 2020 15:49:24 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id 6DF4AC848 for ; Mon, 19 Oct 2020 15:49:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115360; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HPW23EPJPNQBp6Vs4sB62tQiYKmhe+QjbEUPstxnPa4=; b=NHrkEw0shL59mWYcOLTmGZXaBfcldCqcDHZrmLGHJd9s/cfRalE+f0JKzf0U5qQv0UIbNc EpGLKXz96Yoyhzx+QIokL35FOIryIWl4ikz9yY8Y4FULJiOYV6/Phg5xKtN8YeWOEPN6vK WCN7Y07SBecCXwRbL+gxSIgtNXswG3c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-364-SWI7v_qXPJaQ9g-X8dxM7Q-1; Mon, 19 Oct 2020 09:49:16 -0400 X-MC-Unique: SWI7v_qXPJaQ9g-X8dxM7Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C665E64157; Mon, 19 Oct 2020 13:49:14 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 572EE55775; Mon, 19 Oct 2020 13:49:12 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Ray Kinsella Date: Mon, 19 Oct 2020 15:48:41 +0200 Message-Id: <20201019134858.32507-2-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 01/18] eal: control max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power This patch adds a max SIMD bitwidth EAL configuration. The API allows for an app to set this value. It can also be set using EAL argument --force-max-simd-bitwidth, which will lock the value and override any modifications made by the app. Each arch has a define for the default SIMD bitwidth value, this is used on EAL init to set the config max SIMD bitwidth. Signed-off-by: Ciara Power Acked-by: Konstantin Ananyev Reviewed-by: Ruifeng Wang Acked-by: Ray Kinsella --- v10: - moved new API to rte_vect.h, prefixed with rte_vect_ and added experimental banner, - split patch 2 and squashed eal flags documentation in patch 1, v9: Added release note. v7: Reordered entries in the symbol .map file. v4: - Used RTE_SIMD_MAX instead of UINT16_MAX. - Renamed enums to better reflect usage. - Added functions to windows symbol export file. - Modified Doxygen comments. - Modified enum name. - Changed RTE_SIMD_MAX value to a power of 2. - Merged patch 2 into this patch. - Enum now used for default value defines. - Fixed some small comments on v3. v3: - Added enum value to essentially disable using max SIMD to choose paths, intended for use by ARM SVE. - Fixed parsing bitwidth argument to return an error for values greater than uint16_t. - Removed unnecessary define in generic rte_vect.h - Changed default bitwidth for ARM to UINT16_MAX, to allow for SVE. v2: - Added to Doxygen comment for API. - Changed default bitwidth for Arm to 128. --- doc/guides/linux_gsg/eal_args.include.rst | 16 +++++ .../prog_guide/env_abstraction_layer.rst | 34 ++++++++++ doc/guides/rel_notes/release_20_11.rst | 6 ++ lib/librte_eal/arm/include/rte_vect.h | 2 + lib/librte_eal/common/eal_common_options.c | 66 +++++++++++++++++++ lib/librte_eal/common/eal_internal_cfg.h | 8 +++ lib/librte_eal/common/eal_options.h | 2 + lib/librte_eal/include/generic/rte_vect.h | 53 ++++++++++++++- lib/librte_eal/ppc/include/rte_vect.h | 2 + lib/librte_eal/rte_eal_exports.def | 3 + lib/librte_eal/rte_eal_version.map | 2 + lib/librte_eal/x86/include/rte_vect.h | 2 + 12 files changed, 194 insertions(+), 2 deletions(-) diff --git a/doc/guides/linux_gsg/eal_args.include.rst b/doc/guides/linux_gsg/eal_args.include.rst index 0fe4457968..a0bfbd1a98 100644 --- a/doc/guides/linux_gsg/eal_args.include.rst +++ b/doc/guides/linux_gsg/eal_args.include.rst @@ -210,3 +210,19 @@ Other options * ``--no-telemetry``: Disable telemetry. + +* ``--force-max-simd-bitwidth=``: + + Specify the maximum SIMD bitwidth size to handle. This limits which vector paths, + if any, are taken, as any paths taken must use a bitwidth below the max bitwidth limit. + For example, to allow all SIMD bitwidths up to and including AVX-512:: + + --force-max-simd-bitwidth=512 + + The following example shows limiting the bitwidth to 64-bits to disable all vector code:: + + --force-max-simd-bitwidth=64 + + To disable use of max SIMD bitwidth limit:: + + --force-max-simd-bitwidth=0 diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 936c885081..fafed83fba 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -486,6 +486,40 @@ the desired addressing mode when virtual devices that are not directly attached To facilitate forcing the IOVA mode to a specific value the EAL command line option ``--iova-mode`` can be used to select either physical addressing('pa') or virtual addressing('va'). +.. _max_simd_bitwidth: + + +Max SIMD bitwidth +~~~~~~~~~~~~~~~~~ + +The EAL provides a single setting to limit the max SIMD bitwidth used by DPDK, +which is used in determining the vector path, if any, chosen by a component. +The value can be set at runtime by an application using the +'rte_vect_set_max_simd_bitwidth(uint16_t bitwidth)' function, +which should only be called once at initialization, before EAL init. +The value can be overridden by the user using the EAL command-line option '--force-max-simd-bitwidth'. + +When choosing a vector path, along with checking the CPU feature support, +the value of the max SIMD bitwidth must also be checked, and can be retrieved using the +'rte_vect_get_max_simd_bitwidth()' function. +The value should be compared against the enum values for accepted max SIMD bitwidths: + +.. code-block:: c + + enum rte_vect_max_simd { + RTE_VECT_SIMD_DISABLED = 64, + RTE_VECT_SIMD_128 = 128, + RTE_VECT_SIMD_256 = 256, + RTE_VECT_SIMD_512 = 512, + RTE_VECT_SIMD_MAX = INT16_MAX + 1, + }; + + if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512) + /* Take AVX-512 vector path */ + else if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) + /* Take AVX2 vector path */ + + Memory Segments and Memory Zones (memzone) ------------------------------------------ diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst index 25ea3c20b4..c2bd6ee741 100644 --- a/doc/guides/rel_notes/release_20_11.rst +++ b/doc/guides/rel_notes/release_20_11.rst @@ -77,6 +77,12 @@ New Features This API is specific to x86 and implemented as a stub for other architectures. +* **Added support for limiting maximum SIMD bitwidth.** + + Added a new EAL config setting ``max_simd_bitwidth`` to limit the vector + path selection at runtime. This value can be set by apps using the + ``rte_vect_set_max_simd_bitwidth`` function, or by the user with EAL flag ``--force-max-simd-bitwidth``. + * **Updated CRC modules of the net library.** * Added runtime selection of the optimal architecture-specific CRC path. diff --git a/lib/librte_eal/arm/include/rte_vect.h b/lib/librte_eal/arm/include/rte_vect.h index f6a455b4e7..a739e6e66a 100644 --- a/lib/librte_eal/arm/include/rte_vect.h +++ b/lib/librte_eal/arm/include/rte_vect.h @@ -14,6 +14,8 @@ extern "C" { #endif +#define RTE_VECT_DEFAULT_SIMD_BITWIDTH RTE_VECT_SIMD_MAX + typedef int32x4_t xmm_t; #define XMM_SIZE (sizeof(xmm_t)) diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index a5426e1234..60f0e4c69b 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -35,6 +35,7 @@ #ifndef RTE_EXEC_ENV_WINDOWS #include #endif +#include #include "eal_internal_cfg.h" #include "eal_options.h" @@ -102,6 +103,7 @@ eal_long_options[] = { {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, {OPT_TELEMETRY, 0, NULL, OPT_TELEMETRY_NUM }, {OPT_NO_TELEMETRY, 0, NULL, OPT_NO_TELEMETRY_NUM }, + {OPT_FORCE_MAX_SIMD_BITWIDTH, 1, NULL, OPT_FORCE_MAX_SIMD_BITWIDTH_NUM}, {0, 0, NULL, 0 } }; @@ -343,6 +345,8 @@ eal_reset_internal_config(struct internal_config *internal_cfg) internal_cfg->user_mbuf_pool_ops_name = NULL; CPU_ZERO(&internal_cfg->ctrl_cpuset); internal_cfg->init_complete = 0; + internal_cfg->max_simd_bitwidth.bitwidth = RTE_VECT_DEFAULT_SIMD_BITWIDTH; + internal_cfg->max_simd_bitwidth.forced = 0; } static int @@ -1309,6 +1313,34 @@ eal_parse_iova_mode(const char *name) return 0; } +static int +eal_parse_simd_bitwidth(const char *arg) +{ + char *end; + unsigned long bitwidth; + int ret; + struct internal_config *internal_conf = + eal_get_internal_configuration(); + + if (arg == NULL || arg[0] == '\0') + return -1; + + errno = 0; + bitwidth = strtoul(arg, &end, 0); + + /* check for errors */ + if (errno != 0 || end == NULL || *end != '\0' || bitwidth > RTE_VECT_SIMD_MAX) + return -1; + + if (bitwidth == 0) + bitwidth = (unsigned long) RTE_VECT_SIMD_MAX; + ret = rte_vect_set_max_simd_bitwidth(bitwidth); + if (ret < 0) + return -1; + internal_conf->max_simd_bitwidth.forced = 1; + return 0; +} + static int eal_parse_base_virtaddr(const char *arg) { @@ -1707,6 +1739,13 @@ eal_parse_common_option(int opt, const char *optarg, case OPT_NO_TELEMETRY_NUM: conf->no_telemetry = 1; break; + case OPT_FORCE_MAX_SIMD_BITWIDTH_NUM: + if (eal_parse_simd_bitwidth(optarg) < 0) { + RTE_LOG(ERR, EAL, "invalid parameter for --" + OPT_FORCE_MAX_SIMD_BITWIDTH "\n"); + return -1; + } + break; /* don't know what to do, leave this to caller */ default: @@ -1903,6 +1942,32 @@ eal_check_common_options(struct internal_config *internal_cfg) return 0; } +uint16_t +rte_vect_get_max_simd_bitwidth(void) +{ + const struct internal_config *internal_conf = + eal_get_internal_configuration(); + return internal_conf->max_simd_bitwidth.bitwidth; +} + +int +rte_vect_set_max_simd_bitwidth(uint16_t bitwidth) +{ + struct internal_config *internal_conf = + eal_get_internal_configuration(); + if (internal_conf->max_simd_bitwidth.forced) { + RTE_LOG(NOTICE, EAL, "Cannot set max SIMD bitwidth - user runtime override enabled"); + return -EPERM; + } + + if (bitwidth < RTE_VECT_SIMD_DISABLED || !rte_is_power_of_2(bitwidth)) { + RTE_LOG(ERR, EAL, "Invalid bitwidth value!\n"); + return -EINVAL; + } + internal_conf->max_simd_bitwidth.bitwidth = bitwidth; + return 0; +} + void eal_common_usage(void) { @@ -1981,6 +2046,7 @@ eal_common_usage(void) " --"OPT_BASE_VIRTADDR" Base virtual address\n" " --"OPT_TELEMETRY" Enable telemetry support (on by default)\n" " --"OPT_NO_TELEMETRY" Disable telemetry support\n" + " --"OPT_FORCE_MAX_SIMD_BITWIDTH" Force the max SIMD bitwidth\n" "\nEAL options for DEBUG use only:\n" " --"OPT_HUGE_UNLINK" Unlink hugepage files after init\n" " --"OPT_NO_HUGE" Use malloc instead of hugetlbfs\n" diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h index de627c7627..51dbe86e2b 100644 --- a/lib/librte_eal/common/eal_internal_cfg.h +++ b/lib/librte_eal/common/eal_internal_cfg.h @@ -33,6 +33,12 @@ struct hugepage_info { int lock_descriptor; /**< file descriptor for hugepage dir */ }; +struct simd_bitwidth { + bool forced; + /**< flag indicating if bitwidth is forced and can't be modified */ + uint16_t bitwidth; /**< bitwidth value */ +}; + /** * internal configuration */ @@ -85,6 +91,8 @@ struct internal_config { volatile unsigned int init_complete; /**< indicates whether EAL has completed initialization */ unsigned int no_telemetry; /**< true to disable Telemetry */ + struct simd_bitwidth max_simd_bitwidth; + /**< max simd bitwidth path to use */ }; void eal_reset_internal_config(struct internal_config *internal_cfg); diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 89769d48b4..ef33979664 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -85,6 +85,8 @@ enum { OPT_TELEMETRY_NUM, #define OPT_NO_TELEMETRY "no-telemetry" OPT_NO_TELEMETRY_NUM, +#define OPT_FORCE_MAX_SIMD_BITWIDTH "force-max-simd-bitwidth" + OPT_FORCE_MAX_SIMD_BITWIDTH_NUM, OPT_LONG_MAX_NUM }; diff --git a/lib/librte_eal/include/generic/rte_vect.h b/lib/librte_eal/include/generic/rte_vect.h index 3fc47979f8..736066d430 100644 --- a/lib/librte_eal/include/generic/rte_vect.h +++ b/lib/librte_eal/include/generic/rte_vect.h @@ -7,13 +7,16 @@ /** * @file - * SIMD vector types + * SIMD vector types and control * - * This file defines types to use vector instructions with generic C code. + * This file defines types to use vector instructions with generic C code + * and APIs to enable the code using them. */ #include +#include + /* Unsigned vector types */ /** @@ -183,4 +186,50 @@ typedef int32_t rte_v256s32_t __attribute__((vector_size(32), aligned(32))); */ typedef int64_t rte_v256s64_t __attribute__((vector_size(32), aligned(32))); +/** + * The max SIMD bitwidth value to limit vector path selection. + */ +enum rte_vect_max_simd { + RTE_VECT_SIMD_DISABLED = 64, + /**< Limits path selection to scalar, disables all vector paths. */ + RTE_VECT_SIMD_128 = 128, + /**< Limits path selection to SSE/NEON/Altivec or below. */ + RTE_VECT_SIMD_256 = 256, /**< Limits path selection to AVX2 or below. */ + RTE_VECT_SIMD_512 = 512, /**< Limits path selection to AVX512 or below. */ + RTE_VECT_SIMD_MAX = INT16_MAX + 1, + /**< + * Disables limiting by max SIMD bitwidth, allows all suitable paths. + * This value is used as it is a large number and a power of 2. + */ +}; + +/** + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice + * + * Get the supported SIMD bitwidth. + * + * @return + * uint16_t bitwidth. + */ +__rte_experimental +uint16_t rte_vect_get_max_simd_bitwidth(void); + +/** + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice + * + * Set the supported SIMD bitwidth. + * This API should only be called once at initialization, before EAL init. + * + * @param bitwidth + * uint16_t bitwidth. + * @return + * - 0 on success. + * - -EINVAL on invalid bitwidth parameter. + * - -EPERM if bitwidth is forced. + */ +__rte_experimental +int rte_vect_set_max_simd_bitwidth(uint16_t bitwidth); + #endif /* _RTE_VECT_H_ */ diff --git a/lib/librte_eal/ppc/include/rte_vect.h b/lib/librte_eal/ppc/include/rte_vect.h index b0545c878c..c1f0b0672c 100644 --- a/lib/librte_eal/ppc/include/rte_vect.h +++ b/lib/librte_eal/ppc/include/rte_vect.h @@ -15,6 +15,8 @@ extern "C" { #endif +#define RTE_VECT_DEFAULT_SIMD_BITWIDTH RTE_VECT_SIMD_256 + typedef vector signed int xmm_t; #define XMM_SIZE (sizeof(xmm_t)) diff --git a/lib/librte_eal/rte_eal_exports.def b/lib/librte_eal/rte_eal_exports.def index 975acb8ffe..7eab9fbd2c 100644 --- a/lib/librte_eal/rte_eal_exports.def +++ b/lib/librte_eal/rte_eal_exports.def @@ -303,6 +303,9 @@ EXPORTS rte_thread_register rte_thread_unregister + rte_vect_get_max_simd_bitwidth + rte_vect_set_max_simd_bitwidth + rte_mem_lock rte_mem_map rte_mem_page_size diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index df8e0af758..a625ca2dc8 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -400,6 +400,8 @@ EXPERIMENTAL { __rte_eal_trace_generic_size_t; rte_epoll_wait_interruptible; rte_service_lcore_may_be_active; + rte_vect_get_max_simd_bitwidth; + rte_vect_set_max_simd_bitwidth; }; INTERNAL { diff --git a/lib/librte_eal/x86/include/rte_vect.h b/lib/librte_eal/x86/include/rte_vect.h index 64383c3606..2e40b77da9 100644 --- a/lib/librte_eal/x86/include/rte_vect.h +++ b/lib/librte_eal/x86/include/rte_vect.h @@ -36,6 +36,8 @@ extern "C" { #endif +#define RTE_VECT_DEFAULT_SIMD_BITWIDTH RTE_VECT_SIMD_256 + typedef __m128i xmm_t; #define XMM_SIZE (sizeof(xmm_t)) From patchwork Mon Oct 19 13:48:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81364 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 728CBA04DC; Mon, 19 Oct 2020 15:50:07 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 09854D027; Mon, 19 Oct 2020 15:49:28 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by dpdk.org (Postfix) with ESMTP id C0026C848 for ; Mon, 19 Oct 2020 15:49:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y4bvEfo4owJI4qavGoQ3hQlxEZmrOU/ceb+afU1okCs=; b=aPvIn7a76IWFZ2M9AKVmbR75m7FrZXqc8BNQiOZ1vGm+0zno9wjNw0zpcNi8EuQ7JSSNEM /khQNMTtzD9Ej2/NUDphmL59RU/g6/C6Z1ZmDExck+XHcf2XvVFO+MjjPYToJhf+4ImOEz 8O4Op4QDJ6p86qidKVMctiQ1mNRJKKk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-205-_fcjS3SqO1m94CWYjejPeg-1; Mon, 19 Oct 2020 09:49:19 -0400 X-MC-Unique: _fcjS3SqO1m94CWYjejPeg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 77C87106F8DB; Mon, 19 Oct 2020 13:49:17 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 30F1C55775; Mon, 19 Oct 2020 13:49:15 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Kevin Laatz Date: Mon, 19 Oct 2020 15:48:42 +0200 Message-Id: <20201019134858.32507-3-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 02/18] doc: describe how to enable AVX512 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power This patch adds documentation on the usage of the max SIMD bitwidth EAL setting to enable AVX-512 at runtime. Signed-off-by: Ciara Power Acked-by: Kevin Laatz --- v6: Updated enum value. v4: Updated docs to reflect renamed enum. v3: - Added enum value for disabling use of max SIMD to doc. - Added entry to HowTo index. --- doc/guides/howto/avx512.rst | 36 ++++++++++++++++++++++++++++++++++++ doc/guides/howto/index.rst | 1 + 2 files changed, 37 insertions(+) create mode 100644 doc/guides/howto/avx512.rst diff --git a/doc/guides/howto/avx512.rst b/doc/guides/howto/avx512.rst new file mode 100644 index 0000000000..6c034b617d --- /dev/null +++ b/doc/guides/howto/avx512.rst @@ -0,0 +1,36 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Intel Corporation. + + +Using AVX-512 with DPDK +======================= + +AVX-512 is not used by default in DPDK, but it can be selected at runtime by apps through the use of EAL API, +and by the user with a commandline argument. DPDK has a setting for max SIMD bitwidth, +which can be modified and will then limit the vector path taken by the code. + + +Using the API in apps +--------------------- + +Apps can request DPDK uses AVX-512 at runtime, if it provides improved application performance. +This can be done by modifying the EAL setting for max SIMD bitwidth to 512, as by default it is 256, +which does not allow for AVX-512. + +.. code-block:: c + + rte_vect_set_max_simd_bitwidth(RTE_VECT_SIMD_512); + +This API should only be called once at initialization, before EAL init. +For more information on the possible enum values to use as a parameter, go to :ref:`max_simd_bitwidth`: + + +Using the command-line argument +--------------------------------------------- + +The user can select to use AVX-512 at runtime, using the following argument to set the max bitwidth:: + + ./app/dpdk-testpmd --force-max-simd-bitwidth=512 + +This will override any further changes to the max SIMD bitwidth in DPDK, +which is useful for testing purposes. diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst index 5a97ea508c..c2a2c60ddb 100644 --- a/doc/guides/howto/index.rst +++ b/doc/guides/howto/index.rst @@ -20,3 +20,4 @@ HowTo Guides telemetry debug_troubleshoot openwrt + avx512 From patchwork Mon Oct 19 13:48:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81365 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C5635A04DC; Mon, 19 Oct 2020 15:50:29 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 979E2E186; Mon, 19 Oct 2020 15:49:30 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by dpdk.org (Postfix) with ESMTP id 80D83D019 for ; Mon, 19 Oct 2020 15:49:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kTIX7Xm3Jh9VZgGU8ZKmYL0PbBjB/to99ubDlg+ihl8=; b=hJxMqLBsB21MIj97Uw03F0FqM55gsRaBXtDqreuuJkd3rNz0nhU2+xkuUsX7nGL3kD1Tyh K8Na2vZ8SrLn/UxisCZSvGaUB7Q5hv8c56N5q0HhJ5w3/0mc+5YNcPJvalYY6uO53mvbnV FNwRFEdVcvdlXVCHu2w4Srj2WR1CI3s= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-499-ukA_7SRrP4OzbreExrZheg-1; Mon, 19 Oct 2020 09:49:21 -0400 X-MC-Unique: ukA_7SRrP4OzbreExrZheg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DEDC080F05B; Mon, 19 Oct 2020 13:49:19 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id D701A55775; Mon, 19 Oct 2020 13:49:17 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com Date: Mon, 19 Oct 2020 15:48:43 +0200 Message-Id: <20201019134858.32507-4-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 03/18] net/i40e: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: Konstantin Ananyev --- v4: Updated enum names. --- drivers/net/i40e/i40e_rxtx.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c index f2844d3f74..5df9a9df56 100644 --- a/drivers/net/i40e/i40e_rxtx.c +++ b/drivers/net/i40e/i40e_rxtx.c @@ -23,6 +23,7 @@ #include #include #include +#include #include "i40e_logs.h" #include "base/i40e_prototype.h" @@ -3098,7 +3099,8 @@ static eth_rx_burst_t i40e_get_latest_rx_vec(bool scatter) { #if defined(RTE_ARCH_X86) && defined(CC_AVX2_SUPPORT) - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2)) + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) return scatter ? i40e_recv_scattered_pkts_vec_avx2 : i40e_recv_pkts_vec_avx2; #endif @@ -3115,7 +3117,8 @@ i40e_get_recommend_rx_vec(bool scatter) * use of AVX2 version to later plaforms, not all those that could * theoretically run it. */ - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F)) + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) return scatter ? i40e_recv_scattered_pkts_vec_avx2 : i40e_recv_pkts_vec_avx2; #endif @@ -3154,7 +3157,8 @@ i40e_set_rx_function(struct rte_eth_dev *dev) } } - if (ad->rx_vec_allowed) { + if (ad->rx_vec_allowed && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { /* Vec Rx path */ PMD_INIT_LOG(DEBUG, "Vector Rx path will be used on port=%d.", dev->data->port_id); @@ -3268,7 +3272,8 @@ static eth_tx_burst_t i40e_get_latest_tx_vec(void) { #if defined(RTE_ARCH_X86) && defined(CC_AVX2_SUPPORT) - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2)) + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) return i40e_xmit_pkts_vec_avx2; #endif return i40e_xmit_pkts_vec; @@ -3283,7 +3288,8 @@ i40e_get_recommend_tx_vec(void) * use of AVX2 version to later plaforms, not all those that could * theoretically run it. */ - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F)) + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) return i40e_xmit_pkts_vec_avx2; #endif return i40e_xmit_pkts_vec; @@ -3311,7 +3317,8 @@ i40e_set_tx_function(struct rte_eth_dev *dev) } if (ad->tx_simple_allowed) { - if (ad->tx_vec_allowed) { + if (ad->tx_vec_allowed && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { PMD_INIT_LOG(DEBUG, "Vector tx finally be used."); if (ad->use_latest_vec) dev->tx_pkt_burst = From patchwork Mon Oct 19 13:48:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81366 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6F999A04DC; Mon, 19 Oct 2020 15:50:46 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1DDB6E235; Mon, 19 Oct 2020 15:49:32 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id 49EA2D033 for ; Mon, 19 Oct 2020 15:49:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115366; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ErXYjcSaG/3DzJlGSDssDj27sKlWoY1eCkuCZ6cc6Zc=; b=TKK2s15ZgNX9Y6lpjFM7kitD66/G0PWDOGt2a+0T1zAnQRVTkZmaaLNFqdYGcQO4JtRw98 Y8MFwJRdrvfhyJGZcJ7r+gK87L4huLSAhyuQDcbx8hz9rOigmc5LNlSJ7LbygpPXs1tVfp bpW8Y67zn4A9xZE7QvkAs/ny1WOt00s= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-148-4YDqIocaOJa6k9EJbBBFug-1; Mon, 19 Oct 2020 09:49:24 -0400 X-MC-Unique: 4YDqIocaOJa6k9EJbBBFug-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8CAFB18BE170; Mon, 19 Oct 2020 13:49:22 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4AF1D7513E; Mon, 19 Oct 2020 13:49:20 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Amaranath Somalapuram Date: Mon, 19 Oct 2020 15:48:44 +0200 Message-Id: <20201019134858.32507-5-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 04/18] net/axgbe: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: Amaranath Somalapuram --- v4: Updated enum name. --- drivers/net/axgbe/axgbe_rxtx.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/net/axgbe/axgbe_rxtx.c b/drivers/net/axgbe/axgbe_rxtx.c index bc93becaa5..032e3cebce 100644 --- a/drivers/net/axgbe/axgbe_rxtx.c +++ b/drivers/net/axgbe/axgbe_rxtx.c @@ -10,6 +10,7 @@ #include #include #include +#include static void axgbe_rx_queue_release(struct axgbe_rx_queue *rx_queue) @@ -557,7 +558,8 @@ int axgbe_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (!pdata->tx_queues) pdata->tx_queues = dev->data->tx_queues; - if (txq->vector_disable) + if (txq->vector_disable || + rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_128) dev->tx_pkt_burst = &axgbe_xmit_pkts; else #ifdef RTE_ARCH_X86 From patchwork Mon Oct 19 13:48:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81368 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 47408A04DC; Mon, 19 Oct 2020 15:51:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 901AFE24F; Mon, 19 Oct 2020 15:49:37 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by dpdk.org (Postfix) with ESMTP id F2E3CCFC1 for ; Mon, 19 Oct 2020 15:49:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115372; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xjU2n1bweA2ySl2CUDbxxeVa4aZMmSJDQ4vmZ7qDwtw=; b=PdOYvlyeWfEocLeCl9xMvzdpMMhgihQVXKbdPzvdnPMhaSXgRWWq2tticbiwOFZBSdbqt0 Jley7QaaeAJUxUJgkhBAj1OmZeWnaOYCRvOt5Vb1ffEnF5Jn3DFopUO9+wsQ+wpOSAeRnh cF942VY5DaZICuQMIo6/oD0bweKBpPM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-383-sd5nexMQPfKmzhoxc8nX3A-1; Mon, 19 Oct 2020 09:49:27 -0400 X-MC-Unique: sd5nexMQPfKmzhoxc8nX3A-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3A709802B60; Mon, 19 Oct 2020 13:49:25 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id ECBAC55775; Mon, 19 Oct 2020 13:49:22 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Somnath Kotur Date: Mon, 19 Oct 2020 15:48:45 +0200 Message-Id: <20201019134858.32507-6-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 05/18] net/bnxt: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: Somnath Kotur --- v4: Updated enum name. --- drivers/net/bnxt/bnxt_ethdev.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 6c1236953a..32318cfd00 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "bnxt.h" #include "bnxt_filter.h" @@ -1169,7 +1170,8 @@ bnxt_receive_function(struct rte_eth_dev *eth_dev) DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_RSS_HASH | DEV_RX_OFFLOAD_VLAN_FILTER)) && - !BNXT_TRUFLOW_EN(bp) && BNXT_NUM_ASYNC_CPR(bp)) { + !BNXT_TRUFLOW_EN(bp) && BNXT_NUM_ASYNC_CPR(bp) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { PMD_DRV_LOG(INFO, "Using vector mode receive for port %d\n", eth_dev->data->port_id); bp->flags |= BNXT_FLAG_RX_VECTOR_PKT_MODE; @@ -1202,7 +1204,8 @@ bnxt_transmit_function(__rte_unused struct rte_eth_dev *eth_dev) */ if (!eth_dev->data->scattered_rx && !(offloads & ~DEV_TX_OFFLOAD_MBUF_FAST_FREE) && - !BNXT_TRUFLOW_EN(bp)) { + !BNXT_TRUFLOW_EN(bp) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { PMD_DRV_LOG(INFO, "Using vector mode transmit for port %d\n", eth_dev->data->port_id); return bnxt_xmit_pkts_vec; From patchwork Mon Oct 19 13:48:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81367 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0316DA04DC; Mon, 19 Oct 2020 15:51:15 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E7305E241; Mon, 19 Oct 2020 15:49:35 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id 34033E23F for ; Mon, 19 Oct 2020 15:49:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115371; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=487y0oq+CgPAubGbxIx1cKePzMos+LW5fmJ/JLYVcC8=; b=NDcQdW989sCKUFvlSA/1perQPb+0zxe0VGZ5//fTaJR5P2p+3sN3q+p5MSPciIn7rjYa8i TzctVhWo3Hkh6htD8v0wkWGBQzgrvsvNFvNLhOSB/0tvzlpAgSkTH7Ndfitb1NbUCKEtyE uzs3fyjEFsT+7jHzRvS0yR1tKOkBEyI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-160-91bT8dOaP36bjZVyU0PRXw-1; Mon, 19 Oct 2020 09:49:29 -0400 X-MC-Unique: 91bT8dOaP36bjZVyU0PRXw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D78681020900; Mon, 19 Oct 2020 13:49:27 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 97AC37513C; Mon, 19 Oct 2020 13:49:25 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Hyong Youb Kim Date: Mon, 19 Oct 2020 15:48:46 +0200 Message-Id: <20201019134858.32507-7-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 06/18] net/enic: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: Hyong Youb Kim --- v4: Updated enum name. --- drivers/net/enic/enic_rxtx_vec_avx2.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/net/enic/enic_rxtx_vec_avx2.c b/drivers/net/enic/enic_rxtx_vec_avx2.c index 676b9f5fdb..d39021f30c 100644 --- a/drivers/net/enic/enic_rxtx_vec_avx2.c +++ b/drivers/net/enic/enic_rxtx_vec_avx2.c @@ -5,6 +5,7 @@ #include #include +#include #include "enic_compat.h" #include "rq_enet_desc.h" @@ -821,7 +822,8 @@ enic_use_vector_rx_handler(struct rte_eth_dev *eth_dev) fconf = ð_dev->data->dev_conf.fdir_conf; if (fconf->mode != RTE_FDIR_MODE_NONE) return false; - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2)) { + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) { ENICPMD_LOG(DEBUG, " use the non-scatter avx2 Rx handler"); eth_dev->rx_pkt_burst = &enic_noscatter_vec_recv_pkts; enic->use_noscatter_vec_rx_handler = 1; From patchwork Mon Oct 19 13:48:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81369 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4780DA04DC; Mon, 19 Oct 2020 15:52:02 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 408F7D019; Mon, 19 Oct 2020 15:49:40 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id E34E7E249 for ; Mon, 19 Oct 2020 15:49:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dUuqhkl62V8/DbSwDXas9k/eVmJdyHE9Ad7bh1kAMtA=; b=G4BAxwraSxLyykQ0T5NMMvrDA9XO/l5hWkAs1DhUlphTMBt3+bdiJuJ+Og6WjlwxRNtnoS aqxXyLorAmXrcdTp1zXe2bGywOVU95eX7Ij5BkNTYyeoIf2zmJC1E3jzFS3Wn0OHGu8Ms/ pRM6j/4qXn0x4t5OERxaAXJVLa7k0BA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-290-FQFyIaDjP7WMCAFTre3kMw-1; Mon, 19 Oct 2020 09:49:32 -0400 X-MC-Unique: FQFyIaDjP7WMCAFTre3kMw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8E721102091B; Mon, 19 Oct 2020 13:49:30 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 41C1555775; Mon, 19 Oct 2020 13:49:28 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Qi Zhang Date: Mon, 19 Oct 2020 15:48:47 +0200 Message-Id: <20201019134858.32507-8-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 07/18] net/fm10k: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: Qi Zhang --- v4: Updated enum name. --- drivers/net/fm10k/fm10k_ethdev.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c index dc2979bdd2..c187088a33 100644 --- a/drivers/net/fm10k/fm10k_ethdev.c +++ b/drivers/net/fm10k/fm10k_ethdev.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "fm10k.h" #include "base/fm10k_api.h" @@ -2939,7 +2940,8 @@ fm10k_set_tx_function(struct rte_eth_dev *dev) if (rte_eal_process_type() != RTE_PROC_PRIMARY) { /* primary process has set the ftag flag and offloads */ txq = dev->data->tx_queues[0]; - if (fm10k_tx_vec_condition_check(txq)) { + if (fm10k_tx_vec_condition_check(txq) || + rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_128) { dev->tx_pkt_burst = fm10k_xmit_pkts; dev->tx_pkt_prepare = fm10k_prep_pkts; PMD_INIT_LOG(DEBUG, "Use regular Tx func"); @@ -2958,7 +2960,8 @@ fm10k_set_tx_function(struct rte_eth_dev *dev) txq = dev->data->tx_queues[i]; txq->tx_ftag_en = tx_ftag_en; /* Check if Vector Tx is satisfied */ - if (fm10k_tx_vec_condition_check(txq)) + if (fm10k_tx_vec_condition_check(txq) || + rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_128) use_sse = 0; } @@ -2992,7 +2995,8 @@ fm10k_set_rx_function(struct rte_eth_dev *dev) * conditions to be met. */ if (!fm10k_rx_vec_condition_check(dev) && - dev_info->rx_vec_allowed && !rx_ftag_en) { + dev_info->rx_vec_allowed && !rx_ftag_en && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { if (dev->data->scattered_rx) dev->rx_pkt_burst = fm10k_recv_scattered_pkts_vec; else From patchwork Mon Oct 19 13:48:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81370 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3AA35A04DC; Mon, 19 Oct 2020 15:52:29 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E2F31E25C; Mon, 19 Oct 2020 15:49:43 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by dpdk.org (Postfix) with ESMTP id 89301E25A for ; Mon, 19 Oct 2020 15:49:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7XpJFJtjgOTXoXEa0u9PvknftxJPozLkwz5Yc6U62CY=; b=QfaOhW+fINlG2JkEG/VaFnHdX5LtsGeNtJZnIKZtTuOdAyltNseSVYTZhWkAAeGaV7Wwrb fUic/wBTQOZFIUAZyDsDMTipkF5R6Vpw75CCJvtAjfDecLkfSp+GHb0DzqIz6h283AYgXx ehqxpcNJurrqMTyCWzMrUagGWJeTfc4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-191-WlFQy_r5Ov6r0MbtWSIgTg-1; Mon, 19 Oct 2020 09:49:34 -0400 X-MC-Unique: WlFQy_r5Ov6r0MbtWSIgTg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0DE5A18BE163; Mon, 19 Oct 2020 13:49:33 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id F220955775; Mon, 19 Oct 2020 13:49:30 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com Date: Mon, 19 Oct 2020 15:48:48 +0200 Message-Id: <20201019134858.32507-9-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 08/18] net/iavf: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Reviewed-by: Bruce Richardson --- v4: Updated enum name. --- drivers/net/iavf/iavf_rxtx.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/drivers/net/iavf/iavf_rxtx.c b/drivers/net/iavf/iavf_rxtx.c index 1b0efe0433..edb2dc3ca4 100644 --- a/drivers/net/iavf/iavf_rxtx.c +++ b/drivers/net/iavf/iavf_rxtx.c @@ -23,6 +23,7 @@ #include #include #include +#include #include "iavf.h" #include "iavf_rxtx.h" @@ -2104,14 +2105,16 @@ iavf_set_rx_function(struct rte_eth_dev *dev) int i; bool use_avx2 = false; - if (!iavf_rx_vec_dev_check(dev)) { + if (!iavf_rx_vec_dev_check(dev) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { for (i = 0; i < dev->data->nb_rx_queues; i++) { rxq = dev->data->rx_queues[i]; (void)iavf_rxq_vec_setup(rxq); } - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1 || - rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) + if ((rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1 || + rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) use_avx2 = true; if (dev->data->scattered_rx) { @@ -2177,7 +2180,8 @@ iavf_set_tx_function(struct rte_eth_dev *dev) int i; bool use_avx2 = false; - if (!iavf_tx_vec_dev_check(dev)) { + if (!iavf_tx_vec_dev_check(dev) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { for (i = 0; i < dev->data->nb_tx_queues; i++) { txq = dev->data->tx_queues[i]; if (!txq) @@ -2185,8 +2189,9 @@ iavf_set_tx_function(struct rte_eth_dev *dev) iavf_txq_vec_setup(txq); } - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1 || - rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) + if ((rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1 || + rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) use_avx2 = true; PMD_DRV_LOG(DEBUG, "Using %sVector Tx (port %d).", From patchwork Mon Oct 19 13:48:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81371 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 565B6A04DC; Mon, 19 Oct 2020 15:52:56 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E0422E275; Mon, 19 Oct 2020 15:49:46 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id 37FF0E260 for ; Mon, 19 Oct 2020 15:49:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115381; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uFWm6BSMhEuuiW0GIMUIIQcCgOv/7OU8KHFcZMc66FM=; b=VlqXHNORD+MIo1EXZXTY1eyaPPBHwtOsiTn/rbrUto9jb2XM95k5vLK8pBwW19h35hjcQK qqy3oIpgFT0oq5UpFMjSBYQJhetsMRoKYdw2OAJ3UfoQ93taCs8jUzhG55NAs838XJ8HW5 vvRFmgAo87ogzEVjKub1RpJXVKG9EZc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-368-tkFIfcQLN-C2IX-O0ZQIzw-1; Mon, 19 Oct 2020 09:49:37 -0400 X-MC-Unique: tkFIfcQLN-C2IX-O0ZQIzw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A6560879515; Mon, 19 Oct 2020 13:49:35 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6C2AF55793; Mon, 19 Oct 2020 13:49:33 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Qi Zhang Date: Mon, 19 Oct 2020 15:48:49 +0200 Message-Id: <20201019134858.32507-10-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 09/18] net/ice: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: Qi Zhang --- v4: Updated enum name. --- drivers/net/ice/ice_rxtx.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c index 79e6df11f4..ee576c362a 100644 --- a/drivers/net/ice/ice_rxtx.c +++ b/drivers/net/ice/ice_rxtx.c @@ -4,6 +4,7 @@ #include #include +#include #include "rte_pmd_ice.h" #include "ice_rxtx.h" @@ -2989,7 +2990,8 @@ ice_set_rx_function(struct rte_eth_dev *dev) bool use_avx2 = false; if (rte_eal_process_type() == RTE_PROC_PRIMARY) { - if (!ice_rx_vec_dev_check(dev) && ad->rx_bulk_alloc_allowed) { + if (!ice_rx_vec_dev_check(dev) && ad->rx_bulk_alloc_allowed && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { ad->rx_vec_allowed = true; for (i = 0; i < dev->data->nb_rx_queues; i++) { rxq = dev->data->rx_queues[i]; @@ -2999,8 +3001,9 @@ ice_set_rx_function(struct rte_eth_dev *dev) } } - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1 || - rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) + if ((rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1 || + rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) use_avx2 = true; } else { @@ -3167,7 +3170,8 @@ ice_set_tx_function(struct rte_eth_dev *dev) bool use_avx2 = false; if (rte_eal_process_type() == RTE_PROC_PRIMARY) { - if (!ice_tx_vec_dev_check(dev)) { + if (!ice_tx_vec_dev_check(dev) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { ad->tx_vec_allowed = true; for (i = 0; i < dev->data->nb_tx_queues; i++) { txq = dev->data->tx_queues[i]; @@ -3177,8 +3181,9 @@ ice_set_tx_function(struct rte_eth_dev *dev) } } - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1 || - rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) + if ((rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1 || + rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) use_avx2 = true; } else { From patchwork Mon Oct 19 13:48:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81372 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8AF78A04DC; Mon, 19 Oct 2020 15:53:17 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8A98AE28C; Mon, 19 Oct 2020 15:49:49 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id D1FDCE27F for ; Mon, 19 Oct 2020 15:49:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115385; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E+Z+kGYKdISHGmjZLrO2jt/Q7fPPnFHlOP1U0IoprA8=; b=Z4qXXZ6jUH9yEMdw9kjWCmlqF3cehxDhSHI+FvI6xU5sndOzd059nIvlwrsOfXu6D99T8+ jRctkzHFnH5pBbjZ597ser2n1gWElU3Rx8CdMtDOM7ibHjUZVtBHEVQ6LBT2vULb2j7O2y 3tPCzIsQ1HFDQYCpPz9bEdnCgJX7kCc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-148-tqHwjreAPkmCU1vNrKnvcQ-1; Mon, 19 Oct 2020 09:49:41 -0400 X-MC-Unique: tqHwjreAPkmCU1vNrKnvcQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 54C6718C89E7; Mon, 19 Oct 2020 13:49:38 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 12D2B55775; Mon, 19 Oct 2020 13:49:35 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Haiyue Wang Date: Mon, 19 Oct 2020 15:48:50 +0200 Message-Id: <20201019134858.32507-11-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 10/18] net/ixgbe: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: Konstantin Ananyev Reviewed-by: Haiyue Wang --- v4: - Updated enum name. - Moved placement of condition check. - Added condition check to tx cleanup path selection. --- drivers/net/ixgbe/ixgbe_rxtx.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c index 29d385c062..ce589b97a8 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx.c +++ b/drivers/net/ixgbe/ixgbe_rxtx.c @@ -42,6 +42,7 @@ #include #include #include +#include #include "ixgbe_logs.h" #include "base/ixgbe_api.h" @@ -2405,6 +2406,7 @@ ixgbe_dev_tx_done_cleanup(void *tx_queue, uint32_t free_cnt) #endif txq->tx_rs_thresh >= RTE_PMD_IXGBE_TX_MAX_BURST) { if (txq->tx_rs_thresh <= RTE_IXGBE_TX_MAX_FREE_BUF_SZ && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128 && (rte_eal_process_type() != RTE_PROC_PRIMARY || txq->sw_ring_v != NULL)) { return ixgbe_tx_done_cleanup_vec(txq, free_cnt); @@ -2503,6 +2505,7 @@ ixgbe_set_tx_function(struct rte_eth_dev *dev, struct ixgbe_tx_queue *txq) PMD_INIT_LOG(DEBUG, "Using simple tx code path"); dev->tx_pkt_prepare = NULL; if (txq->tx_rs_thresh <= RTE_IXGBE_TX_MAX_FREE_BUF_SZ && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128 && (rte_eal_process_type() != RTE_PROC_PRIMARY || ixgbe_txq_vec_setup(txq) == 0)) { PMD_INIT_LOG(DEBUG, "Vector tx enabled."); @@ -4744,7 +4747,8 @@ ixgbe_set_rx_function(struct rte_eth_dev *dev) * conditions to be met and Rx Bulk Allocation should be allowed. */ if (ixgbe_rx_vec_dev_conf_condition_check(dev) || - !adapter->rx_bulk_alloc_allowed) { + !adapter->rx_bulk_alloc_allowed || + rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_128) { PMD_INIT_LOG(DEBUG, "Port[%d] doesn't meet Vector Rx " "preconditions", dev->data->port_id); From patchwork Mon Oct 19 13:48:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81373 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F1A68A04DC; Mon, 19 Oct 2020 15:53:40 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D5FEFE299; Mon, 19 Oct 2020 15:49:59 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by dpdk.org (Postfix) with ESMTP id E7382E293 for ; Mon, 19 Oct 2020 15:49:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115396; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VAjpzEZUM1NPvzH1laNX8o7sylth34d/1o06mDKMS00=; b=W1vniDS2VItEF2/YRU/Lp+6VaqYcQbx534c3Qa41EhzLO+t2UR/PyvKqnBBZf4LPa5QM6/ 8noEJdVcb9RyeM/608fP3uwkKra12OmucJf9B37ZJcOEcXSfYn+NQaie23OSKg+g9E/ciw luzk2rbWC1E+yuXC5bCwsQl11T5loMY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-141-ukiUhfLmNwKTQA4FyNQHaw-1; Mon, 19 Oct 2020 09:49:52 -0400 X-MC-Unique: ukiUhfLmNwKTQA4FyNQHaw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 38670835BBF; Mon, 19 Oct 2020 13:49:41 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id B4DE27514B; Mon, 19 Oct 2020 13:49:38 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Viacheslav Ovsiienko Date: Mon, 19 Oct 2020 15:48:51 +0200 Message-Id: <20201019134858.32507-12-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 11/18] net/mlx5: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: Viacheslav Ovsiienko --- v4: Updated enum name. v2: Moved check for max bitwidth into existing check vec support function. --- drivers/net/mlx5/mlx5_rxtx_vec.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.c b/drivers/net/mlx5/mlx5_rxtx_vec.c index 711dcd35fa..f083038682 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec.c +++ b/drivers/net/mlx5/mlx5_rxtx_vec.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -148,6 +149,8 @@ mlx5_check_vec_rx_support(struct rte_eth_dev *dev) struct mlx5_priv *priv = dev->data->dev_private; uint32_t i; + if (rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_128) + return -ENOTSUP; if (!priv->config.rx_vec_en) return -ENOTSUP; if (mlx5_mprq_enabled(dev)) From patchwork Mon Oct 19 13:48:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81374 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E921A04DC; Mon, 19 Oct 2020 15:54:03 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6C495E27D; Mon, 19 Oct 2020 15:50:28 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id AF69ECB06 for ; Mon, 19 Oct 2020 15:50:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115426; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q8g2XxZ5q3cl6dXyCpgQzBPZFVJvTeSnaKKdBJpX4EI=; b=Y7extFeRIYHDmqhoJXotl6JuJuvBMEqXMVLiOnE0b/rZ79tHfPcT1lX7WdFqZAGoMmGDt+ 05elEy73vLf3sqMIMrRPiZ4zm7pw7HC+El3jpJMFxJifL9/VKCCnqUXEC1OPNZpng4G+RE x9VRR45+w+gxCO+no0bEtbmWILFlMO0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-530-AEVtwmmgPvmNlk5RhCgZ_A-1; Mon, 19 Oct 2020 09:50:22 -0400 X-MC-Unique: AEVtwmmgPvmNlk5RhCgZ_A-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D503B8044BC; Mon, 19 Oct 2020 13:49:47 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 720277515D; Mon, 19 Oct 2020 13:49:41 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Chenbo Xia , Maxime Coquelin Date: Mon, 19 Oct 2020 15:48:52 +0200 Message-Id: <20201019134858.32507-13-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 12/18] net/virtio: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Reviewed-by: Chenbo Xia Acked-by: Maxime Coquelin --- v4: Updated enum name. v3: Moved max SIMD bitwidth check to configure function with other vec support checks. --- drivers/net/virtio/virtio_ethdev.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index 516c277f9c..6c233b75ba 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include @@ -2312,7 +2313,8 @@ virtio_dev_configure(struct rte_eth_dev *dev) if ((hw->use_vec_rx || hw->use_vec_tx) && (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) || !vtpci_with_feature(hw, VIRTIO_F_IN_ORDER) || - !vtpci_with_feature(hw, VIRTIO_F_VERSION_1))) { + !vtpci_with_feature(hw, VIRTIO_F_VERSION_1) || + rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_512)) { PMD_DRV_LOG(INFO, "disabled packed ring vectorized path for requirements not met"); hw->use_vec_rx = 0; @@ -2365,6 +2367,12 @@ virtio_dev_configure(struct rte_eth_dev *dev) "disabled split ring vectorized rx for offloading enabled"); hw->use_vec_rx = 0; } + + if (rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_128) { + PMD_DRV_LOG(INFO, + "disabled split ring vectorized rx, max SIMD bitwidth too low"); + hw->use_vec_rx = 0; + } } } From patchwork Mon Oct 19 13:48:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81376 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EE5D3A04DC; Mon, 19 Oct 2020 15:54:50 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C2955E2AD; Mon, 19 Oct 2020 15:50:45 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by dpdk.org (Postfix) with ESMTP id C0B54C9D2 for ; Mon, 19 Oct 2020 15:50:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115440; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RYi1EFBX3tp4XAXCdnZ+HrQoZnxUONb0VH88O3T08AU=; b=D3ImLZWG6AHHYJ+Bx5CwEveNjizPMjAom/L6My0Pzn6XO/ZHPVRYImCxHj6NM/KfBRKszx sVESkuNPHrM7RVq9sBFV9M0fgXOJysEkhR8Bp41DeICRbKanlzDZXs9MxD5uKTRWkSquh8 OEVB9b3csvXMszg8sU2ab0g26idr9G4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-313-28cg5I7cOiugKweRZ835iw-1; Mon, 19 Oct 2020 09:50:35 -0400 X-MC-Unique: 28cg5I7cOiugKweRZ835iw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 003C289BBBF; Mon, 19 Oct 2020 13:49:51 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 40FA67D4FA; Mon, 19 Oct 2020 13:49:48 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, David Hunt Date: Mon, 19 Oct 2020 15:48:53 +0200 Message-Id: <20201019134858.32507-14-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 13/18] distributor: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: David Hunt --- v4: Updated enum name. v3: Moved max SIMD bitwidth check to configure function with other vec support checks. --- lib/librte_distributor/rte_distributor.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index ef34facba6..07e385a259 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -14,6 +14,7 @@ #include #include #include +#include #include "rte_distributor.h" #include "rte_distributor_single.h" @@ -762,7 +763,8 @@ rte_distributor_create(const char *name, d->dist_match_fn = RTE_DIST_MATCH_SCALAR; #if defined(RTE_ARCH_X86) - d->dist_match_fn = RTE_DIST_MATCH_VECTOR; + if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) + d->dist_match_fn = RTE_DIST_MATCH_VECTOR; #endif /* From patchwork Mon Oct 19 13:48:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81375 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4FD84A04DC; Mon, 19 Oct 2020 15:54:27 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3241CCABE; Mon, 19 Oct 2020 15:50:43 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by dpdk.org (Postfix) with ESMTP id 1DAA1CABE for ; Mon, 19 Oct 2020 15:50:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115439; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rQdtENCqdfAelcZYosACVy255xQQrg/WBJ5EsjMORvk=; b=W3YKaEt5V3tfJxR3X87NXre/gKZf9FDmw1p9jFo8h5K3yHfztFrYkNFDyjj2fAQq5gprst FzLtQck/ckDYFvnnMx9Vg9vP0wzcuat/5rRb1X/KbSYxvKL4K5umT6xh5/1AFsQNSwZEE7 52UMDz1dAagIUCx9+2Kin+WLAYDDqDc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-577-KIsNc9sPOWerTtgvu4MhwQ-1; Mon, 19 Oct 2020 09:50:36 -0400 X-MC-Unique: KIsNc9sPOWerTtgvu4MhwQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 791138914EA; Mon, 19 Oct 2020 13:49:53 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 260A57D51C; Mon, 19 Oct 2020 13:49:50 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Yipeng Wang Date: Mon, 19 Oct 2020 15:48:54 +0200 Message-Id: <20201019134858.32507-15-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 14/18] member: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: Yipeng Wang --- v4: Updated enum name. --- lib/librte_member/rte_member_ht.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c index 3ea293a094..a85561b472 100644 --- a/lib/librte_member/rte_member_ht.c +++ b/lib/librte_member/rte_member_ht.c @@ -7,6 +7,7 @@ #include #include #include +#include #include "rte_member.h" #include "rte_member_ht.h" @@ -113,7 +114,8 @@ rte_member_create_ht(struct rte_member_setsum *ss, } #if defined(RTE_ARCH_X86) if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) && - RTE_MEMBER_BUCKET_ENTRIES == 16) + RTE_MEMBER_BUCKET_ENTRIES == 16 && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) ss->sig_cmp_fn = RTE_MEMBER_COMPARE_AVX2; else #endif From patchwork Mon Oct 19 13:48:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81378 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 26A62A04DC; Mon, 19 Oct 2020 15:55:42 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0BE0BE2CA; Mon, 19 Oct 2020 15:51:07 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id D4339E2C8 for ; Mon, 19 Oct 2020 15:51:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115463; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RxSTpIhITo2yuWzc3NiNKEvb6HqRCo3BvG9hy64ssZw=; b=Ts38VCrGpyncdwoW++Rc8y5c4QDBmRSaM/QDQjLLllGYBT+NKaoUproftFahJPLta72KOL BsKyWvYHugQJ7c3SnbBrUhlzEe/U1BtgcxrvfjNG8Ew8Mu7pazd+rreIQapk89t0GX55hB sBdK11nU8KV2U5jxdpt4amjxVcmfVIY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-284-dGGoPvluNCiG0UMRpDhfqA-1; Mon, 19 Oct 2020 09:50:59 -0400 X-MC-Unique: dGGoPvluNCiG0UMRpDhfqA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 16DAB88F300; Mon, 19 Oct 2020 13:49:56 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id D2A927D4FE; Mon, 19 Oct 2020 13:49:53 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Yipeng Wang Date: Mon, 19 Oct 2020 15:48:55 +0200 Message-Id: <20201019134858.32507-16-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 15/18] efd: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. Signed-off-by: Ciara Power Acked-by: Yipeng Wang --- v4: Updated enum name. --- lib/librte_efd/rte_efd.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/lib/librte_efd/rte_efd.c b/lib/librte_efd/rte_efd.c index 6a799556d4..ec3a4cd58e 100644 --- a/lib/librte_efd/rte_efd.c +++ b/lib/librte_efd/rte_efd.c @@ -21,6 +21,7 @@ #include #include #include +#include #include "rte_efd.h" #if defined(RTE_ARCH_X86) @@ -645,7 +646,9 @@ rte_efd_create(const char *name, uint32_t max_num_rules, uint32_t key_len, * For less than 4 bits, scalar function performs better * than vectorised version */ - if (RTE_EFD_VALUE_NUM_BITS > 3 && rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2)) + if (RTE_EFD_VALUE_NUM_BITS > 3 + && rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) + && rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) table->lookup_fn = EFD_LOOKUP_AVX2; else #endif @@ -655,7 +658,8 @@ rte_efd_create(const char *name, uint32_t max_num_rules, uint32_t key_len, * than vectorised version */ if (RTE_EFD_VALUE_NUM_BITS > 16 && - rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON)) + rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) table->lookup_fn = EFD_LOOKUP_NEON; else #endif From patchwork Mon Oct 19 13:48:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81380 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 059E3A04DC; Mon, 19 Oct 2020 15:56:25 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 098A9E2C0; Mon, 19 Oct 2020 15:51:14 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by dpdk.org (Postfix) with ESMTP id 70EFFE2C0 for ; Mon, 19 Oct 2020 15:51:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115470; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DrMMO9q0l0zsFcoui7pBdT0Q31AIND9SdkxTeb7WfOg=; b=MBapKjPdgHEBRb5YYGSksnbinTULRtXZuCKLXY+OfZbBeepHPehh0AdANKlBWT7VWPIekN etOalSeNOLfTb1jhngjOA8/QFQQUr3DTqgO2qYxHXQ2SyYSRQE27PJ5cMnmiXtQ1b6Ttot lU0PldJYcLsjbsSKLIsVyM/peynJ7MU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-175-0Ra3QuIcNoa1sSp5rtVKYw-1; Mon, 19 Oct 2020 09:51:04 -0400 X-MC-Unique: 0Ra3QuIcNoa1sSp5rtVKYw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 101B68B8D34; Mon, 19 Oct 2020 13:49:59 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 75E2555775; Mon, 19 Oct 2020 13:49:56 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Jasvinder Singh , Olivier Matz Date: Mon, 19 Oct 2020 15:48:56 +0200 Message-Id: <20201019134858.32507-17-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 16/18] net: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. The vector path was initially chosen in RTE_INIT, however this is no longer suitable as we cannot check the max SIMD bitwidth at that time. Default handlers are now chosen on initialisation, these default handlers are used the first time the crc calc is called, and they set the suitable handlers to be used going forward. Suggested-by: Jasvinder Singh Suggested-by: Olivier Matz Signed-off-by: Ciara Power Acked-by: Jasvinder Singh --- v7: Removed unnecessary log variable. v6: - Moved log variable and macro to c file instead of public header. - Added the max_simd_bitwidth condition check to the recently added handler helper functions. - Modified default handlers to follow the approach of the set alg function. v4: - Added default handlers to be set at RTE_INIT time, rather than choosing scalar handlers. - Modified logging. - Updated enum name. v3: - Moved choosing vector paths out of RTE_INIT. - Moved checking max_simd_bitwidth into the set_alg function. --- lib/librte_net/rte_net_crc.c | 117 +++++++++++++++++++++++++---------- 1 file changed, 86 insertions(+), 31 deletions(-) diff --git a/lib/librte_net/rte_net_crc.c b/lib/librte_net/rte_net_crc.c index 32a3665908..1fe58f7c75 100644 --- a/lib/librte_net/rte_net_crc.c +++ b/lib/librte_net/rte_net_crc.c @@ -9,6 +9,9 @@ #include #include #include +#include +#include +#include #include "net_crc.h" @@ -22,6 +25,12 @@ static uint32_t crc32_eth_lut[CRC_LUT_SIZE]; static uint32_t crc16_ccitt_lut[CRC_LUT_SIZE]; +static uint32_t +rte_crc16_ccitt_default_handler(const uint8_t *data, uint32_t data_len); + +static uint32_t +rte_crc32_eth_default_handler(const uint8_t *data, uint32_t data_len); + static uint32_t rte_crc16_ccitt_handler(const uint8_t *data, uint32_t data_len); @@ -31,7 +40,12 @@ rte_crc32_eth_handler(const uint8_t *data, uint32_t data_len); typedef uint32_t (*rte_net_crc_handler)(const uint8_t *data, uint32_t data_len); -static const rte_net_crc_handler *handlers; +static rte_net_crc_handler handlers_default[] = { + [RTE_NET_CRC16_CCITT] = rte_crc16_ccitt_default_handler, + [RTE_NET_CRC32_ETH] = rte_crc32_eth_default_handler, +}; + +static const rte_net_crc_handler *handlers = handlers_default; static const rte_net_crc_handler handlers_scalar[] = { [RTE_NET_CRC16_CCITT] = rte_crc16_ccitt_handler, @@ -56,6 +70,14 @@ static const rte_net_crc_handler handlers_neon[] = { }; #endif +static uint16_t max_simd_bitwidth; + +#define NET_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, libnet_logtype, "%s(): " fmt "\n", \ + __func__, ## args) + +RTE_LOG_REGISTER(libnet_logtype, lib.net, INFO); + /* Scalar handling */ /** @@ -155,22 +177,21 @@ static const rte_net_crc_handler * avx512_vpclmulqdq_get_handlers(void) { #ifdef CC_X86_64_AVX512_VPCLMULQDQ_SUPPORT - if (AVX512_VPCLMULQDQ_CPU_SUPPORTED) + if (AVX512_VPCLMULQDQ_CPU_SUPPORTED && + max_simd_bitwidth >= RTE_VECT_SIMD_512) return handlers_avx512; #endif + NET_LOG(INFO, "Requirements not met, can't use AVX512\n"); return NULL; } -static uint8_t +static void avx512_vpclmulqdq_init(void) { #ifdef CC_X86_64_AVX512_VPCLMULQDQ_SUPPORT - if (AVX512_VPCLMULQDQ_CPU_SUPPORTED) { + if (AVX512_VPCLMULQDQ_CPU_SUPPORTED) rte_net_crc_avx512_init(); - return 1; - } #endif - return 0; } /* SSE4.2/PCLMULQDQ handling */ @@ -182,22 +203,21 @@ static const rte_net_crc_handler * sse42_pclmulqdq_get_handlers(void) { #ifdef CC_X86_64_SSE42_PCLMULQDQ_SUPPORT - if (SSE42_PCLMULQDQ_CPU_SUPPORTED) + if (SSE42_PCLMULQDQ_CPU_SUPPORTED && + max_simd_bitwidth >= RTE_VECT_SIMD_128) return handlers_sse42; #endif + NET_LOG(INFO, "Requirements not met, can't use SSE\n"); return NULL; } -static uint8_t +static void sse42_pclmulqdq_init(void) { #ifdef CC_X86_64_SSE42_PCLMULQDQ_SUPPORT - if (SSE42_PCLMULQDQ_CPU_SUPPORTED) { + if (SSE42_PCLMULQDQ_CPU_SUPPORTED) rte_net_crc_sse42_init(); - return 1; - } #endif - return 0; } /* NEON/PMULL handling */ @@ -209,22 +229,63 @@ static const rte_net_crc_handler * neon_pmull_get_handlers(void) { #ifdef CC_ARM64_NEON_PMULL_SUPPORT - if (NEON_PMULL_CPU_SUPPORTED) + if (NEON_PMULL_CPU_SUPPORTED && + max_simd_bitwidth >= RTE_VECT_SIMD_128) return handlers_neon; #endif + NET_LOG(INFO, "Requirements not met, can't use NEON\n"); return NULL; } -static uint8_t +static void neon_pmull_init(void) { #ifdef CC_ARM64_NEON_PMULL_SUPPORT - if (NEON_PMULL_CPU_SUPPORTED) { + if (NEON_PMULL_CPU_SUPPORTED) rte_net_crc_neon_init(); - return 1; - } #endif - return 0; +} + +/* Default handling */ + +static uint32_t +rte_crc16_ccitt_default_handler(const uint8_t *data, uint32_t data_len) +{ + handlers = NULL; + if (max_simd_bitwidth == 0) + max_simd_bitwidth = rte_vect_get_max_simd_bitwidth(); + + handlers = avx512_vpclmulqdq_get_handlers(); + if (handlers != NULL) + return handlers[RTE_NET_CRC16_CCITT](data, data_len); + handlers = sse42_pclmulqdq_get_handlers(); + if (handlers != NULL) + return handlers[RTE_NET_CRC16_CCITT](data, data_len); + handlers = neon_pmull_get_handlers(); + if (handlers != NULL) + return handlers[RTE_NET_CRC16_CCITT](data, data_len); + handlers = handlers_scalar; + return handlers[RTE_NET_CRC16_CCITT](data, data_len); +} + +static uint32_t +rte_crc32_eth_default_handler(const uint8_t *data, uint32_t data_len) +{ + handlers = NULL; + if (max_simd_bitwidth == 0) + max_simd_bitwidth = rte_vect_get_max_simd_bitwidth(); + + handlers = avx512_vpclmulqdq_get_handlers(); + if (handlers != NULL) + return handlers[RTE_NET_CRC32_ETH](data, data_len); + handlers = sse42_pclmulqdq_get_handlers(); + if (handlers != NULL) + return handlers[RTE_NET_CRC32_ETH](data, data_len); + handlers = neon_pmull_get_handlers(); + if (handlers != NULL) + return handlers[RTE_NET_CRC32_ETH](data, data_len); + handlers = handlers_scalar; + return handlers[RTE_NET_CRC32_ETH](data, data_len); } /* Public API */ @@ -233,6 +294,8 @@ void rte_net_crc_set_alg(enum rte_net_crc_alg alg) { handlers = NULL; + if (max_simd_bitwidth == 0) + max_simd_bitwidth = rte_vect_get_max_simd_bitwidth(); switch (alg) { case RTE_NET_CRC_AVX512: @@ -270,19 +333,11 @@ rte_net_crc_calc(const void *data, return ret; } -/* Select highest available crc algorithm as default one */ +/* Call initialisation helpers for all crc algorithm handlers */ RTE_INIT(rte_net_crc_init) { - enum rte_net_crc_alg alg = RTE_NET_CRC_SCALAR; - rte_net_crc_scalar_init(); - - if (sse42_pclmulqdq_init()) - alg = RTE_NET_CRC_SSE42; - if (avx512_vpclmulqdq_init()) - alg = RTE_NET_CRC_AVX512; - if (neon_pmull_init()) - alg = RTE_NET_CRC_NEON; - - rte_net_crc_set_alg(alg); + sse42_pclmulqdq_init(); + avx512_vpclmulqdq_init(); + neon_pmull_init(); } From patchwork Mon Oct 19 13:48:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81377 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E45C1A04DC; Mon, 19 Oct 2020 15:55:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 344DCE23C; Mon, 19 Oct 2020 15:50:57 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id D0D02BE19 for ; Mon, 19 Oct 2020 15:50:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115454; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qu7WxxQsp+6o4P1xORKEkeaG7wafPCcrUJCljdzOBI8=; b=ITOTrm7Z7lbpUTqi1XnWEHK+hI7B0CIMlufYeHcZc3sayqkVWmPry0Otys9s+ORoA/FbYF 1Uw6QGt8fsn0YRhSeKaPxG3wNZbN9x5UTB1OcLuHkZ04ytUXmOjsXdIlj5nkVafAujEwSZ bEdLmnpsoZeMCLV9Rka0gELYSHOupZk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-308-yFSo9a5kPXap8k0nncq0TA-1; Mon, 19 Oct 2020 09:50:50 -0400 X-MC-Unique: yFSo9a5kPXap8k0nncq0TA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A8CE484BA66; Mon, 19 Oct 2020 13:50:01 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6DCD255775; Mon, 19 Oct 2020 13:49:59 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com, Nithin Dabilpuram Date: Mon, 19 Oct 2020 15:48:57 +0200 Message-Id: <20201019134858.32507-18-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 17/18] node: choose vector path at runtime X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing the vector path, max SIMD bitwidth is now checked to ensure the vector path is suitable. To do this, the scalar function is chosen by default in the struct, but at node initialisation time, this function pointer is updated to the vector version if supported, and if it is within the max SIMD bitwidth limit. Signed-off-by: Ciara Power Acked-by: Nithin Dabilpuram --- v6: - Removed generic process function. - Change the process function pointer at node init time to vector function if suitable. --- lib/librte_node/ip4_lookup.c | 15 ++++++++++----- lib/librte_node/ip4_lookup_neon.h | 2 +- lib/librte_node/ip4_lookup_sse.h | 2 +- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/lib/librte_node/ip4_lookup.c b/lib/librte_node/ip4_lookup.c index 293c77f39e..8835aab9dd 100644 --- a/lib/librte_node/ip4_lookup.c +++ b/lib/librte_node/ip4_lookup.c @@ -15,6 +15,7 @@ #include #include #include +#include #include "rte_node_ip4_api.h" @@ -34,10 +35,10 @@ static struct ip4_lookup_node_main ip4_lookup_nm; #include "ip4_lookup_neon.h" #elif defined(RTE_ARCH_X86) #include "ip4_lookup_sse.h" -#else +#endif static uint16_t -ip4_lookup_node_process(struct rte_graph *graph, struct rte_node *node, +ip4_lookup_node_process_scalar(struct rte_graph *graph, struct rte_node *node, void **objs, uint16_t nb_objs) { struct rte_ipv4_hdr *ipv4_hdr; @@ -109,8 +110,6 @@ ip4_lookup_node_process(struct rte_graph *graph, struct rte_node *node, return nb_objs; } -#endif - int rte_node_ip4_route_add(uint32_t ip, uint8_t depth, uint16_t next_hop, enum rte_node_ip4_lookup_next next_node) @@ -194,13 +193,19 @@ ip4_lookup_node_init(const struct rte_graph *graph, struct rte_node *node) init_once = 1; } *lpm_p = ip4_lookup_nm.lpm_tbl[graph->socket]; + +#if defined(__ARM_NEON) || defined(RTE_ARCH_X86) + if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) + node->process = ip4_lookup_node_process_vec; +#endif + node_dbg("ip4_lookup", "Initialized ip4_lookup node"); return 0; } static struct rte_node_register ip4_lookup_node = { - .process = ip4_lookup_node_process, + .process = ip4_lookup_node_process_scalar, .name = "ip4_lookup", .init = ip4_lookup_node_init, diff --git a/lib/librte_node/ip4_lookup_neon.h b/lib/librte_node/ip4_lookup_neon.h index 5e5a7d87be..0ad2763b82 100644 --- a/lib/librte_node/ip4_lookup_neon.h +++ b/lib/librte_node/ip4_lookup_neon.h @@ -7,7 +7,7 @@ /* ARM64 NEON */ static uint16_t -ip4_lookup_node_process(struct rte_graph *graph, struct rte_node *node, +ip4_lookup_node_process_vec(struct rte_graph *graph, struct rte_node *node, void **objs, uint16_t nb_objs) { struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3, **pkts; diff --git a/lib/librte_node/ip4_lookup_sse.h b/lib/librte_node/ip4_lookup_sse.h index a071cc5919..264c986071 100644 --- a/lib/librte_node/ip4_lookup_sse.h +++ b/lib/librte_node/ip4_lookup_sse.h @@ -7,7 +7,7 @@ /* X86 SSE */ static uint16_t -ip4_lookup_node_process(struct rte_graph *graph, struct rte_node *node, +ip4_lookup_node_process_vec(struct rte_graph *graph, struct rte_node *node, void **objs, uint16_t nb_objs) { struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3, **pkts; From patchwork Mon Oct 19 13:48:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 81379 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B9A0BA04DC; Mon, 19 Oct 2020 15:56:03 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8BC5AE2C1; Mon, 19 Oct 2020 15:51:09 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id EDFD9E2C8 for ; Mon, 19 Oct 2020 15:51:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603115464; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ieCmNvJ9rrrhKkGhP0oUzSZK0lbI8+6MBDu2PascNxc=; b=gDn8uEoXRSZgRivsafQpRLp/pGJaNqEUga9g4rRdqaSRjYIzID55jqV2EBBh/9pQKcfDuN 69PzauiQepGUQJgBqyCNxdG0SImc5CJXI4MNmUu3CjIsND1MrJrqk5YuEzgPs5Ij6mmX0q yjORgYfWCCGl8MV7hO8WxN0wwlKfuR0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-562-5WgfbR7_POW4rezzkOJ0mg-1; Mon, 19 Oct 2020 09:51:00 -0400 X-MC-Unique: 5WgfbR7_POW4rezzkOJ0mg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 24B8C640AE; Mon, 19 Oct 2020 13:50:04 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1AF5155798; Mon, 19 Oct 2020 13:50:01 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: viktorin@rehivetech.com, ruifeng.wang@arm.com, jerinj@marvell.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, ciara.power@intel.com Date: Mon, 19 Oct 2020 15:48:58 +0200 Message-Id: <20201019134858.32507-19-david.marchand@redhat.com> In-Reply-To: <20201019134858.32507-1-david.marchand@redhat.com> References: <20201019134858.32507-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v10 18/18] acl: check max SIMD bitwidth X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ciara Power When choosing a vector path to take, an extra condition must be satisfied to ensure the max SIMD bitwidth allows for the CPU enabled path. These checks are added in the check alg helper functions. Signed-off-by: Ciara Power Acked-by: Konstantin Ananyev Tested-by: Konstantin Ananyev --- v10: - updated #ifdef magic for arm and ppc checks, v8: - Added AVX512X32 into default options for get best alg function. - Updated doc to reflect the above change. v7: - Removed global variable for max SIMD bitwidth. - Added helper function for checking AVX512 cpu flags. - Separated condition checking for the AVX512 algorithms to allow for checking 256/512 max SIMD bitwidth, respectively. - Added to docs to reflect the added changes in algorithm selection. --- .../prog_guide/packet_classif_access_ctrl.rst | 16 +++--- lib/librte_acl/rte_acl.c | 53 +++++++++++++------ lib/librte_acl/rte_acl.h | 1 + 3 files changed, 47 insertions(+), 23 deletions(-) diff --git a/doc/guides/prog_guide/packet_classif_access_ctrl.rst b/doc/guides/prog_guide/packet_classif_access_ctrl.rst index 7659af8eb5..1811db4618 100644 --- a/doc/guides/prog_guide/packet_classif_access_ctrl.rst +++ b/doc/guides/prog_guide/packet_classif_access_ctrl.rst @@ -368,24 +368,27 @@ After rte_acl_build() over given AC context has finished successfully, it can be There are several implementations of classify algorithm: * **RTE_ACL_CLASSIFY_SCALAR**: generic implementation, doesn't require any specific HW support. + Requires max SIMD bitwidth to be at least 64. * **RTE_ACL_CLASSIFY_SSE**: vector implementation, can process up to 8 flows in parallel. Requires SSE 4.1 support. + Requires max SIMD bitwidth to be at least 128. * **RTE_ACL_CLASSIFY_AVX2**: vector implementation, can process up to 16 flows in parallel. Requires AVX2 support. + Requires max SIMD bitwidth to be at least 256. * **RTE_ACL_CLASSIFY_NEON**: vector implementation, can process up to 8 flows - in parallel. Requires NEON support. + in parallel. Requires NEON support. Requires max SIMD bitwidth to be at least 128. * **RTE_ACL_CLASSIFY_ALTIVEC**: vector implementation, can process up to 8 - flows in parallel. Requires ALTIVEC support. + flows in parallel. Requires ALTIVEC support. Requires max SIMD bitwidth to be at least 128. * **RTE_ACL_CLASSIFY_AVX512X16**: vector implementation, can process up to 16 flows in parallel. Uses 256-bit width SIMD registers. - Requires AVX512 support. + Requires AVX512 support. Requires max SIMD bitwidth to be at least 256. * **RTE_ACL_CLASSIFY_AVX512X32**: vector implementation, can process up to 32 flows in parallel. Uses 512-bit width SIMD registers. - Requires AVX512 support. + Requires AVX512 support. Requires max SIMD bitwidth to be at least 512. It is purely a runtime decision which method to choose, there is no build-time difference. All implementations operates over the same internal RT structures and use similar principles. The main difference is that vector implementations can manually exploit IA SIMD instructions and process several input data flows in parallel. @@ -393,9 +396,8 @@ At startup ACL library determines the highest available classify method for the .. note:: - Right now ``RTE_ACL_CLASSIFY_AVX512X32`` is not selected by default - (due to possible frequency level change), but it can be selected at - runtime by apps through the use of ACL API: ``rte_acl_set_ctx_classify``. + Runtime algorithm selection obeys EAL max SIMD bitwidth parameter. + For more details about expected behaviour please see :ref:`max_simd_bitwidth` Application Programming Interface (API) Usage --------------------------------------------- diff --git a/lib/librte_acl/rte_acl.c b/lib/librte_acl/rte_acl.c index 7c2f60b2d6..4e693b2488 100644 --- a/lib/librte_acl/rte_acl.c +++ b/lib/librte_acl/rte_acl.c @@ -6,6 +6,7 @@ #include #include #include +#include #include "acl.h" @@ -114,14 +115,14 @@ acl_check_alg_arm(enum rte_acl_classify_alg alg) { if (alg == RTE_ACL_CLASSIFY_NEON) { #if defined(RTE_ARCH_ARM64) - return 0; + if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) + return 0; #elif defined(RTE_ARCH_ARM) - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON)) + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) return 0; - return -ENOTSUP; -#else - return -ENOTSUP; #endif + return -ENOTSUP; } return -EINVAL; @@ -136,15 +137,26 @@ acl_check_alg_ppc(enum rte_acl_classify_alg alg) { if (alg == RTE_ACL_CLASSIFY_ALTIVEC) { #if defined(RTE_ARCH_PPC_64) - return 0; -#else - return -ENOTSUP; + if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) + return 0; #endif + return -ENOTSUP; } return -EINVAL; } +#ifdef CC_AVX512_SUPPORT +static int +acl_check_avx512_cpu_flags(void) +{ + return (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) && + rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512VL) && + rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512CD) && + rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW)); +} +#endif + /* * Helper function for acl_check_alg. * Check support for x86 specific classify methods. @@ -152,13 +164,19 @@ acl_check_alg_ppc(enum rte_acl_classify_alg alg) static int acl_check_alg_x86(enum rte_acl_classify_alg alg) { - if (alg == RTE_ACL_CLASSIFY_AVX512X16 || - alg == RTE_ACL_CLASSIFY_AVX512X32) { + if (alg == RTE_ACL_CLASSIFY_AVX512X32) { #ifdef CC_AVX512_SUPPORT - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) && - rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512VL) && - rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512CD) && - rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW)) + if (acl_check_avx512_cpu_flags() != 0 && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512) + return 0; +#endif + return -ENOTSUP; + } + + if (alg == RTE_ACL_CLASSIFY_AVX512X16) { +#ifdef CC_AVX512_SUPPORT + if (acl_check_avx512_cpu_flags() != 0 && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) return 0; #endif return -ENOTSUP; @@ -166,7 +184,8 @@ acl_check_alg_x86(enum rte_acl_classify_alg alg) if (alg == RTE_ACL_CLASSIFY_AVX2) { #ifdef CC_AVX2_SUPPORT - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2)) + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256) return 0; #endif return -ENOTSUP; @@ -174,7 +193,8 @@ acl_check_alg_x86(enum rte_acl_classify_alg alg) if (alg == RTE_ACL_CLASSIFY_SSE) { #ifdef RTE_ARCH_X86 - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_SSE4_1)) + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_SSE4_1) && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) return 0; #endif return -ENOTSUP; @@ -226,6 +246,7 @@ acl_get_best_alg(void) #elif defined(RTE_ARCH_PPC_64) RTE_ACL_CLASSIFY_ALTIVEC, #elif defined(RTE_ARCH_X86) + RTE_ACL_CLASSIFY_AVX512X32, RTE_ACL_CLASSIFY_AVX512X16, RTE_ACL_CLASSIFY_AVX2, RTE_ACL_CLASSIFY_SSE, diff --git a/lib/librte_acl/rte_acl.h b/lib/librte_acl/rte_acl.h index 1bfed00743..f7f5f08701 100644 --- a/lib/librte_acl/rte_acl.h +++ b/lib/librte_acl/rte_acl.h @@ -329,6 +329,7 @@ rte_acl_classify_alg(const struct rte_acl_ctx *ctx, * New default classify algorithm for given ACL context. * It is the caller responsibility to ensure that the value refers to the * existing algorithm, and that it could be run on the given CPU. + * The max SIMD bitwidth value in EAL must also allow for the chosen algorithm. * @return * - -EINVAL if the parameters are invalid. * - -ENOTSUP requested algorithm is not supported by given platform.