From patchwork Mon Jul 20 06:26:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74461 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 317D1A0540; Mon, 20 Jul 2020 08:26:40 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E1DB22C2B; Mon, 20 Jul 2020 08:26:30 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 220AC2BF3 for ; Mon, 20 Jul 2020 08:26:28 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:26:24 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKud008177; Mon, 20 Jul 2020 09:26:24 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Thomas Monjalon , John McNamara , Marko Kovacevic , Parav Pandit , Shahaf Shuler , Ray Kinsella , Neil Horman Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Mon, 20 Jul 2020 06:26:05 +0000 Message-Id: <1595226378-81144-2-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 01/13] regex/mlx5: add RegEx PMD layer and mlx5 driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery This commit introduce the RegEx pull mode drivers class, and adds Mellanox RegEx PMD. Signed-off-by: Yuval Avnery Signed-off-by: Ori Kam --- v6: * Address ML comments. v5: * Move to mlx5 bus v3: * Address ML comments. v2: * Add docs. --- .gitignore | 1 + MAINTAINERS | 11 +++ config/common_base | 5 ++ doc/guides/conf.py | 7 +- doc/guides/index.rst | 1 + doc/guides/regexdevs/features/default.ini | 32 +++++++ doc/guides/regexdevs/features/mlx5.ini | 9 ++ doc/guides/regexdevs/features_overview.rst | 100 ++++++++++++++++++++++ doc/guides/regexdevs/index.rst | 15 ++++ doc/guides/regexdevs/mlx5.rst | 53 ++++++++++++ doc/guides/rel_notes/release_20_08.rst | 2 + drivers/Makefile | 2 + drivers/bus/Makefile | 2 +- drivers/bus/mlx5_pci/mlx5_pci_bus.c | 6 +- drivers/common/Makefile | 2 +- drivers/common/mlx5/Makefile | 4 +- drivers/common/mlx5/mlx5_common.h | 1 + drivers/meson.build | 1 + drivers/regex/Makefile | 8 ++ drivers/regex/meson.build | 7 ++ drivers/regex/mlx5/Makefile | 34 ++++++++ drivers/regex/mlx5/meson.build | 26 ++++++ drivers/regex/mlx5/mlx5_regex.c | 5 ++ drivers/regex/mlx5/mlx5_regex.h | 8 ++ drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map | 3 + mk/rte.app.mk | 5 +- 26 files changed, 342 insertions(+), 8 deletions(-) create mode 100644 doc/guides/regexdevs/features/default.ini create mode 100644 doc/guides/regexdevs/features/mlx5.ini create mode 100644 doc/guides/regexdevs/features_overview.rst create mode 100644 doc/guides/regexdevs/index.rst create mode 100644 doc/guides/regexdevs/mlx5.rst create mode 100644 drivers/regex/Makefile create mode 100644 drivers/regex/meson.build create mode 100644 drivers/regex/mlx5/Makefile create mode 100644 drivers/regex/mlx5/meson.build create mode 100644 drivers/regex/mlx5/mlx5_regex.c create mode 100644 drivers/regex/mlx5/mlx5_regex.h create mode 100644 drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map diff --git a/.gitignore b/.gitignore index f2f8892..f73d93c 100644 --- a/.gitignore +++ b/.gitignore @@ -9,6 +9,7 @@ doc/guides/cryptodevs/overview_auth_table.txt doc/guides/cryptodevs/overview_aead_table.txt doc/guides/cryptodevs/overview_asym_table.txt doc/guides/compressdevs/overview_feature_table.txt +doc/guides/regexdevs/overview_feature_table.txt doc/guides/vdpadevs/overview_feature_table.txt doc/guides/bbdevs/overview_feature_table.txt diff --git a/MAINTAINERS b/MAINTAINERS index 9e3bd16..dc4bf21 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -454,6 +454,7 @@ RegEx API - EXPERIMENTAL M: Ori Kam F: lib/librte_regexdev/ F: doc/guides/prog_guide/regexdev.rst +F: doc/guides/regexdevs/features/default.ini Eventdev API M: Jerin Jacob @@ -1133,6 +1134,16 @@ F: doc/guides/compressdevs/zlib.rst F: doc/guides/compressdevs/features/zlib.ini +RegEx Drivers +------------- + +Mellanox MLX5 +M: Ori Kam +F: drivers/regex/mlx5/ +F: doc/guides/regexdevs/mlx5.rst +F: doc/guides/regexdevs/features/mlx5.ini + + vDPA Drivers ------------ T: git://dpdk.org/next/dpdk-next-virtio diff --git a/config/common_base b/config/common_base index f7a8824..f76585f 100644 --- a/config/common_base +++ b/config/common_base @@ -375,6 +375,11 @@ CONFIG_RTE_LIBRTE_MLX5_PMD=n CONFIG_RTE_LIBRTE_MLX5_DEBUG=n # +# Compile regex-oriented Mellanox PMD +# +CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD=n + +# # Compile vdpa-oriented Mellanox ConnectX-6 & BlueField (MLX5) PMD # CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD=n diff --git a/doc/guides/conf.py b/doc/guides/conf.py index 700e05e..d8fe5cc 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -47,7 +47,7 @@ master_doc = 'index' # Maximum feature description string length -feature_str_len = 25 +feature_str_len = 30 # Figures, tables and code-blocks automatically numbered if they have caption numfig = True @@ -409,6 +409,11 @@ def setup(app): 'Features', 'Features availability in compression drivers', 'Feature') + table_file = dirname(__file__) + '/regexdevs/overview_feature_table.txt' + generate_overview_table(table_file, 1, + 'Features', + 'Features availability in regex drivers', + 'Feature') table_file = dirname(__file__) + '/vdpadevs/overview_feature_table.txt' generate_overview_table(table_file, 1, 'Features', diff --git a/doc/guides/index.rst b/doc/guides/index.rst index 988c6ea..857f036 100644 --- a/doc/guides/index.rst +++ b/doc/guides/index.rst @@ -20,6 +20,7 @@ DPDK documentation cryptodevs/index compressdevs/index vdpadevs/index + regexdevs/index eventdevs/index rawdevs/index mempool/index diff --git a/doc/guides/regexdevs/features/default.ini b/doc/guides/regexdevs/features/default.ini new file mode 100644 index 0000000..10f6c50 --- /dev/null +++ b/doc/guides/regexdevs/features/default.ini @@ -0,0 +1,32 @@ +; +; Features of a default RegEx driver. +; +; This file defines the features that are valid for inclusion in +; the other driver files and also the order that they appear in +; the features table in the documentation. The feature description +; string should not exceed feature_str_len defined in conf.py. +; +[Features] +Cross buffer = +PCRE start anchor = +PCRE atomic grouping = +PCRE back reference = +PCRE back tracking ctrl = +PCRE call outs = +PCRE forward reference = +PCRE greedy = +PCRE match all = +PCRE match as end = +PCRE match point rst = +PCRE New line conventions = +PCRE new line SEQ = +PCRE look around = +PCRE possessive qualifiers = +PCRE subroutine references = +PCRE UTF 8 = +PCRE UTF 16 = +PCRE UTF 32 = +PCRE word boundary = +Run time compilation = +Armv8 = +x86 = diff --git a/doc/guides/regexdevs/features/mlx5.ini b/doc/guides/regexdevs/features/mlx5.ini new file mode 100644 index 0000000..9fe5f73 --- /dev/null +++ b/doc/guides/regexdevs/features/mlx5.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'mlx5' RegEx driver. +; +; Refer to default.ini for the full list of available driver features. +; +[Features] +Armv8 = Y +x86 = Y + diff --git a/doc/guides/regexdevs/features_overview.rst b/doc/guides/regexdevs/features_overview.rst new file mode 100644 index 0000000..f90b394 --- /dev/null +++ b/doc/guides/regexdevs/features_overview.rst @@ -0,0 +1,100 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2020 Mellanox Technologies, Ltd + +Overview of RegEx Drivers Features +================================== + +This section explains the supported features that are listed in the table below. + +Cross buffer + Support cross buffer detection. + +PCRE start anchor + Support PCRE start anchor. + +PCRE atomic grouping + Support PCRE atomic grouping. + +PCRE back reference + Support PCRE back regerence. + +PCRE back tracking ctrl + Support PCRE back tracking ctrl. + +PCRE call outs + Support PCRE call outes. + +PCRE forward reference + Support Forward reference. + +PCRE greedy + Support PCRE greedy mode. + +PCRE match all + Support PCRE match all. + +PCRE match as end + Support match as end. + +PCRE match point rst + Support PCRE match point reset directive. + +PCRE New line conventions + Support new line conventions. + +PCRE new line SEQ + Support new line sequence. + +PCRE look around + Support PCRE look around. + +PCRE possessive qualifiers + Support PCRE possessive qualifiers. + +PCRE subroutine references + Support PCRE subroutine references. + +PCRE UTF 8 + Support UTF-8. + +PCRE UTF 16 + Support UTF-16. + +PCRE UTF 32 + Support UTF-32. + +PCRE word boundary + Support word boundaries. + +Run time compilation + Support compilation during run time. + +Armv8 + Support armv8 (64bit) architecture. + +x86 + Support x86 architecture. + +.. note:: + + Most of the features capabilities should be provided by the drivers via the + RegEx ``info_get`` operation. + + +References +========== + + * `PCRE: PCRE pattern man page `_ + + +Features Table +============== + +.. _table_regex_pmd_features: + +.. include:: overview_feature_table.txt + +.. Note:: + + Features marked with "P" are partially supported. Refer to the appropriate + driver guide in the following sections for details. diff --git a/doc/guides/regexdevs/index.rst b/doc/guides/regexdevs/index.rst new file mode 100644 index 0000000..49216a9 --- /dev/null +++ b/doc/guides/regexdevs/index.rst @@ -0,0 +1,15 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2020 Mellanox Technologies, Ltd + +REGEX Device Drivers +==================== + +The following are a list of RegEx (Regular Expression) device drivers, +which can be used from an application through RegEx API. + +.. toctree:: + :maxdepth: 2 + :numbered: + + features_overview + mlx5 diff --git a/doc/guides/regexdevs/mlx5.rst b/doc/guides/regexdevs/mlx5.rst new file mode 100644 index 0000000..0e6ee8c --- /dev/null +++ b/doc/guides/regexdevs/mlx5.rst @@ -0,0 +1,53 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2020 Mellanox Technologies, Ltd + +.. include:: + +MLX5 RegEx driver +================= + +The MLX5 RegEx (Regular Expression) driver library +(**librte_pmd_mlx5_regex**) provides support for **Mellanox BlueField 2** +families of 25/50/100/200 Gb/s adapters. + +Design +------ + +This PMD is configuring the RegEx HW engine. +For the PMD to work, the application must supply +a precompiled rule file in rof2 format. + +The PMD uses libibverbs and libmlx5 to access the device firmware +or directly the hardware components. +There are different levels of objects and bypassing abilities +to get the best performances: + +- Verbs is a complete high-level generic API +- Direct Verbs is a device-specific API +- DevX allows to access firmware objects + +Enabling librte_pmd_mlx5_regex causes DPDK applications to be linked against +libibverbs. + +Mellanox mlx5 pci device can be probed by number of different pci devices, +for example net / vDPA / RegEx. To select the RegEx PMD ``class=regex`` should +be specified as device parameter. The RegEx device can be probed and used with +other Mellanox devices, by adding more options in the class. +For example: ``class=net:regex`` will probe both the net PMD and the RegEx PMD. + +Supported NICs +-------------- + +* Mellanox\ |reg| BlueField 2 SmartNIC + +Prerequisites +------------- + +- BlueField 2 running Mellanox supported kernel. +- Enable the RegEx capabilities using system call from the BlueField 2. +- Official support is not yet released. + +Run-time configuration +~~~~~~~~~~~~~~~~~~~~~~ + +- **ethtool** operations on related kernel interfaces also affect the PMD. diff --git a/doc/guides/rel_notes/release_20_08.rst b/doc/guides/rel_notes/release_20_08.rst index 917a5b8..3f18273 100644 --- a/doc/guides/rel_notes/release_20_08.rst +++ b/doc/guides/rel_notes/release_20_08.rst @@ -81,6 +81,8 @@ New Features Added the RegEx library which provides an API for offload of regular expressions search operations to hardware or software accelerator devices. + Added Mellanox RegEx PMD, allowing to offload RegEx searches. + * **Added eCPRI protocol support in rte_flow.** The ``ECPRI`` item has been added to support eCPRI packet offloading for diff --git a/drivers/Makefile b/drivers/Makefile index c70bdf9..b814e05 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -18,6 +18,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += common/qat DEPDIRS-common/qat := bus mempool DIRS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += compress DEPDIRS-compress := bus mempool +DIRS-$(CONFIG_RTE_LIBRTE_REGEXDEV) += regex +DEPDIRS-regex := common DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vdpa DEPDIRS-vdpa := common bus mempool DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += event diff --git a/drivers/bus/Makefile b/drivers/bus/Makefile index a70f213..b07786a 100644 --- a/drivers/bus/Makefile +++ b/drivers/bus/Makefile @@ -9,7 +9,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += fslmc endif DIRS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) += ifpga DIRS-$(CONFIG_RTE_LIBRTE_PCI_BUS) += pci -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) DEPDIRS-mlx5_pci := pci DIRS-y += mlx5_pci endif diff --git a/drivers/bus/mlx5_pci/mlx5_pci_bus.c b/drivers/bus/mlx5_pci/mlx5_pci_bus.c index 6f21973..22c24f8 100644 --- a/drivers/bus/mlx5_pci/mlx5_pci_bus.c +++ b/drivers/bus/mlx5_pci/mlx5_pci_bus.c @@ -25,13 +25,17 @@ static TAILQ_HEAD(mlx5_pci_devices_head, mlx5_pci_device) devices_list = const char *name; unsigned int dev_class; } mlx5_classes[] = { - { .name = "vdpa", .dev_class = MLX5_CLASS_VDPA }, { .name = "net", .dev_class = MLX5_CLASS_NET }, + { .name = "vdpa", .dev_class = MLX5_CLASS_VDPA }, + { .name = "regex", .dev_class = MLX5_CLASS_REGEX }, }; static const unsigned int mlx5_class_combinations[] = { MLX5_CLASS_NET, MLX5_CLASS_VDPA, + MLX5_CLASS_REGEX, + MLX5_CLASS_REGEX | MLX5_CLASS_NET, + MLX5_CLASS_REGEX | MLX5_CLASS_VDPA, /* New class combination should be added here. * For example a new multi class device combination * can be MLX5_CLASS_FOO | MLX5_CLASS_BAR. diff --git a/drivers/common/Makefile b/drivers/common/Makefile index df2e840..cbc7107 100644 --- a/drivers/common/Makefile +++ b/drivers/common/Makefile @@ -36,7 +36,7 @@ ifneq (,$(findstring y,$(IAVF-y))) DIRS-y += iavf endif -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) DIRS-y += mlx5 endif diff --git a/drivers/common/mlx5/Makefile b/drivers/common/mlx5/Makefile index 96a2dae..6b89a6c 100644 --- a/drivers/common/mlx5/Makefile +++ b/drivers/common/mlx5/Makefile @@ -10,7 +10,7 @@ LIB_GLUE_BASE = librte_pmd_mlx5_glue.so LIB_GLUE_VERSION = 20.02.0 # Sources. -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) ifneq ($(CONFIG_RTE_IBVERBS_LINK_DLOPEN),y) SRCS-y += linux/mlx5_glue.c endif @@ -370,7 +370,7 @@ mlx5_autoconf.h: mlx5_autoconf.h.new cmp '$<' '$@' $(AUTOCONF_OUTPUT) || \ mv '$<' '$@' -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) $(SRCS-y:.c=.o): mlx5_autoconf.h endif diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 5b9b7bd..a1f34c3 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -212,6 +212,7 @@ enum mlx5_class { MLX5_CLASS_INVALID, MLX5_CLASS_NET = RTE_BIT64(0), MLX5_CLASS_VDPA = RTE_BIT64(1), + MLX5_CLASS_REGEX = RTE_BIT64(2), }; #define MLX5_DBR_PAGE_SIZE 4096 /* Must be >= 512. */ diff --git a/drivers/meson.build b/drivers/meson.build index 161cfda..e76ebdd 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -9,6 +9,7 @@ dpdk_driver_classes = ['common', 'raw', # depends on common, bus and net. 'crypto', # depends on common, bus and mempool (net in future). 'compress', # depends on common, bus, mempool. + 'regex', # depends on common, bus, regexdev. 'vdpa', # depends on common, bus and mempool. 'event', # depends on common, bus, mempool and net. 'baseband'] # depends on common and bus. diff --git a/drivers/regex/Makefile b/drivers/regex/Makefile new file mode 100644 index 0000000..906b205 --- /dev/null +++ b/drivers/regex/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +include $(RTE_SDK)/mk/rte.vars.mk + +DIRS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5 + +include $(RTE_SDK)/mk/rte.subdir.mk diff --git a/drivers/regex/meson.build b/drivers/regex/meson.build new file mode 100644 index 0000000..8edeba3 --- /dev/null +++ b/drivers/regex/meson.build @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +drivers = ['mlx5'] +std_deps = ['ethdev', 'kvargs'] # 'ethdev' also pulls in mbuf, net, eal etc +config_flag_fmt = 'RTE_LIBRTE_@0@_PMD' +driver_name_fmt = 'rte_pmd_@0@' diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile new file mode 100644 index 0000000..2aba90e --- /dev/null +++ b/drivers/regex/mlx5/Makefile @@ -0,0 +1,34 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +include $(RTE_SDK)/mk/rte.vars.mk + +# Library name. +LIB = librte_pmd_mlx5_regex.a + +# Sources. +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c + +# Basic CFLAGS. +CFLAGS += -O3 +CFLAGS += -std=c11 -Wall -Wextra +CFLAGS += -g +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 +CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -Wno-strict-prototypes +LDLIBS += -lrte_common_mlx5 +LDLIBS += -lm +LDLIBS += -lrte_eal -lrte_mbuf -lrte_regexdev +LDLIBS += -lrte_kvargs +LDLIBS += -lrte_bus_pci -lrte_buf_mlx5_pci + +# A few warnings cannot be avoided in external headers. +CFLAGS += -Wno-error=cast-qual + +EXPORT_MAP := rte_pmd_mlx5_regex_version.map + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build new file mode 100644 index 0000000..27bb0ad --- /dev/null +++ b/drivers/regex/mlx5/meson.build @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +if not is_linux + build = false + reason = 'only supported on Linux' + subdir_done() +endif + +fmt_name = 'mlx5_regex' +deps += ['common_mlx5', 'bus_pci', 'bus_mlx5_pci', 'eal', 'regexdev'] +sources = files( + 'mlx5_regex.c', +) +cflags_options = [ + '-std=c11', + '-Wno-strict-prototypes', + '-D_BSD_SOURCE', + '-D_DEFAULT_SOURCE', + '-D_XOPEN_SOURCE=600' +] +foreach option:cflags_options + if cc.has_argument(option) + cflags += option + endif +endforeach diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c new file mode 100644 index 0000000..b942a75 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -0,0 +1,5 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include "mlx5_regex.h" diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h new file mode 100644 index 0000000..0e0495c --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#ifndef MLX5_REGEX_H +#define MLX5_REGEX_H + +#endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map b/drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map new file mode 100644 index 0000000..4a76d1d --- /dev/null +++ b/drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map @@ -0,0 +1,3 @@ +DPDK_21 { + local: *; +}; diff --git a/mk/rte.app.mk b/mk/rte.app.mk index d7c1d90..a7f30c4 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -201,12 +201,13 @@ endif _LDLIBS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += -lrte_pmd_lio _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_MEMIF) += -lrte_pmd_memif _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += -lrte_pmd_mlx4 -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) _LDLIBS-y += -lrte_common_mlx5 _LDLIBS-y += -lrte_bus_mlx5_pci endif _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += -lrte_pmd_mlx5 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD) += -lrte_pmd_mlx5_vdpa +_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += -lrte_pmd_mlx5_regex ifeq ($(CONFIG_RTE_IBVERBS_LINK_DLOPEN),y) _LDLIBS-y += -ldl else ifeq ($(CONFIG_RTE_IBVERBS_LINK_STATIC),y) @@ -215,7 +216,7 @@ _LDLIBS-y += --no-whole-archive _LDLIBS-y += $(LIBS_IBVERBS_STATIC) _LDLIBS-y += --whole-archive else -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) _LDLIBS-y += -libverbs -lmlx5 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += -libverbs -lmlx4 From patchwork Mon Jul 20 06:26:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74462 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D2F32A0540; Mon, 20 Jul 2020 08:26:51 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4769C2B86; Mon, 20 Jul 2020 08:26:35 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 239472C57 for ; Mon, 20 Jul 2020 08:26:33 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:26:28 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKue008177; Mon, 20 Jul 2020 09:26:28 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Mon, 20 Jul 2020 06:26:06 +0000 Message-Id: <1595226378-81144-3-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 02/13] regex/mlx5: add log utils X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Add the MLX5_REGEX_LOG macro which should be used for error prints. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/mlx5_regex.c | 5 +++++ drivers/regex/mlx5/mlx5_regex_utils.h | 19 +++++++++++++++++++ 3 files changed, 25 insertions(+) create mode 100644 drivers/regex/mlx5/mlx5_regex_utils.h diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 2aba90e..4a2f4e0 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -15,6 +15,7 @@ CFLAGS += -std=c11 -Wall -Wextra CFLAGS += -g CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux CFLAGS += -D_BSD_SOURCE CFLAGS += -D_DEFAULT_SOURCE CFLAGS += -D_XOPEN_SOURCE=600 diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index b942a75..922c236 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -3,3 +3,8 @@ */ #include "mlx5_regex.h" +#include "mlx5_regex_utils.h" + +int mlx5_regex_logtype; + +RTE_LOG_REGISTER(mlx5_regex_logtype, pmd.regex.mlx5, NOTICE) diff --git a/drivers/regex/mlx5/mlx5_regex_utils.h b/drivers/regex/mlx5/mlx5_regex_utils.h new file mode 100644 index 0000000..adca846 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex_utils.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#ifndef RTE_PMD_MLX5_REGEX_UTILS_H_ +#define RTE_PMD_MLX5_REGEX_UTILS_H_ + +#include + +extern int mlx5_regex_logtype; + +#define MLX5_REGEX_LOG_PREFIX "regex_mlx5" +/* Generic printf()-like logging macro with automatic line feed. */ +#define DRV_LOG(level, ...) \ + PMD_DRV_LOG_(level, mlx5_regex_logtype, MLX5_REGEX_LOG_PREFIX, \ + __VA_ARGS__ PMD_DRV_LOG_STRIP PMD_DRV_LOG_OPAREN, \ + PMD_DRV_LOG_CPAREN) + +#endif /* RTE_PMD_MLX5_REGEX_UTILS_H_ */ From patchwork Mon Jul 20 06:26:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74463 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 08004A0540; Mon, 20 Jul 2020 08:27:03 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BBB0E2D13; Mon, 20 Jul 2020 08:26:36 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 3A0D72C5E for ; Mon, 20 Jul 2020 08:26:33 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:26:32 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKuf008177; Mon, 20 Jul 2020 09:26:31 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler , Thomas Monjalon Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, orika@mellanox.com, rasland@mellanox.com, Parav Pandit Date: Mon, 20 Jul 2020 06:26:07 +0000 Message-Id: <1595226378-81144-4-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 03/13] regex/mlx5: add probe function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the probe function to the RegEx PMD. Signed-off-by: Parav Pandit Signed-off-by: Ori Kam --- drivers/common/mlx5/mlx5_devx_cmds.c | 3 + drivers/common/mlx5/mlx5_devx_cmds.h | 2 + drivers/common/mlx5/mlx5_prm.h | 9 +- drivers/regex/mlx5/Makefile | 2 + drivers/regex/mlx5/mlx5_regex.c | 172 +++++++++++++++++++++++++++++++++++ drivers/regex/mlx5/mlx5_regex.h | 6 ++ mk/rte.app.mk | 1 + 7 files changed, 193 insertions(+), 2 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 757c55e..66b8d24 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -706,6 +706,9 @@ struct mlx5_devx_obj * device_frequency_khz); attr->scatter_fcs_w_decap_disable = MLX5_GET(cmd_hca_cap, hcattr, scatter_fcs_w_decap_disable); + attr->regex = MLX5_GET(cmd_hca_cap, hcattr, regexp); + attr->regexp_num_of_engines = MLX5_GET(cmd_hca_cap, hcattr, + regexp_num_of_engines); if (attr->qos.sup) { MLX5_SET(query_hca_cap_in, in, op_mod, MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP | diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index a7e3c39..ab61454 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -99,6 +99,8 @@ struct mlx5_hca_attr { uint32_t log_max_static_sq_wq:5; /* Static WQE size SQ. */ uint32_t dev_freq_khz; /* Timestamp counter frequency, kHz. */ uint32_t scatter_fcs_w_decap_disable:1; + uint32_t regex:1; + uint32_t regexp_num_of_engines; struct mlx5_hca_qos_attr qos; struct mlx5_hca_vdpa_attr vdpa; }; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index d7d6cf8..364db81 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -1043,9 +1043,14 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 wqe_index_ignore_cap[0x1]; u8 dynamic_qp_allocation[0x1]; u8 log_max_qp[0x5]; - u8 reserved_at_a0[0xb]; + u8 regexp[0x1]; + u8 reserved_at_a1[0x3]; + u8 regexp_num_of_engines[0x4]; + u8 reserved_at_a8[0x3]; u8 log_max_srq[0x5]; - u8 reserved_at_b0[0xb]; + u8 reserved_at_b0[0x3]; + u8 regexp_log_crspace_size[0x5]; + u8 reserved_at_b8[0x3]; u8 scatter_fcs_w_decap_disable[0x1]; u8 reserved_at_bc[0x4]; u8 reserved_at_c0[0x8]; diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 4a2f4e0..1bddf21 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -16,6 +16,8 @@ CFLAGS += -g CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux +CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5/linux CFLAGS += -D_BSD_SOURCE CFLAGS += -D_DEFAULT_SOURCE CFLAGS += -D_XOPEN_SOURCE=600 diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 922c236..fe44429 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -2,9 +2,181 @@ * Copyright 2020 Mellanox Technologies, Ltd */ +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include +#include +#include + #include "mlx5_regex.h" #include "mlx5_regex_utils.h" int mlx5_regex_logtype; +static const struct rte_regexdev_ops mlx5_regexdev_ops = {0}; + +static struct ibv_device * +mlx5_regex_get_ib_device_match(struct rte_pci_addr *addr) +{ + int n; + struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n); + struct ibv_device *ibv_match = NULL; + + if (!ibv_list) { + rte_errno = ENOSYS; + return NULL; + } + while (n-- > 0) { + struct rte_pci_addr pci_addr; + + DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name); + if (mlx5_dev_to_pci_addr(ibv_list[n]->ibdev_path, &pci_addr)) + continue; + if (rte_pci_addr_cmp(addr, &pci_addr)) + continue; + ibv_match = ibv_list[n]; + break; + } + if (!ibv_match) + rte_errno = ENOENT; + mlx5_glue->free_device_list(ibv_list); + return ibv_match; +} + +static void +mlx5_regex_get_name(char *name, struct rte_pci_device *pci_dev __rte_unused) +{ + sprintf(name, "mlx5_regex_%02x:%02x.%02x", pci_dev->addr.bus, + pci_dev->addr.devid, pci_dev->addr.function); +} + +static int +mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) +{ + struct ibv_device *ibv; + struct mlx5_regex_priv *priv = NULL; + struct ibv_context *ctx = NULL; + struct mlx5_hca_attr attr; + char name[RTE_REGEXDEV_NAME_MAX_LEN]; + int ret; + + ibv = mlx5_regex_get_ib_device_match(&pci_dev->addr); + if (!ibv) { + DRV_LOG(ERR, "No matching IB device for PCI slot " + PCI_PRI_FMT ".", pci_dev->addr.domain, + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + return -rte_errno; + } + DRV_LOG(INFO, "PCI information matches for device \"%s\".", + ibv->name); + ctx = mlx5_glue->dv_open_device(ibv); + if (!ctx) { + DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); + rte_errno = ENODEV; + return -rte_errno; + } + ret = mlx5_devx_cmd_query_hca_attr(ctx, &attr); + if (ret) { + DRV_LOG(ERR, "Unable to read HCA capabilities."); + rte_errno = ENOTSUP; + goto error; + } else if (!attr.regex || attr.regexp_num_of_engines == 0) { + DRV_LOG(ERR, "Not enough capabilities to support RegEx, maybe " + "old FW/OFED version?"); + rte_errno = ENOTSUP; + goto error; + } + priv = rte_zmalloc("mlx5 regex device private", sizeof(*priv), + RTE_CACHE_LINE_SIZE); + if (!priv) { + DRV_LOG(ERR, "Failed to allocate private memory."); + rte_errno = ENOMEM; + goto error; + } + priv->ctx = ctx; + mlx5_regex_get_name(name, pci_dev); + priv->regexdev = rte_regexdev_register(name); + if (priv->regexdev == NULL) { + DRV_LOG(ERR, "Failed to register RegEx device."); + rte_errno = rte_errno ? rte_errno : EINVAL; + goto error; + } + priv->regexdev->dev_ops = &mlx5_regexdev_ops; + priv->regexdev->device = (struct rte_device *)pci_dev; + priv->regexdev->data->dev_private = priv; + return 0; + +error: + if (ctx) + mlx5_glue->close_device(ctx); + if (priv) + rte_free(priv); + return -rte_errno; +} + +static int +mlx5_regex_pci_remove(struct rte_pci_device *pci_dev) +{ + char name[RTE_REGEXDEV_NAME_MAX_LEN]; + struct rte_regexdev *dev; + struct mlx5_regex_priv *priv = NULL; + + mlx5_regex_get_name(name, pci_dev); + dev = rte_regexdev_get_device_by_name(name); + if (!dev) + return 0; + priv = dev->data->dev_private; + if (priv) { + if (priv->ctx) + mlx5_glue->close_device(priv->ctx); + if (priv->regexdev) + rte_regexdev_unregister(priv->regexdev); + rte_free(priv); + } + return 0; +} + +static const struct rte_pci_id mlx5_regex_pci_id_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6DXBF) + }, + { + .vendor_id = 0 + } +}; + +static struct rte_mlx5_pci_driver mlx5_regex_driver = { + .dev_class = MLX5_CLASS_REGEX, + .pci_driver = { + .driver = { + .name = "mlx5_regex", + }, + .id_table = mlx5_regex_pci_id_map, + .probe = mlx5_regex_pci_probe, + .remove = mlx5_regex_pci_remove, + .drv_flags = 0, + }, +}; + +RTE_INIT_PRIO(rte_mlx5_regex_init, CLASS) +{ + if (mlx5_glue) + rte_mlx5_pci_driver_register(&mlx5_regex_driver); +} + RTE_LOG_REGISTER(mlx5_regex_logtype, pmd.regex.mlx5, NOTICE) +RTE_PMD_EXPORT_NAME(net_mlx5_regex, __COUNTER__); +RTE_PMD_REGISTER_PCI_TABLE(net_mlx5_regex, mlx5_regex_pci_id_map); +RTE_PMD_REGISTER_KMOD_DEP(net_mlx5_regex, "* ib_uverbs & mlx5_core & mlx5_ib"); diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 0e0495c..0ce1e4d 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -5,4 +5,10 @@ #ifndef MLX5_REGEX_H #define MLX5_REGEX_H +struct mlx5_regex_priv { + TAILQ_ENTRY(mlx5_regex_priv) next; + struct ibv_context *ctx; /* Device context. */ + struct rte_pci_device *pci_dev; + struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ +}; #endif /* MLX5_REGEX_H */ diff --git a/mk/rte.app.mk b/mk/rte.app.mk index a7f30c4..5241fbe 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -101,6 +101,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched _LDLIBS-$(CONFIG_RTE_LIBRTE_RCU) += -lrte_rcu _LDLIBS-$(CONFIG_RTE_LIBRTE_GRAPH) += -lrte_graph _LDLIBS-$(CONFIG_RTE_LIBRTE_NODE) += -lrte_node +_LDLIBS-$(CONFIG_RTE_LIBRTE_REGEXDEV) += -lrte_regexdev ifeq ($(CONFIG_RTE_EXEC_ENV_LINUX),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_KNI) += -lrte_kni From patchwork Mon Jul 20 06:26:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74464 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4FB4EA0540; Mon, 20 Jul 2020 08:27:17 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C91EC2C6E; Mon, 20 Jul 2020 08:26:45 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 458311BED3 for ; Mon, 20 Jul 2020 08:26:38 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:26:35 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKug008177; Mon, 20 Jul 2020 09:26:35 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Mon, 20 Jul 2020 06:26:08 +0000 Message-Id: <1595226378-81144-5-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 04/13] regex/mlx5: add get info function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the get info function. Signed-off-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/meson.build | 1 + drivers/regex/mlx5/mlx5_regex.c | 4 +++- drivers/regex/mlx5/mlx5_regex.h | 5 +++++ drivers/regex/mlx5/mlx5_rxp.c | 29 +++++++++++++++++++++++++++++ 5 files changed, 39 insertions(+), 1 deletion(-) create mode 100644 drivers/regex/mlx5/mlx5_rxp.c diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 1bddf21..8fe61f8 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -8,6 +8,7 @@ LIB = librte_pmd_mlx5_regex.a # Sources. SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_rxp.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index 27bb0ad..bfb25e2 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -11,6 +11,7 @@ fmt_name = 'mlx5_regex' deps += ['common_mlx5', 'bus_pci', 'bus_mlx5_pci', 'eal', 'regexdev'] sources = files( 'mlx5_regex.c', + 'mlx5_rxp.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index fe44429..1f4d967 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -22,7 +22,9 @@ int mlx5_regex_logtype; -static const struct rte_regexdev_ops mlx5_regexdev_ops = {0}; +const struct rte_regexdev_ops mlx5_regexdev_ops = { + .dev_info_get = mlx5_regex_info_get, +}; static struct ibv_device * mlx5_regex_get_ib_device_match(struct rte_pci_addr *addr) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 0ce1e4d..9d0fc16 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -11,4 +11,9 @@ struct mlx5_regex_priv { struct rte_pci_device *pci_dev; struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ }; + +/* mlx5_rxp.c */ +int mlx5_regex_info_get(struct rte_regexdev *dev, + struct rte_regexdev_info *info); + #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c new file mode 100644 index 0000000..a5a6f15 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include +#include +#include +#include +#include + +#include "mlx5_regex.h" + +#define MLX5_REGEX_MAX_MATCHES 255 +#define MLX5_REGEX_MAX_PAYLOAD_SIZE UINT16_MAX +#define MLX5_REGEX_MAX_RULES_PER_GROUP UINT16_MAX +#define MLX5_REGEX_MAX_GROUPS UINT16_MAX + +int +mlx5_regex_info_get(struct rte_regexdev *dev __rte_unused, + struct rte_regexdev_info *info) +{ + info->max_matches = MLX5_REGEX_MAX_MATCHES; + info->max_payload_size = MLX5_REGEX_MAX_PAYLOAD_SIZE; + info->max_rules_per_group = MLX5_REGEX_MAX_RULES_PER_GROUP; + info->max_groups = MLX5_REGEX_MAX_GROUPS; + info->regexdev_capa = RTE_REGEXDEV_SUPP_PCRE_GREEDY_F; + info->rule_flags = 0; + return 0; +} From patchwork Mon Jul 20 06:26:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74465 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E7178A0540; Mon, 20 Jul 2020 08:27:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0E8571BED7; Mon, 20 Jul 2020 08:26:47 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 36D7BF04 for ; Mon, 20 Jul 2020 08:26:43 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:26:39 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKuh008177; Mon, 20 Jul 2020 09:26:39 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Mon, 20 Jul 2020 06:26:09 +0000 Message-Id: <1595226378-81144-6-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 05/13] regex/mlx5: add engine status check X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit checks the engine status. Signed-off-by: Ori Kam --- drivers/common/mlx5/mlx5_prm.h | 91 ++++++++++ drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/meson.build | 1 + drivers/regex/mlx5/mlx5_regex.c | 28 +++ drivers/regex/mlx5/mlx5_regex.h | 8 + drivers/regex/mlx5/mlx5_regex_devx.c | 61 +++++++ drivers/regex/mlx5/mlx5_rxp_csrs.h | 338 +++++++++++++++++++++++++++++++++++ 7 files changed, 528 insertions(+) create mode 100644 drivers/regex/mlx5/mlx5_regex_devx.c create mode 100644 drivers/regex/mlx5/mlx5_rxp_csrs.h diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 364db81..a59acc7 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -806,6 +806,10 @@ enum { MLX5_CMD_OP_CREATE_GENERAL_OBJECT = 0xa00, MLX5_CMD_OP_MODIFY_GENERAL_OBJECT = 0xa01, MLX5_CMD_OP_QUERY_GENERAL_OBJECT = 0xa02, + MLX5_CMD_SET_REGEX_PARAMS = 0xb04, + MLX5_CMD_QUERY_REGEX_PARAMS = 0xb05, + MLX5_CMD_SET_REGEX_REGISTERS = 0xb06, + MLX5_CMD_QUERY_REGEX_REGISTERS = 0xb07, MLX5_CMD_OP_ACCESS_REGISTER_USER = 0xB0C, }; @@ -2680,6 +2684,93 @@ struct mlx5_ifc_parse_graph_flex_out_bits { struct mlx5_ifc_parse_graph_flex_bits capability; }; +struct regexp_params_field_select_bits { + u8 reserved_at_0[0x1e]; + u8 stop_engine[0x1]; + u8 db_umem_id[0x1]; +}; + +struct mlx5_ifc_regexp_params_bits { + u8 reserved_at_0[0x1f]; + u8 stop_engine[0x1]; + u8 db_umem_id[0x20]; + u8 db_umem_offset[0x40]; + u8 reserved_at_80[0x100]; +}; + +struct mlx5_ifc_set_regexp_params_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + struct regexp_params_field_select_bits field_select; + struct mlx5_ifc_regexp_params_bits regexp_params; +}; + +struct mlx5_ifc_set_regexp_params_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved_at_18[0x40]; +}; + +struct mlx5_ifc_query_regexp_params_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + u8 reserved[0x20]; +}; + +struct mlx5_ifc_query_regexp_params_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved[0x40]; + struct mlx5_ifc_regexp_params_bits regexp_params; +}; + +struct mlx5_ifc_set_regexp_register_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + u8 register_address[0x20]; + u8 register_data[0x20]; + u8 reserved[0x40]; +}; + +struct mlx5_ifc_set_regexp_register_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved[0x40]; +}; + +struct mlx5_ifc_query_regexp_register_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + u8 register_address[0x20]; +}; + +struct mlx5_ifc_query_regexp_register_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved[0x20]; + u8 register_data[0x20]; +}; + /* CQE format mask. */ #define MLX5E_CQE_FORMAT_MASK 0xc diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 8fe61f8..01853246 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -9,6 +9,7 @@ LIB = librte_pmd_mlx5_regex.a # Sources. SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_rxp.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_devx.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index bfb25e2..9354145 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -12,6 +12,7 @@ deps += ['common_mlx5', 'bus_pci', 'bus_mlx5_pci', 'eal', 'regexdev'] sources = files( 'mlx5_regex.c', 'mlx5_rxp.c', + 'mlx5_regex_devx.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 1f4d967..f37c2df 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -19,6 +19,7 @@ #include "mlx5_regex.h" #include "mlx5_regex_utils.h" +#include "mlx5_rxp_csrs.h" int mlx5_regex_logtype; @@ -53,6 +54,28 @@ mlx5_glue->free_device_list(ibv_list); return ibv_match; } +static int +mlx5_regex_engines_status(struct ibv_context *ctx, int num_engines) +{ + uint32_t fpga_ident = 0; + int err; + int i; + + for (i = 0; i < num_engines; i++) { + err = mlx5_devx_regex_register_read(ctx, i, + MLX5_RXP_CSR_IDENTIFIER, + &fpga_ident); + fpga_ident = (fpga_ident & (0x0000FFFF)); + if (err || fpga_ident != MLX5_RXP_IDENTIFIER) { + DRV_LOG(ERR, "Failed setup RXP %d err %d database " + "memory 0x%x", i, err, fpga_ident); + if (!err) + err = EINVAL; + return err; + } + } + return 0; +} static void mlx5_regex_get_name(char *name, struct rte_pci_device *pci_dev __rte_unused) @@ -99,6 +122,11 @@ rte_errno = ENOTSUP; goto error; } + if (mlx5_regex_engines_status(ctx, 2)) { + DRV_LOG(ERR, "RegEx engine error."); + rte_errno = ENOMEM; + goto error; + } priv = rte_zmalloc("mlx5 regex device private", sizeof(*priv), RTE_CACHE_LINE_SIZE); if (!priv) { diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 9d0fc16..082d134 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -5,6 +5,8 @@ #ifndef MLX5_REGEX_H #define MLX5_REGEX_H +#include + struct mlx5_regex_priv { TAILQ_ENTRY(mlx5_regex_priv) next; struct ibv_context *ctx; /* Device context. */ @@ -16,4 +18,10 @@ struct mlx5_regex_priv { int mlx5_regex_info_get(struct rte_regexdev *dev, struct rte_regexdev_info *info); +/* mlx5_regex_devx.c */ +int mlx5_devx_regex_register_write(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t data); +int mlx5_devx_regex_register_read(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t *data); + #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_devx.c b/drivers/regex/mlx5/mlx5_regex_devx.c new file mode 100644 index 0000000..1ffc008 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex_devx.c @@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include +#include + +#include +#include +#include + +#include "mlx5_regex.h" +#include "mlx5_regex_utils.h" + +int +mlx5_devx_regex_register_write(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t data) +{ + uint32_t out[MLX5_ST_SZ_DW(set_regexp_register_out)] = {0}; + uint32_t in[MLX5_ST_SZ_DW(set_regexp_register_in)] = {0}; + int ret; + + MLX5_SET(set_regexp_register_in, in, opcode, + MLX5_CMD_SET_REGEX_REGISTERS); + MLX5_SET(set_regexp_register_in, in, engine_id, engine_id); + MLX5_SET(set_regexp_register_in, in, register_address, addr); + MLX5_SET(set_regexp_register_in, in, register_data, data); + + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Set regexp register failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + +int +mlx5_devx_regex_register_read(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t *data) +{ + uint32_t out[MLX5_ST_SZ_DW(query_regexp_register_out)] = {0}; + uint32_t in[MLX5_ST_SZ_DW(query_regexp_register_in)] = {0}; + int ret; + + MLX5_SET(query_regexp_register_in, in, opcode, + MLX5_CMD_QUERY_REGEX_REGISTERS); + MLX5_SET(query_regexp_register_in, in, engine_id, engine_id); + MLX5_SET(query_regexp_register_in, in, register_address, addr); + + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Query regexp register failed %d", ret); + rte_errno = errno; + return -errno; + } + *data = MLX5_GET(query_regexp_register_out, out, register_data); + return 0; +} diff --git a/drivers/regex/mlx5/mlx5_rxp_csrs.h b/drivers/regex/mlx5/mlx5_rxp_csrs.h new file mode 100644 index 0000000..acc927a --- /dev/null +++ b/drivers/regex/mlx5/mlx5_rxp_csrs.h @@ -0,0 +1,338 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ +#ifndef _MLX5_RXP_CSRS_H_ +#define _MLX5_RXP_CSRS_H_ + +/* + * Common to all RXP implementations + */ +#define MLX5_RXP_CSR_BASE_ADDRESS 0x0000ul +#define MLX5_RXP_RTRU_CSR_BASE_ADDRESS 0x0100ul +#define MLX5_RXP_STATS_CSR_BASE_ADDRESS 0x0200ul +#define MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS 0x0600ul + +#define MLX5_RXP_CSR_WIDTH 4 + +/* This is the identifier we expect to see in the first RXP CSR */ +#define MLX5_RXP_IDENTIFIER 0x5254 + +/* Hyperion specific BAR0 offsets */ +#define MLX5_RXP_FPGA_BASE_ADDRESS 0x0000ul +#define MLX5_RXP_PCIE_BASE_ADDRESS 0x1000ul +#define MLX5_RXP_IDMA_BASE_ADDRESS 0x2000ul +#define MLX5_RXP_EDMA_BASE_ADDRESS 0x3000ul +#define MLX5_RXP_SYSMON_BASE_ADDRESS 0xf300ul +#define MLX5_RXP_ISP_CSR_BASE_ADDRESS 0xf400ul + +/* Offset to the RXP common 4K CSR space */ +#define MLX5_RXP_PCIE_CSR_BASE_ADDRESS 0xf000ul + +/* FPGA CSRs */ + +#define MLX5_RXP_FPGA_VERSION (MLX5_RXP_FPGA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) + +/* PCIe CSRs */ +#define MLX5_RXP_PCIE_INIT_ISR (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_PCIE_INIT_IMR (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_PCIE_INIT_CFG_STAT (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +#define MLX5_RXP_PCIE_INIT_FLR (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 3) +#define MLX5_RXP_PCIE_INIT_CTRL (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 4) + +/* IDMA CSRs */ +#define MLX5_RXP_IDMA_ISR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_IDMA_IMR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_IDMA_CSR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 4) +#define MLX5_RXP_IDMA_CSR_RST_MSK 0x0001 +#define MLX5_RXP_IDMA_CSR_PDONE_MSK 0x0002 +#define MLX5_RXP_IDMA_CSR_INIT_MSK 0x0004 +#define MLX5_RXP_IDMA_CSR_EN_MSK 0x0008 +#define MLX5_RXP_IDMA_QCR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 5) +#define MLX5_RXP_IDMA_QCR_QAVAIL_MSK 0x00FF +#define MLX5_RXP_IDMA_QCR_QEN_MSK 0xFF00 +#define MLX5_RXP_IDMA_DCR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 6) +#define MLX5_RXP_IDMA_DWCTR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 7) +#define MLX5_RXP_IDMA_DWTOR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 8) +#define MLX5_RXP_IDMA_PADCR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 9) +#define MLX5_RXP_IDMA_DFCR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_IDMA_FOFLR0 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_IDMA_FOFLR1 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_IDMA_FOFLR2 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +#define MLX5_RXP_IDMA_FUFLR0 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_IDMA_FUFLR1 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_IDMA_FUFLR2 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) + +#define MLX5_RXP_IDMA_QCSR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 128) +#define MLX5_RXP_IDMA_QCSR_RST_MSK 0x0001 +#define MLX5_RXP_IDMA_QCSR_PDONE_MSK 0x0002 +#define MLX5_RXP_IDMA_QCSR_INIT_MSK 0x0004 +#define MLX5_RXP_IDMA_QCSR_EN_MSK 0x0008 +#define MLX5_RXP_IDMA_QDPTR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 192) +#define MLX5_RXP_IDMA_QTPTR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 256) +#define MLX5_RXP_IDMA_QDRPTR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 320) +#define MLX5_RXP_IDMA_QDRALR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 384) +#define MLX5_RXP_IDMA_QDRAHR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 385) + +/* EDMA CSRs */ +#define MLX5_RXP_EDMA_ISR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_EDMA_IMR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_EDMA_CSR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 4) +#define MLX5_RXP_EDMA_CSR_RST_MSK 0x0001 +#define MLX5_RXP_EDMA_CSR_PDONE_MSK 0x0002 +#define MLX5_RXP_EDMA_CSR_INIT_MSK 0x0004 +#define MLX5_RXP_EDMA_CSR_EN_MSK 0x0008 +#define MLX5_RXP_EDMA_QCR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 5) +#define MLX5_RXP_EDMA_QCR_QAVAIL_MSK 0x00FF +#define MLX5_RXP_EDMA_QCR_QEN_MSK 0xFF00 +#define MLX5_RXP_EDMA_DCR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 6) +#define MLX5_RXP_EDMA_DWCTR (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 7) +#define MLX5_RXP_EDMA_DWTOR (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 8) +#define MLX5_RXP_EDMA_DFCR (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_EDMA_FOFLR0 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_EDMA_FOFLR1 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_EDMA_FOFLR2 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +#define MLX5_RXP_EDMA_FUFLR0 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_EDMA_FUFLR1 (MLX5_RXP_EDMA_BASE_ADDRESS +\ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_EDMA_FUFLR2 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) + +#define MLX5_RXP_EDMA_QCSR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 128) +#define MLX5_RXP_EDMA_QCSR_RST_MSK 0x0001 +#define MLX5_RXP_EDMA_QCSR_PDONE_MSK 0x0002 +#define MLX5_RXP_EDMA_QCSR_INIT_MSK 0x0004 +#define MLX5_RXP_EDMA_QCSR_EN_MSK 0x0008 +#define MLX5_RXP_EDMA_QTPTR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 256) +#define MLX5_RXP_EDMA_QDRPTR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 320) +#define MLX5_RXP_EDMA_QDRALR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 384) +#define MLX5_RXP_EDMA_QDRAHR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 385) + +/* Main CSRs */ +#define MLX5_RXP_CSR_IDENTIFIER (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_CSR_REVISION (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_CSR_CAPABILITY_0 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +#define MLX5_RXP_CSR_CAPABILITY_1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 3) +#define MLX5_RXP_CSR_CAPABILITY_2 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 4) +#define MLX5_RXP_CSR_CAPABILITY_3 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 5) +#define MLX5_RXP_CSR_CAPABILITY_4 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 6) +#define MLX5_RXP_CSR_CAPABILITY_5 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 7) +#define MLX5_RXP_CSR_CAPABILITY_6 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 8) +#define MLX5_RXP_CSR_CAPABILITY_7 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 9) +#define MLX5_RXP_CSR_STATUS (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_CSR_STATUS_INIT_DONE 0x0001 +#define MLX5_RXP_CSR_STATUS_GOING 0x0008 +#define MLX5_RXP_CSR_STATUS_IDLE 0x0040 +#define MLX5_RXP_CSR_STATUS_TRACKER_OK 0x0080 +#define MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT 0x0100 +#define MLX5_RXP_CSR_FIFO_STATUS_0 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 11) +#define MLX5_RXP_CSR_FIFO_STATUS_1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 12) +#define MLX5_RXP_CSR_JOB_DDOS_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 13) +/* 14 + 15 reserved */ +#define MLX5_RXP_CSR_CORE_CLK_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_CSR_WRITE_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_CSR_JOB_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +#define MLX5_RXP_CSR_JOB_ERROR_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 19) +#define MLX5_RXP_CSR_JOB_BYTE_COUNT0 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 20) +#define MLX5_RXP_CSR_JOB_BYTE_COUNT1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 21) +#define MLX5_RXP_CSR_RESPONSE_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 22) +#define MLX5_RXP_CSR_MATCH_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 23) +#define MLX5_RXP_CSR_CTRL (MLX5_RXP_CSR_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_CSR_CTRL_INIT 0x0001 +#define MLX5_RXP_CSR_CTRL_GO 0x0008 +#define MLX5_RXP_CSR_MAX_MATCH (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_CSR_MAX_PREFIX (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) +#define MLX5_RXP_CSR_MAX_PRI_THREAD (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 27) +#define MLX5_RXP_CSR_MAX_LATENCY (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 28) +#define MLX5_RXP_CSR_SCRATCH_1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 29) +#define MLX5_RXP_CSR_CLUSTER_MASK (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 30) +#define MLX5_RXP_CSR_INTRA_CLUSTER_MASK (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 31) + +/* Runtime Rule Update CSRs */ +/* 0 + 1 reserved */ +#define MLX5_RXP_RTRU_CSR_CAPABILITY (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +/* 3-9 reserved */ +#define MLX5_RXP_RTRU_CSR_STATUS (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_RTRU_CSR_STATUS_UPDATE_DONE 0x0002 +#define MLX5_RXP_RTRU_CSR_STATUS_IM_INIT_DONE 0x0010 +#define MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE 0x0020 +#define MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE 0x0040 +#define MLX5_RXP_RTRU_CSR_STATUS_EM_INIT_DONE 0x0080 +#define MLX5_RXP_RTRU_CSR_FIFO_STAT (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 11) +/* 12-15 reserved */ +#define MLX5_RXP_RTRU_CSR_CHECKSUM_0 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_RTRU_CSR_CHECKSUM_1 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_RTRU_CSR_CHECKSUM_2 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +/* 19 + 20 reserved */ +#define MLX5_RXP_RTRU_CSR_RTRU_COUNT (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 21) +#define MLX5_RXP_RTRU_CSR_ROF_REV (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 22) +/* 23 reserved */ +#define MLX5_RXP_RTRU_CSR_CTRL (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT 0x0001 +#define MLX5_RXP_RTRU_CSR_CTRL_GO 0x0002 +#define MLX5_RXP_RTRU_CSR_CTRL_SIP 0x0004 +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_MASK (3 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2_EM (0 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2 (1 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2 (2 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_EM (3 << 4) +#define MLX5_RXP_RTRU_CSR_ADDR (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_RTRU_CSR_DATA_0 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) +#define MLX5_RXP_RTRU_CSR_DATA_1 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 27) +/* 28-31 reserved */ + +/* Statistics CSRs */ +#define MLX5_RXP_STATS_CSR_CLUSTER (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_STATS_CSR_L2_CACHE (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_STATS_CSR_MPFE_FIFO (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_STATS_CSR_PE (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 28) +#define MLX5_RXP_STATS_CSR_CP (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 30) +#define MLX5_RXP_STATS_CSR_DP (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 31) + +/* Sysmon Stats CSRs */ +#define MLX5_RXP_SYSMON_CSR_T_FPGA (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_SYSMON_CSR_V_VCCINT (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_SYSMON_CSR_V_VCCAUX (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +#define MLX5_RXP_SYSMON_CSR_T_U1 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 20) +#define MLX5_RXP_SYSMON_CSR_I_EDG12V (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 21) +#define MLX5_RXP_SYSMON_CSR_I_VCC3V3 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 22) +#define MLX5_RXP_SYSMON_CSR_I_VCC2V5 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 23) +#define MLX5_RXP_SYSMON_CSR_T_U2 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 28) +#define MLX5_RXP_SYSMON_CSR_I_AUX12V (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 29) +#define MLX5_RXP_SYSMON_CSR_I_VCC1V8 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 30) +#define MLX5_RXP_SYSMON_CSR_I_VDDR3 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 31) + +/* In Service Programming CSRs */ + +/* RXP-F1 and RXP-ZYNQ specific CSRs */ +#define MLX5_RXP_MQ_CP_BASE (0x0500ul) +#define MLX5_RXP_MQ_CP_CAPABILITY_BASE (MLX5_RXP_MQ_CP_BASE + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_0 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 0 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_1 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 1 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_2 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_3 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 3 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_BASE (MLX5_RXP_MQ_CP_BASE + \ + 11 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C0 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 0 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C1 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 1 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C2 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C3 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 3 * MLX5_RXP_CSR_WIDTH) + +/* Royalty tracker / licensing related CSRs */ +#define MLX5_RXPL__CSR_IDENT (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 0 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__IDENTIFIER 0x4c505852 /* MLX5_RXPL_ */ +#define MLX5_RXPL__CSR_CAPABILITY (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__TYPE_MASK 0xFF +#define MLX5_RXPL__TYPE_NONE 0 +#define MLX5_RXPL__TYPE_MAXIM 1 +#define MLX5_RXPL__TYPE_XILINX_DNA 2 +#define MLX5_RXPL__CSR_STATUS (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 10 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__CSR_IDENT_0 (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 16 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__CSR_KEY_0 (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 24 * MLX5_RXP_CSR_WIDTH) + +#endif /* _MLX5_RXP_CSRS_H_ */ From patchwork Mon Jul 20 06:26:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74466 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 38B5DA0540; Mon, 20 Jul 2020 08:27:41 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 45BB92BF3; Mon, 20 Jul 2020 08:26:54 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 47BE91BEDF for ; Mon, 20 Jul 2020 08:26:48 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:26:42 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKui008177; Mon, 20 Jul 2020 09:26:42 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Mon, 20 Jul 2020 06:26:10 +0000 Message-Id: <1595226378-81144-7-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 06/13] regex/mlx5: add configure function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit implements the configure function. This function is responsible to configure the RegEx engine. Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 2 + drivers/regex/mlx5/mlx5_regex.h | 15 +++ drivers/regex/mlx5/mlx5_rxp.c | 235 +++++++++++++++++++++++++++++++++++++++- 3 files changed, 251 insertions(+), 1 deletion(-) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index f37c2df..f834ac0 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -25,6 +25,7 @@ const struct rte_regexdev_ops mlx5_regexdev_ops = { .dev_info_get = mlx5_regex_info_get, + .dev_configure = mlx5_regex_configure, }; static struct ibv_device * @@ -145,6 +146,7 @@ priv->regexdev->dev_ops = &mlx5_regexdev_ops; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; + priv->regexdev->state = RTE_REGEXDEV_READY; return 0; error: diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 082d134..f17b4f8 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -7,16 +7,31 @@ #include +struct mlx5_regex_sq { + uint32_t nb_desc; /* Number of desc for this object. */ +}; + +struct mlx5_regex_qp { + uint32_t flags; /* QP user flags. */ + uint32_t nb_desc; /* Total number of desc for this qp. */ + struct mlx5_regex_sq *sqs; /* Pointer to sq array. */ +}; + struct mlx5_regex_priv { TAILQ_ENTRY(mlx5_regex_priv) next; struct ibv_context *ctx; /* Device context. */ struct rte_pci_device *pci_dev; struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ + uint16_t nb_queues; /* Number of queues. */ + struct mlx5_regex_qp *qps; /* Pointer to the qp array. */ + uint16_t nb_max_matches; /* Max number of matches. */ }; /* mlx5_rxp.c */ int mlx5_regex_info_get(struct rte_regexdev *dev, struct rte_regexdev_info *info); +int mlx5_regex_configure(struct rte_regexdev *dev, + const struct rte_regexdev_config *cfg); /* mlx5_regex_devx.c */ int mlx5_devx_regex_register_write(struct ibv_context *ctx, int engine_id, diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c index a5a6f15..18e2338 100644 --- a/drivers/regex/mlx5/mlx5_rxp.c +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -2,13 +2,22 @@ * Copyright 2020 Mellanox Technologies, Ltd */ +#include + #include #include +#include #include #include #include +#include +#include +#include + #include "mlx5_regex.h" +#include "mlx5_regex_utils.h" +#include "mlx5_rxp_csrs.h" #define MLX5_REGEX_MAX_MATCHES 255 #define MLX5_REGEX_MAX_PAYLOAD_SIZE UINT16_MAX @@ -17,7 +26,7 @@ int mlx5_regex_info_get(struct rte_regexdev *dev __rte_unused, - struct rte_regexdev_info *info) + struct rte_regexdev_info *info) { info->max_matches = MLX5_REGEX_MAX_MATCHES; info->max_payload_size = MLX5_REGEX_MAX_PAYLOAD_SIZE; @@ -27,3 +36,227 @@ info->rule_flags = 0; return 0; } + +static int +rxp_poll_csr_for_value(struct ibv_context *ctx, uint32_t *value, + uint32_t address, uint32_t expected_value, + uint32_t expected_mask, uint32_t timeout_ms, uint8_t id) +{ + unsigned int i; + int ret; + + ret = -EBUSY; + for (i = 0; i < timeout_ms; i++) { + if (mlx5_devx_regex_register_read(ctx, id, address, value)) + return -1; + + if ((*value & expected_mask) == expected_value) { + ret = 0; + break; + } + rte_delay_us(1000); + } + return ret; +} + +static int +rxp_start_engine(struct ibv_context *ctx, uint8_t id) +{ + uint32_t ctrl; + int ret; + + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + ctrl |= MLX5_RXP_CSR_CTRL_GO; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + return ret; +} + +static int +rxp_stop_engine(struct ibv_context *ctx, uint8_t id) +{ + uint32_t ctrl; + int ret; + + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + ctrl &= ~MLX5_RXP_CSR_CTRL_GO; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + return ret; +} + +static int +rxp_init_rtru(struct ibv_context *ctx, uint8_t id, uint32_t init_bits) +{ + uint32_t ctrl_value; + uint32_t poll_value; + uint32_t expected_value; + uint32_t expected_mask; + int ret = 0; + + /* Read the rtru ctrl CSR */ + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + &ctrl_value); + if (ret) + return -1; + /* Clear any previous init modes */ + ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_MASK); + if (ctrl_value & MLX5_RXP_RTRU_CSR_CTRL_INIT) { + ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + } + /* Set the init_mode bits in the rtru ctrl CSR */ + ctrl_value |= init_bits; + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + /* Need to sleep for a short period after pulsing the rtru init bit. */ + rte_delay_us(20000); + /* Poll the rtru status CSR until all the init done bits are set. */ + DRV_LOG(DEBUG, "waiting for RXP rule memory to complete init"); + /* Set the init bit in the rtru ctrl CSR. */ + ctrl_value |= MLX5_RXP_RTRU_CSR_CTRL_INIT; + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + /* Clear the init bit in the rtru ctrl CSR */ + ctrl_value &= ~MLX5_RXP_RTRU_CSR_CTRL_INIT; + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + /* Check that the following bits are set in the RTRU_CSR. */ + if (init_bits == MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2) { + /* Must be incremental mode */ + expected_value = MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + } else { + expected_value = MLX5_RXP_RTRU_CSR_STATUS_IM_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + } + expected_mask = expected_value; + ret = rxp_poll_csr_for_value(ctx, &poll_value, + MLX5_RXP_RTRU_CSR_STATUS, + expected_value, expected_mask, + MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT, id); + if (ret) + return ret; + DRV_LOG(DEBUG, "rule Memory initialise: 0x%08X", poll_value); + /* Clear the init bit in the rtru ctrl CSR */ + ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + return 0; +} + +static int +rxp_init(struct mlx5_regex_priv *priv, uint8_t id) +{ + uint32_t ctrl; + uint32_t reg; + struct ibv_context *ctx = priv->ctx; + int ret; + + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + if (ctrl & MLX5_RXP_CSR_CTRL_INIT) { + ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, + ctrl); + if (ret) + return ret; + } + ctrl |= MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + if (ret) + return ret; + ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + rte_delay_us(20000); + + ret = rxp_poll_csr_for_value(ctx, &ctrl, MLX5_RXP_CSR_STATUS, + MLX5_RXP_CSR_STATUS_INIT_DONE, + MLX5_RXP_CSR_STATUS_INIT_DONE, + MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT, id); + if (ret) + return ret; + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, + ctrl); + if (ret) + return ret; + rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); + ret = rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); + if (ret) + return ret; + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CAPABILITY_5, + ®); + if (ret) + return ret; + DRV_LOG(DEBUG, "max matches: %d, DDOS threshold: %d", reg >> 16, + reg & 0xffff); + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_MATCH, + priv->nb_max_matches); + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_LATENCY, 0); + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_PRI_THREAD, 0); + return ret; +} + +int +mlx5_regex_configure(struct rte_regexdev *dev, + const struct rte_regexdev_config *cfg) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + int ret; + uint8_t id; + + priv->nb_queues = cfg->nb_queue_pairs; + priv->qps = rte_zmalloc(NULL, sizeof(struct mlx5_regex_qp) * + priv->nb_queues, 0); + if (!priv->nb_queues) { + DRV_LOG(ERR, "can't allocate qps memory"); + rte_errno = ENOMEM; + return -rte_errno; + } + priv->nb_max_matches = cfg->nb_max_matches; + for (id = 0; id < 2; id++) { + ret = rxp_stop_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "can't stop engine."); + rte_errno = ENODEV; + return -rte_errno; + } + ret = rxp_init(priv, id); + if (ret) { + DRV_LOG(ERR, "can't init engine."); + rte_errno = ENODEV; + return -rte_errno; + } + ret = mlx5_devx_regex_register_write(priv->ctx, id, + MLX5_RXP_CSR_MAX_MATCH, + priv->nb_max_matches); + if (ret) { + DRV_LOG(ERR, "can't update number of matches."); + rte_errno = ENODEV; + goto configure_error; + } + ret = rxp_start_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "can't start engine."); + rte_errno = ENODEV; + goto configure_error; + } + + } + return 0; +configure_error: + if (priv->qps) + rte_free(priv->qps); + return -rte_errno; +} From patchwork Mon Jul 20 06:26:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74467 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2F1C2A0540; Mon, 20 Jul 2020 08:27:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 780631BF0B; Mon, 20 Jul 2020 08:26:55 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 6ACA31BEE3 for ; Mon, 20 Jul 2020 08:26:48 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:26:46 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKuj008177; Mon, 20 Jul 2020 09:26:46 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Francis Kelly Date: Mon, 20 Jul 2020 06:26:11 +0000 Message-Id: <1595226378-81144-8-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 07/13] regex/mlx5: add program rules support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Francis Kelly This commit introduce the ability to program rules to the RegEx engine. Signed-off-by: Francis Kelly Acked-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 34 ++ drivers/regex/mlx5/mlx5_regex.h | 52 ++- drivers/regex/mlx5/mlx5_regex_devx.c | 67 +++ drivers/regex/mlx5/mlx5_rxp.c | 842 +++++++++++++++++++++++++++++++++-- drivers/regex/mlx5/mlx5_rxp.h | 138 ++++++ 5 files changed, 1084 insertions(+), 49 deletions(-) create mode 100644 drivers/regex/mlx5/mlx5_rxp.h diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index f834ac0..72da3e2 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -26,6 +26,7 @@ const struct rte_regexdev_ops mlx5_regexdev_ops = { .dev_info_get = mlx5_regex_info_get, .dev_configure = mlx5_regex_configure, + .dev_db_import = mlx5_regex_rules_db_import, }; static struct ibv_device * @@ -136,6 +137,9 @@ goto error; } priv->ctx = ctx; + priv->nb_engines = 2; /* attr.regexp_num_of_engines */ + /* Default RXP programming mode to Shared. */ + priv->prog_mode = MLX5_RXP_SHARED_PROG_MODE; mlx5_regex_get_name(name, pci_dev); priv->regexdev = rte_regexdev_register(name); if (priv->regexdev == NULL) { @@ -143,6 +147,24 @@ rte_errno = rte_errno ? rte_errno : EINVAL; goto error; } + ret = mlx5_glue->devx_query_eqn(ctx, 0, &priv->eqn); + if (ret) { + DRV_LOG(ERR, "can't query event queue number."); + rte_errno = ENOMEM; + goto error; + } + priv->uar = mlx5_glue->devx_alloc_uar(ctx, 0); + if (!priv->uar) { + DRV_LOG(ERR, "can't allocate uar."); + rte_errno = ENOMEM; + goto error; + } + priv->pd = mlx5_glue->alloc_pd(ctx); + if (!priv->pd) { + DRV_LOG(ERR, "can't allocate pd."); + rte_errno = ENOMEM; + goto error; + } priv->regexdev->dev_ops = &mlx5_regexdev_ops; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; @@ -150,6 +172,12 @@ return 0; error: + if (priv->pd) + mlx5_glue->dealloc_pd(priv->pd); + if (priv->uar) + mlx5_glue->devx_free_uar(priv->uar); + if (priv->regexdev) + rte_regexdev_unregister(priv->regexdev); if (ctx) mlx5_glue->close_device(ctx); if (priv) @@ -170,6 +198,12 @@ return 0; priv = dev->data->dev_private; if (priv) { + if (priv->pd) + mlx5_glue->dealloc_pd(priv->pd); + if (priv->uar) + mlx5_glue->devx_free_uar(priv->uar); + if (priv->regexdev) + rte_regexdev_unregister(priv->regexdev); if (priv->ctx) mlx5_glue->close_device(priv->ctx); if (priv->regexdev) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index f17b4f8..a2d2c9e 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -7,14 +7,47 @@ #include +#include +#include + +#include + +#include "mlx5_rxp.h" + struct mlx5_regex_sq { - uint32_t nb_desc; /* Number of desc for this object. */ + uint16_t log_nb_desc; /* Log 2 number of desc for this object. */ + struct mlx5_devx_obj *obj; /* The SQ DevX object. */ + int64_t dbr_offset; /* Door bell record offset. */ + uint32_t dbr_umem; /* Door bell record umem id. */ + volatile struct mlx5_cqe *wqe; /* The SQ ring buffer. */ + struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ +}; + +struct mlx5_regex_cq { + uint32_t log_nb_desc; /* Log 2 number of desc for this object. */ + struct mlx5_devx_obj *obj; /* The CQ DevX object. */ + int64_t dbr_offset; /* Door bell record offset. */ + uint32_t dbr_umem; /* Door bell record umem id. */ + volatile struct mlx5_cqe *cqe; /* The CQ ring buffer. */ + struct mlx5dv_devx_umem *cqe_umem; /* CQ buffer umem. */ }; struct mlx5_regex_qp { uint32_t flags; /* QP user flags. */ uint32_t nb_desc; /* Total number of desc for this qp. */ struct mlx5_regex_sq *sqs; /* Pointer to sq array. */ + uint16_t nb_obj; /* Number of sq objects. */ + struct mlx5_regex_cq cq; /* CQ struct. */ +}; + +struct mlx5_regex_db { + void *ptr; /* Pointer to the db memory. */ + uint32_t len; /* The memory len. */ + bool active; /* Active flag. */ + uint8_t db_assigned_to_eng_num; + /**< To which engine the db is connected. */ + struct mlx5_regex_umem umem; + /**< The umem struct. */ }; struct mlx5_regex_priv { @@ -25,6 +58,14 @@ struct mlx5_regex_priv { uint16_t nb_queues; /* Number of queues. */ struct mlx5_regex_qp *qps; /* Pointer to the qp array. */ uint16_t nb_max_matches; /* Max number of matches. */ + enum mlx5_rxp_program_mode prog_mode; + struct mlx5_regex_db db[MLX5_RXP_MAX_ENGINES + + MLX5_RXP_EM_COUNT]; + uint32_t nb_engines; /* Number of RegEx engines. */ + struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ + uint32_t eqn; /* EQ number. */ + struct mlx5dv_devx_uar *uar; /* UAR object. */ + struct ibv_pd *pd; }; /* mlx5_rxp.c */ @@ -32,11 +73,20 @@ int mlx5_regex_info_get(struct rte_regexdev *dev, struct rte_regexdev_info *info); int mlx5_regex_configure(struct rte_regexdev *dev, const struct rte_regexdev_config *cfg); +int mlx5_regex_rules_db_import(struct rte_regexdev *dev, + const char *rule_db, uint32_t rule_db_len); /* mlx5_regex_devx.c */ int mlx5_devx_regex_register_write(struct ibv_context *ctx, int engine_id, uint32_t addr, uint32_t data); int mlx5_devx_regex_register_read(struct ibv_context *ctx, int engine_id, uint32_t addr, uint32_t *data); +int mlx5_devx_regex_database_stop(void *ctx, uint8_t engine); +int mlx5_devx_regex_database_resume(void *ctx, uint8_t engine); +int mlx5_devx_regex_database_program(void *ctx, uint8_t engine, + uint32_t umem_id, uint64_t umem_offset); +/* mlx5_regex_control.c */ +int mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, + const struct rte_regexdev_qp_conf *cfg); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_devx.c b/drivers/regex/mlx5/mlx5_regex_devx.c index 1ffc008..2199687 100644 --- a/drivers/regex/mlx5/mlx5_regex_devx.c +++ b/drivers/regex/mlx5/mlx5_regex_devx.c @@ -59,3 +59,70 @@ *data = MLX5_GET(query_regexp_register_out, out, register_data); return 0; } + +int +mlx5_devx_regex_database_stop(void *ctx, uint8_t engine) +{ + uint32_t out[MLX5_ST_SZ_DW(set_regexp_params_out)] = {0}; + uint32_t in[MLX5_ST_SZ_DW(set_regexp_params_in)] = {0}; + int ret; + + MLX5_SET(set_regexp_params_in, in, opcode, MLX5_CMD_SET_REGEX_PARAMS); + MLX5_SET(set_regexp_params_in, in, engine_id, engine); + MLX5_SET(set_regexp_params_in, in, regexp_params.stop_engine, 1); + MLX5_SET(set_regexp_params_in, in, field_select.stop_engine, 1); + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Database stop failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + +int +mlx5_devx_regex_database_resume(void *ctx, uint8_t engine) +{ + uint32_t out[MLX5_ST_SZ_DW(set_regexp_params_out)] = {0}; + uint32_t in[MLX5_ST_SZ_DW(set_regexp_params_in)] = {0}; + int ret; + + MLX5_SET(set_regexp_params_in, in, opcode, MLX5_CMD_SET_REGEX_PARAMS); + MLX5_SET(set_regexp_params_in, in, engine_id, engine); + MLX5_SET(set_regexp_params_in, in, regexp_params.stop_engine, 0); + MLX5_SET(set_regexp_params_in, in, field_select.stop_engine, 1); + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Database start failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + +int +mlx5_devx_regex_database_program(void *ctx, uint8_t engine, uint32_t umem_id, + uint64_t umem_offset) +{ + uint32_t out[MLX5_ST_SZ_DW(set_regexp_params_out)] = {0}; + uint32_t in[MLX5_ST_SZ_DW(set_regexp_params_in)] = {0}; + int ret; + + MLX5_SET(set_regexp_params_in, in, opcode, MLX5_CMD_SET_REGEX_PARAMS); + MLX5_SET(set_regexp_params_in, in, engine_id, engine); + MLX5_SET(set_regexp_params_in, in, regexp_params.db_umem_id, umem_id); + MLX5_SET64(set_regexp_params_in, in, regexp_params.db_umem_offset, + umem_offset); + MLX5_SET(set_regexp_params_in, in, field_select.db_umem_id, 1); + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Database program failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c index 18e2338..5dfba26 100644 --- a/drivers/regex/mlx5/mlx5_rxp.c +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -2,8 +2,6 @@ * Copyright 2020 Mellanox Technologies, Ltd */ -#include - #include #include #include @@ -14,15 +12,100 @@ #include #include #include +#include #include "mlx5_regex.h" #include "mlx5_regex_utils.h" #include "mlx5_rxp_csrs.h" +#include "mlx5_rxp.h" + +#define MLX5_REGEX_MAX_MATCHES MLX5_RXP_MAX_MATCHES +#define MLX5_REGEX_MAX_PAYLOAD_SIZE MLX5_RXP_MAX_JOB_LENGTH +#define MLX5_REGEX_MAX_RULES_PER_GROUP UINT32_MAX +#define MLX5_REGEX_MAX_GROUPS MLX5_RXP_MAX_SUBSETS + +/* Private Declarations */ +static int +rxp_poll_csr_for_value(struct ibv_context *ctx, uint32_t *value, + uint32_t address, uint32_t expected_value, + uint32_t expected_mask, uint32_t timeout_ms, uint8_t id); +static int +mlnx_set_database(struct mlx5_regex_priv *priv, uint8_t id, uint8_t db_to_use); +static int +mlnx_resume_database(struct mlx5_regex_priv *priv, uint8_t id); +static int +mlnx_update_database(struct mlx5_regex_priv *priv, uint8_t id); +static int +program_rxp_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint8_t id); +static int +rxp_init_eng(struct mlx5_regex_priv *priv, uint8_t id); +static int +write_private_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, + uint8_t id); +static int +write_shared_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint32_t count, + uint8_t db_to_program); +static int +rxp_db_setup(struct mlx5_regex_priv *priv); +static void +rxp_dump_csrs(struct ibv_context *ctx, uint8_t id); +static int +rxp_write_rules_via_cp(struct ibv_context *ctx, + struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id); +static int +rxp_flush_rules(struct ibv_context *ctx, struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id); +static int +rxp_start_engine(struct ibv_context *ctx, uint8_t id); +static int +rxp_stop_engine(struct ibv_context *ctx, uint8_t id); + +static void __rte_unused +rxp_dump_csrs(struct ibv_context *ctx __rte_unused, uint8_t id __rte_unused) +{ + uint32_t reg, i; -#define MLX5_REGEX_MAX_MATCHES 255 -#define MLX5_REGEX_MAX_PAYLOAD_SIZE UINT16_MAX -#define MLX5_REGEX_MAX_RULES_PER_GROUP UINT16_MAX -#define MLX5_REGEX_MAX_GROUPS UINT16_MAX + /* Main CSRs*/ + for (i = 0; i < MLX5_RXP_CSR_NUM_ENTRIES; i++) { + if (mlx5_devx_regex_register_read(ctx, id, + (MLX5_RXP_CSR_WIDTH * i) + + MLX5_RXP_CSR_BASE_ADDRESS, + ®)) { + DRV_LOG(ERR, "Failed to read Main CSRs Engine %d!", id); + return; + } + DRV_LOG(DEBUG, "RXP Main CSRs (Eng%d) register (%d): %08x", + id, i, reg); + } + /* RTRU CSRs*/ + for (i = 0; i < MLX5_RXP_CSR_NUM_ENTRIES; i++) { + if (mlx5_devx_regex_register_read(ctx, id, + (MLX5_RXP_CSR_WIDTH * i) + + MLX5_RXP_RTRU_CSR_BASE_ADDRESS, + ®)) { + DRV_LOG(ERR, "Failed to read RTRU CSRs Engine %d!", id); + return; + } + DRV_LOG(DEBUG, "RXP RTRU CSRs (Eng%d) register (%d): %08x", + id, i, reg); + } + /* STAT CSRs */ + for (i = 0; i < MLX5_RXP_CSR_NUM_ENTRIES; i++) { + if (mlx5_devx_regex_register_read(ctx, id, + (MLX5_RXP_CSR_WIDTH * i) + + MLX5_RXP_STATS_CSR_BASE_ADDRESS, + ®)) { + DRV_LOG(ERR, "Failed to read STAT CSRs Engine %d!", id); + return; + } + DRV_LOG(DEBUG, "RXP STAT CSRs (Eng%d) register (%d): %08x", + id, i, reg); + } +} int mlx5_regex_info_get(struct rte_regexdev *dev __rte_unused, @@ -32,24 +115,120 @@ info->max_payload_size = MLX5_REGEX_MAX_PAYLOAD_SIZE; info->max_rules_per_group = MLX5_REGEX_MAX_RULES_PER_GROUP; info->max_groups = MLX5_REGEX_MAX_GROUPS; + info->max_queue_pairs = 1; info->regexdev_capa = RTE_REGEXDEV_SUPP_PCRE_GREEDY_F; info->rule_flags = 0; return 0; } +/** + * Actual writing of RXP instructions to RXP via CSRs. + */ +static int +rxp_write_rules_via_cp(struct ibv_context *ctx, + struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id) +{ + int i, ret = 0; + uint32_t tmp; + + for (i = 0; i < count; i++) { + tmp = (uint32_t)rules[i].value; + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_RTRU_CSR_DATA_0, + tmp); + tmp = (uint32_t)(rules[i].value >> 32); + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_RTRU_CSR_DATA_0 + + MLX5_RXP_CSR_WIDTH, tmp); + tmp = rules[i].addr; + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_RTRU_CSR_ADDR, + tmp); + if (ret) { + DRV_LOG(ERR, "Failed to copy instructions to RXP."); + return -1; + } + } + DRV_LOG(DEBUG, "Written %d instructions", count); + return 0; +} + +static int +rxp_flush_rules(struct ibv_context *ctx, struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id) +{ + uint32_t val, fifo_depth; + int ret; + + ret = rxp_write_rules_via_cp(ctx, rules, count, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to write rules via CSRs."); + return -1; + } + ret = mlx5_devx_regex_register_read(ctx, id, + MLX5_RXP_RTRU_CSR_CAPABILITY, + &fifo_depth); + if (ret) { + DRV_LOG(ERR, "CSR read failed!"); + return -1; + } + ret = rxp_poll_csr_for_value(ctx, &val, MLX5_RXP_RTRU_CSR_FIFO_STAT, + count, ~0, + MLX5_RXP_POLL_CSR_FOR_VALUE_TIMEOUT, id); + if (ret < 0) { + DRV_LOG(ERR, "Rules not rx by RXP: credit: %d, depth: %d", val, + fifo_depth); + return ret; + } + DRV_LOG(DEBUG, "RTRU FIFO depth: 0x%x", fifo_depth); + DRV_LOG(DEBUG, "Rules flush took %d cycles.", ret); + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + &val); + if (ret) { + DRV_LOG(ERR, "CSR read failed!"); + return -1; + } + val |= MLX5_RXP_RTRU_CSR_CTRL_GO; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + val); + ret = rxp_poll_csr_for_value(ctx, &val, MLX5_RXP_RTRU_CSR_STATUS, + MLX5_RXP_RTRU_CSR_STATUS_UPDATE_DONE, + MLX5_RXP_RTRU_CSR_STATUS_UPDATE_DONE, + MLX5_RXP_POLL_CSR_FOR_VALUE_TIMEOUT, id); + if (ret < 0) { + DRV_LOG(ERR, "Rules update timeout: 0x%08X", val); + return ret; + } + DRV_LOG(DEBUG, "Rules update took %d cycles", ret); + if (mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + &val)) { + DRV_LOG(ERR, "CSR read failed!"); + return -1; + } + val &= ~(MLX5_RXP_RTRU_CSR_CTRL_GO); + if (mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + val)) { + DRV_LOG(ERR, "CSR write write failed!"); + return -1; + } + + DRV_LOG(DEBUG, "RXP Flush rules finished."); + return 0; +} + static int rxp_poll_csr_for_value(struct ibv_context *ctx, uint32_t *value, uint32_t address, uint32_t expected_value, uint32_t expected_mask, uint32_t timeout_ms, uint8_t id) { unsigned int i; - int ret; + int ret = 0; ret = -EBUSY; for (i = 0; i < timeout_ms; i++) { if (mlx5_devx_regex_register_read(ctx, id, address, value)) return -1; - if ((*value & expected_mask) == expected_value) { ret = 0; break; @@ -69,6 +248,7 @@ if (ret) return ret; ctrl |= MLX5_RXP_CSR_CTRL_GO; + ctrl |= MLX5_RXP_CSR_CTRL_DISABLE_L2C; ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); return ret; } @@ -96,23 +276,23 @@ uint32_t expected_mask; int ret = 0; - /* Read the rtru ctrl CSR */ + /* Read the rtru ctrl CSR. */ ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, &ctrl_value); if (ret) return -1; - /* Clear any previous init modes */ + /* Clear any previous init modes. */ ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_MASK); if (ctrl_value & MLX5_RXP_RTRU_CSR_CTRL_INIT) { ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, ctrl_value); } - /* Set the init_mode bits in the rtru ctrl CSR */ + /* Set the init_mode bits in the rtru ctrl CSR. */ ctrl_value |= init_bits; mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, ctrl_value); - /* Need to sleep for a short period after pulsing the rtru init bit. */ + /* Need to sleep for a short period after pulsing the rtru init bit. */ rte_delay_us(20000); /* Poll the rtru status CSR until all the init done bits are set. */ DRV_LOG(DEBUG, "waiting for RXP rule memory to complete init"); @@ -128,11 +308,11 @@ if (init_bits == MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2) { /* Must be incremental mode */ expected_value = MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | - MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; } else { expected_value = MLX5_RXP_RTRU_CSR_STATUS_IM_INIT_DONE | - MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | - MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; } expected_mask = expected_value; ret = rxp_poll_csr_for_value(ctx, &poll_value, @@ -141,7 +321,7 @@ MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT, id); if (ret) return ret; - DRV_LOG(DEBUG, "rule Memory initialise: 0x%08X", poll_value); + DRV_LOG(DEBUG, "rule memory initialise: 0x%08X", poll_value); /* Clear the init bit in the rtru ctrl CSR */ ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, @@ -150,7 +330,206 @@ } static int -rxp_init(struct mlx5_regex_priv *priv, uint8_t id) +rxp_parse_rof(const char *buf, uint32_t len, + struct mlx5_rxp_ctl_rules_pgm **rules) +{ + static const char del[] = "\n\r"; + char *line; + char *tmp; + char *cur_pos; + uint32_t lines = 0; + uint32_t entries; + struct mlx5_rxp_rof_entry *curentry; + + tmp = rte_malloc("", len, 0); + if (!tmp) + return -ENOMEM; + memcpy(tmp, buf, len); + line = strtok(tmp, del); + while (line) { + if (line[0] != '#' && line[0] != '\0') + lines++; + line = strtok(NULL, del); + } + *rules = rte_malloc("", lines * sizeof(*curentry) + sizeof(**rules), 0); + if (!(*rules)) { + rte_free(tmp); + return -ENOMEM; + } + memset(*rules, 0, lines * sizeof(curentry) + sizeof(**rules)); + curentry = (*rules)->rules; + (*rules)->hdr.cmd = MLX5_RXP_CTL_RULES_PGM; + entries = 0; + memcpy(tmp, buf, len); + line = strtok(tmp, del); + while (line) { + if (line[0] == '#' || line[0] == '\0') { + line = strtok(NULL, del); + continue; + } + curentry->type = strtoul(line, &cur_pos, 10); + if (cur_pos == line || cur_pos[0] != ',') + goto parse_error; + cur_pos++; + curentry->addr = strtoul(cur_pos, &cur_pos, 16); + if (cur_pos[0] != ',') + goto parse_error; + cur_pos++; + curentry->value = strtoull(cur_pos, &cur_pos, 16); + if (cur_pos[0] != '\0' && cur_pos[0] != '\n') + goto parse_error; + curentry++; + entries++; + if (entries > lines) + goto parse_error; + line = strtok(NULL, del); + } + (*rules)->count = entries; + (*rules)->hdr.len = entries * sizeof(*curentry) + sizeof(**rules); + rte_free(tmp); + return 0; +parse_error: + rte_free(tmp); + if (*rules) + rte_free(*rules); + return -EINVAL; +} + +static int +mlnx_set_database(struct mlx5_regex_priv *priv, uint8_t id, uint8_t db_to_use) +{ + int ret; + uint32_t umem_id; + + ret = mlx5_devx_regex_database_stop(priv->ctx, id); + if (ret < 0) { + DRV_LOG(ERR, "stop engine failed!"); + return ret; + } + umem_id = mlx5_os_get_umem_id(priv->db[db_to_use].umem.umem); + ret = mlx5_devx_regex_database_program(priv->ctx, id, umem_id, 0); + if (ret < 0) { + DRV_LOG(ERR, "program db failed!"); + return ret; + } + return 0; +} + +static int +mlnx_resume_database(struct mlx5_regex_priv *priv, uint8_t id) +{ + mlx5_devx_regex_database_resume(priv->ctx, id); + return 0; +} + +/* + * Assign db memory for RXP programming. + */ +static int +mlnx_update_database(struct mlx5_regex_priv *priv, uint8_t id) +{ + unsigned int i; + uint8_t db_free = MLX5_RXP_DB_NOT_ASSIGNED; + uint8_t eng_assigned = MLX5_RXP_DB_NOT_ASSIGNED; + + /* Check which database rxp_eng is currently located if any? */ + for (i = 0; i < (priv->nb_engines + MLX5_RXP_EM_COUNT); + i++) { + if (priv->db[i].db_assigned_to_eng_num == id) { + eng_assigned = i; + break; + } + } + /* + * If private mode then, we can keep the same db ptr as RXP will be + * programming EM itself if necessary, however need to see if + * programmed yet. + */ + if ((priv->prog_mode == MLX5_RXP_PRIVATE_PROG_MODE) && + (eng_assigned != MLX5_RXP_DB_NOT_ASSIGNED)) + return eng_assigned; + /* Check for inactive db memory to use. */ + for (i = 0; i < (priv->nb_engines + MLX5_RXP_EM_COUNT); + i++) { + if (priv->db[i].active == true) + continue; /* Already in use, so skip db. */ + /* Set this db to active now as free to use. */ + priv->db[i].active = true; + /* Now unassign last db index in use by RXP Eng. */ + if (eng_assigned != MLX5_RXP_DB_NOT_ASSIGNED) { + priv->db[eng_assigned].active = false; + priv->db[eng_assigned].db_assigned_to_eng_num = + MLX5_RXP_DB_NOT_ASSIGNED; + + /* Set all DB memory to 0's before setting up DB. */ + memset(priv->db[i].ptr, 0x00, MLX5_MAX_DB_SIZE); + } + /* Now reassign new db index with RXP Engine. */ + priv->db[i].db_assigned_to_eng_num = id; + db_free = i; + break; + } + if (db_free == MLX5_RXP_DB_NOT_ASSIGNED) + return -1; + return db_free; +} + +/* + * Program RXP instruction db to RXP engine/s. + */ +static int +program_rxp_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint8_t id) +{ + int ret, db_free; + uint32_t rule_cnt; + + rule_cnt = rules->count; + db_free = mlnx_update_database(priv, id); + if (db_free < 0) { + DRV_LOG(ERR, "Failed to setup db memory!"); + return db_free; + } + if (priv->prog_mode == MLX5_RXP_PRIVATE_PROG_MODE) { + /* Register early to ensure RXP writes to EM use valid addr. */ + ret = mlnx_set_database(priv, id, db_free); + if (ret < 0) { + DRV_LOG(ERR, "Failed to register db memory!"); + return ret; + } + } + ret = write_private_rules(priv, rules, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to write rules!"); + return ret; + } + if (priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE) { + /* Write external rules directly to EM. */ + rules->count = rule_cnt; + /* Now write external instructions to EM. */ + ret = write_shared_rules(priv, rules, rules->hdr.len, db_free); + if (ret < 0) { + DRV_LOG(ERR, "Failed to write EM rules!"); + return ret; + } + ret = mlnx_set_database(priv, id, db_free); + if (ret < 0) { + DRV_LOG(ERR, "Failed to register db memory!"); + return ret; + } + } + ret = mlnx_resume_database(priv, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to resume engine!"); + return ret; + } + DRV_LOG(DEBUG, "Programmed RXP Engine %d\n", id); + rules->count = rule_cnt; + return 0; +} + +static int +rxp_init_eng(struct mlx5_regex_priv *priv, uint8_t id) { uint32_t ctrl; uint32_t reg; @@ -174,7 +553,6 @@ ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); rte_delay_us(20000); - ret = rxp_poll_csr_for_value(ctx, &ctrl, MLX5_RXP_CSR_STATUS, MLX5_RXP_CSR_STATUS_INIT_DONE, MLX5_RXP_CSR_STATUS_INIT_DONE, @@ -189,7 +567,6 @@ ctrl); if (ret) return ret; - rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); ret = rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); if (ret) return ret; @@ -199,8 +576,16 @@ return ret; DRV_LOG(DEBUG, "max matches: %d, DDOS threshold: %d", reg >> 16, reg & 0xffff); - ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_MATCH, - priv->nb_max_matches); + if ((reg >> 16) >= priv->nb_max_matches) + ret = mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_MATCH, + priv->nb_max_matches); + else + ret = mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_MATCH, + (reg >> 16)); + ret |= mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_PREFIX, + (reg & 0xFFFF)); ret |= mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_LATENCY, 0); ret |= mlx5_devx_regex_register_write(ctx, id, @@ -208,15 +593,389 @@ return ret; } +static int +write_private_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, + uint8_t id) +{ + unsigned int pending; + uint32_t block, reg, val, rule_cnt, rule_offset, rtru_max_num_entries; + int ret = 1; + + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) + return -EINVAL; + if (rules->hdr.len == 0 || rules->hdr.cmd < MLX5_RXP_CTL_RULES_PGM || + rules->hdr.cmd > MLX5_RXP_CTL_RULES_PGM_INCR) + return -EINVAL; + /* For a non-incremental rules program, re-init the RXP. */ + if (rules->hdr.cmd == MLX5_RXP_CTL_RULES_PGM) { + ret = rxp_init_eng(priv, id); + if (ret < 0) + return ret; + } else if (rules->hdr.cmd == MLX5_RXP_CTL_RULES_PGM_INCR) { + /* Flush RXP L1 and L2 cache by using MODE_L1_L2. */ + ret = rxp_init_rtru(priv->ctx, id, + MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2); + if (ret < 0) + return ret; + } + if (rules->count == 0) + return -EINVAL; + /* Confirm the RXP is initialised. */ + if (mlx5_devx_regex_register_read(priv->ctx, id, + MLX5_RXP_CSR_STATUS, &val)) { + DRV_LOG(ERR, "Failed to read from RXP!"); + return -ENODEV; + } + if (!(val & MLX5_RXP_CSR_STATUS_INIT_DONE)) { + DRV_LOG(ERR, "RXP not initialised..."); + return -EBUSY; + } + /* Get the RTRU maximum number of entries allowed. */ + if (mlx5_devx_regex_register_read(priv->ctx, id, + MLX5_RXP_RTRU_CSR_CAPABILITY, &rtru_max_num_entries)) { + DRV_LOG(ERR, "Failed to read RTRU capability!"); + return -ENODEV; + } + rtru_max_num_entries = (rtru_max_num_entries & 0x00FF); + rule_cnt = 0; + pending = 0; + while (rules->count > 0) { + if ((rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_INST) || + (rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_IM) || + (rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_EM)) { + if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_EM) && + (priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE)) { + /* Skip EM rules programming. */ + if (pending > 0) { + /* Flush any rules that are pending. */ + rule_offset = (rule_cnt - pending); + ret = rxp_flush_rules(priv->ctx, + &rules->rules[rule_offset], + pending, id); + if (ret < 0) { + DRV_LOG(ERR, "Flushing rules."); + return -ENODEV; + } + pending = 0; + } + rule_cnt++; + } else { + pending++; + rule_cnt++; + /* + * If parsing the last rule, or if reached the + * maximum number of rules for this batch, then + * flush the rules batch to the RXP. + */ + if ((rules->count == 1) || + (pending == rtru_max_num_entries)) { + rule_offset = (rule_cnt - pending); + ret = rxp_flush_rules(priv->ctx, + &rules->rules[rule_offset], + pending, id); + if (ret < 0) { + DRV_LOG(ERR, "Flushing rules."); + return -ENODEV; + } + pending = 0; + } + } + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_EQ) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_GTE) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_LTE) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM_EX_EM)) { + if (pending) { + /* Flush rules before checking reg values. */ + rule_offset = (rule_cnt - pending); + ret = rxp_flush_rules(priv->ctx, + &rules->rules[rule_offset], + pending, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to flush rules."); + return -ENODEV; + } + } + block = (rules->rules[rule_cnt].addr >> 16) & 0xFFFF; + if (block == 0) + reg = MLX5_RXP_CSR_BASE_ADDRESS; + else if (block == 1) + reg = MLX5_RXP_RTRU_CSR_BASE_ADDRESS; + else { + DRV_LOG(ERR, "Invalid ROF register 0x%08X!", + rules->rules[rule_cnt].addr); + return -EINVAL; + } + reg += (rules->rules[rule_cnt].addr & 0xFFFF) * + MLX5_RXP_CSR_WIDTH; + ret = mlx5_devx_regex_register_read(priv->ctx, id, + reg, &val); + if (ret) { + DRV_LOG(ERR, "RXP CSR read failed!"); + return ret; + } + if ((priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE) && + ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM_EX_EM) && + (val != rules->rules[rule_cnt].value))) { + DRV_LOG(ERR, "Unexpected value for register:"); + DRV_LOG(ERR, "reg %x" PRIu32 " got %x" PRIu32, + rules->rules[rule_cnt].addr, val); + DRV_LOG(ERR, "expected %" PRIx64 ".", + rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((priv->prog_mode == + MLX5_RXP_PRIVATE_PROG_MODE) && + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM) && + (val != rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value for register:"); + DRV_LOG(ERR, "reg %x" PRIu32 " got %x" PRIu32, + rules->rules[rule_cnt].addr, val); + DRV_LOG(ERR, "expected %" PRIx64 ".", + rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_EQ) && + (val != rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value for register:"); + DRV_LOG(ERR, "reg %x" PRIu32 " got %x" PRIu32, + rules->rules[rule_cnt].addr, val); + DRV_LOG(ERR, "expected %" PRIx64 ".", + rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_GTE) && + (val < rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value reg 0x%08X,", + rules->rules[rule_cnt].addr); + DRV_LOG(ERR, "got %X, expected >= %" PRIx64 ".", + val, rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_LTE) && + (val > rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value reg 0x%08X,", + rules->rules[rule_cnt].addr); + DRV_LOG(ERR, "got %08X expected <= %" PRIx64, + val, rules->rules[rule_cnt].value); + return -EINVAL; + } + rule_cnt++; + pending = 0; + } else { + DRV_LOG(ERR, "Error: Invalid rule type %d!", + rules->rules[rule_cnt].type); + return -EINVAL; + } + rules->count--; + } + return ret; +} + +/* + * Shared memory programming mode, here all external db instructions are written + * to EM via the host. + */ +static int +write_shared_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint32_t count, + uint8_t db_to_program) +{ + uint32_t rule_cnt, rof_rule_addr; + uint64_t tmp_write_swap[4]; + + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) + return -EINVAL; + if ((rules->count == 0) || (count == 0)) + return -EINVAL; + rule_cnt = 0; + /* + * Note the following section of code carries out a 32byte swap of + * instruction to coincide with HW 32byte swap. This may need removed + * in new variants of this programming function! + */ + while (rule_cnt < rules->count) { + if ((rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_EM) && + (priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE)) { + /* + * Note there are always blocks of 8 instructions for + * 7's written sequentially. However there is no + * guarantee that all blocks are sequential! + */ + if (count >= (rule_cnt + MLX5_RXP_INST_BLOCK_SIZE)) { + /* + * Ensure memory write not exceeding boundary + * Check essential to ensure 0x10000 offset + * accounted for! + */ + if ((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + ((rules->rules[rule_cnt + 7].addr << + MLX5_RXP_INST_OFFSET))) >= + ((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + MLX5_MAX_DB_SIZE))) { + DRV_LOG(ERR, "DB exceeded memory!"); + return -ENODEV; + } + /* + * Rule address Offset to align with RXP + * external instruction offset. + */ + rof_rule_addr = (rules->rules[rule_cnt].addr << + MLX5_RXP_INST_OFFSET); + /* 32 byte instruction swap (sw work around)! */ + tmp_write_swap[0] = le64toh( + rules->rules[(rule_cnt + 4)].value); + tmp_write_swap[1] = le64toh( + rules->rules[(rule_cnt + 5)].value); + tmp_write_swap[2] = le64toh( + rules->rules[(rule_cnt + 6)].value); + tmp_write_swap[3] = le64toh( + rules->rules[(rule_cnt + 7)].value); + /* Write only 4 of the 8 instructions. */ + memcpy((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + rof_rule_addr), &tmp_write_swap, + (sizeof(uint64_t) * 4)); + /* Write 1st 4 rules of block after last 4. */ + rof_rule_addr = (rules->rules[ + (rule_cnt + 4)].addr << + MLX5_RXP_INST_OFFSET); + tmp_write_swap[0] = le64toh( + rules->rules[(rule_cnt + 0)].value); + tmp_write_swap[1] = le64toh( + rules->rules[(rule_cnt + 1)].value); + tmp_write_swap[2] = le64toh( + rules->rules[(rule_cnt + 2)].value); + tmp_write_swap[3] = le64toh( + rules->rules[(rule_cnt + 3)].value); + memcpy((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + rof_rule_addr), &tmp_write_swap, + (sizeof(uint64_t) * 4)); + } else + return -1; + /* Fast forward as already handled block of 8. */ + rule_cnt += MLX5_RXP_INST_BLOCK_SIZE; + } else + rule_cnt++; /* Must be something other than EM rule. */ + } + return 0; +} + +static int +rxp_db_setup(struct mlx5_regex_priv *priv) +{ + int ret; + uint8_t i; + + /* Setup database memories for both RXP engines + reprogram memory. */ + for (i = 0; i < (priv->nb_engines + MLX5_RXP_EM_COUNT); i++) { + priv->db[i].ptr = rte_malloc("", MLX5_MAX_DB_SIZE, 0); + if (!priv->db[i].ptr) { + DRV_LOG(ERR, "Failed to alloc db memory!"); + ret = ENODEV; + goto tidyup_error; + } + /* Register the memory. */ + priv->db[i].umem.umem = mlx5_glue->devx_umem_reg(priv->ctx, + priv->db[i].ptr, + MLX5_MAX_DB_SIZE, 7); + if (!priv->db[i].umem.umem) { + DRV_LOG(ERR, "Failed to register memory!"); + ret = ENODEV; + goto tidyup_error; + } + /* Ensure set all DB memory to 0's before setting up DB. */ + memset(priv->db[i].ptr, 0x00, MLX5_MAX_DB_SIZE); + /* No data currently in database. */ + priv->db[i].len = 0; + priv->db[i].active = false; + priv->db[i].db_assigned_to_eng_num = MLX5_RXP_DB_NOT_ASSIGNED; + } + return 0; +tidyup_error: + for (i = 0; i < (priv->nb_engines + MLX5_RXP_EM_COUNT); i++) { + if (priv->db[i].ptr) + rte_free(priv->db[i].ptr); + if (priv->db[i].umem.umem) + mlx5_glue->devx_umem_dereg(priv->db[i].umem.umem); + } + return -ret; +} + +int +mlx5_regex_rules_db_import(struct rte_regexdev *dev, + const char *rule_db, uint32_t rule_db_len) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_rxp_ctl_rules_pgm *rules = NULL; + uint8_t id; + int ret; + + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) { + DRV_LOG(ERR, "RXP programming mode not set!"); + return -1; + } + if (rule_db == NULL) { + DRV_LOG(ERR, "Database empty!"); + return -ENODEV; + } + if (rule_db_len == 0) + return -EINVAL; + ret = rxp_parse_rof(rule_db, rule_db_len, &rules); + if (ret) { + DRV_LOG(ERR, "Can't parse ROF file."); + return ret; + } + /* Need to ensure RXP not busy before stop! */ + for (id = 0; id < priv->nb_engines; id++) { + ret = rxp_stop_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "Can't stop engine."); + ret = -ENODEV; + goto tidyup_error; + } + ret = program_rxp_rules(priv, rules, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to program rxp rules."); + ret = -ENODEV; + goto tidyup_error; + } + ret = rxp_start_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "Can't start engine."); + ret = -ENODEV; + goto tidyup_error; + } + } + rte_free(rules); + return 0; +tidyup_error: + rte_free(rules); + return ret; +} + int mlx5_regex_configure(struct rte_regexdev *dev, const struct rte_regexdev_config *cfg) { struct mlx5_regex_priv *priv = dev->data->dev_private; int ret; - uint8_t id; + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) + return -1; priv->nb_queues = cfg->nb_queue_pairs; + dev->data->dev_conf.nb_queue_pairs = priv->nb_queues; priv->qps = rte_zmalloc(NULL, sizeof(struct mlx5_regex_qp) * priv->nb_queues, 0); if (!priv->nb_queues) { @@ -225,35 +984,22 @@ return -rte_errno; } priv->nb_max_matches = cfg->nb_max_matches; - for (id = 0; id < 2; id++) { - ret = rxp_stop_engine(priv->ctx, id); - if (ret) { - DRV_LOG(ERR, "can't stop engine."); - rte_errno = ENODEV; - return -rte_errno; - } - ret = rxp_init(priv, id); - if (ret) { - DRV_LOG(ERR, "can't init engine."); - rte_errno = ENODEV; - return -rte_errno; - } - ret = mlx5_devx_regex_register_write(priv->ctx, id, - MLX5_RXP_CSR_MAX_MATCH, - priv->nb_max_matches); - if (ret) { - DRV_LOG(ERR, "can't update number of matches."); - rte_errno = ENODEV; - goto configure_error; - } - ret = rxp_start_engine(priv->ctx, id); - if (ret) { - DRV_LOG(ERR, "can't start engine."); + /* Setup rxp db memories. */ + if (rxp_db_setup(priv)) { + DRV_LOG(ERR, "Failed to setup RXP db memory"); + rte_errno = ENOMEM; + return -rte_errno; + } + if (cfg->rule_db != NULL) { + ret = mlx5_regex_rules_db_import(dev, cfg->rule_db, + cfg->rule_db_len); + if (ret < 0) { + DRV_LOG(ERR, "Failed to program rxp rules."); rte_errno = ENODEV; goto configure_error; } - - } + } else + DRV_LOG(DEBUG, "Regex config without rules programming!"); return 0; configure_error: if (priv->qps) diff --git a/drivers/regex/mlx5/mlx5_rxp.h b/drivers/regex/mlx5/mlx5_rxp.h new file mode 100644 index 0000000..9686e24 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_rxp.h @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#ifndef RTE_PMD_MLX5_REGEX_RXP_H_ +#define RTE_PMD_MLX5_REGEX_RXP_H_ + +#define MLX5_RXP_MAX_JOB_LENGTH 16384 +#define MLX5_RXP_MAX_SUBSETS 4095 +#define MLX5_RXP_CSR_NUM_ENTRIES 31 + +#define MLX5_RXP_CTRL_TYPE_MASK 7 +#define MLX5_RXP_CTRL_TYPE_JOB_DESCRIPTOR 0 +#define MLX5_RXP_CTRL_TYPE_RESPONSE_DESCRIPTOR 1 +#define MLX5_RXP_CTRL_TYPE_MEMORY_WRITE 4 +#define MLX5_RXP_CSR_CTRL_DISABLE_L2C (1 << 7) + +#define MLX5_RXP_CTRL_JOB_DESC_SOF 0x0010 +#define MLX5_RXP_CTRL_JOB_DESC_EOF 0x0020 +#define MLX5_RXP_CTRL_JOB_DESC_HPM_ENABLE 0x0100 +#define MLX5_RXP_CTRL_JOB_DESC_ANYMATCH_ENABLE 0x0200 +#define MLX5_RXP_CTRL_JOB_DESC_FLAGS (MLX5_RXP_CTRL_JOB_DESC_SOF | \ + MLX5_RXP_CTRL_JOB_DESC_EOF | \ + MLX5_RXP_CTRL_JOB_DESC_HPM_ENABLE | \ + MLX5_RXP_CTRL_JOB_DESC_ANYMATCH_ENABLE) + +#define MLX5_RXP_CTRL_VALID 0x8000 + +#define MLX5_RXP_RESP_STATUS_MAX_PRI_THREADS (1 << 3) +#define MLX5_RXP_RESP_STATUS_MAX_SEC_THREADS (1 << 4) +#define MLX5_RXP_RESP_STATUS_MAX_LATENCY (1 << 5) +#define MLX5_RXP_RESP_STATUS_MAX_MATCH (1 << 6) +#define MLX5_RXP_RESP_STATUS_MAX_PREFIX (1 << 7) +#define MLX5_RXP_RESP_STATUS_HPM (1 << 8) +#define MLX5_RXP_RESP_STATUS_ANYMATCH (1 << 9) +#define MLX5_RXP_RESP_STATUS_PMI_SOJ (1 << 13) +#define MLX5_RXP_RESP_STATUS_PMI_EOJ (1 << 14) + +/* This describes the header the RXP expects for any search data. */ +struct mlx5_rxp_job_desc { + uint32_t job_id; + uint16_t ctrl; + uint16_t len; + uint16_t subset[4]; +} __rte_packed; + +struct mlx5_rxp_response_desc { + uint32_t job_id; + uint16_t status; + uint8_t detected_match_count; + uint8_t match_count; + uint16_t primary_thread_count; + uint16_t instruction_count; + uint16_t latency_count; + uint16_t pmi_min_byte_ptr; +} __rte_packed; + +struct mlx5_rxp_match_tuple { + uint32_t rule_id; + uint16_t start_ptr; + uint16_t length; +} __rte_packed; + +struct mlx5_rxp_response { + struct mlx5_rxp_response_desc header; + struct mlx5_rxp_match_tuple matches[0]; +}; + +#define MLX5_RXP_MAX_MATCHES 254 + +#define MLX5_RXP_CTL_RULES_PGM 1 +#define MLX5_RXP_CTL_RULES_PGM_INCR 2 + +#define MLX5_RXP_ROF_ENTRY_INST 0 +#define MLX5_RXP_ROF_ENTRY_EQ 1 +#define MLX5_RXP_ROF_ENTRY_GTE 2 +#define MLX5_RXP_ROF_ENTRY_LTE 3 +#define MLX5_RXP_ROF_ENTRY_CHECKSUM 4 +#define MLX5_RXP_ROF_ENTRY_CHECKSUM_EX_EM 5 +#define MLX5_RXP_ROF_ENTRY_IM 6 +#define MLX5_RXP_ROF_ENTRY_EM 7 +#define MLX5_RXP_ROF_ENTRY_TYPE_MAX 7 + +#define MLX5_RXP_INST_OFFSET 3 +#define MLX5_RXP_INST_BLOCK_SIZE 8 +#define MLX5_MAX_SIZE_RES_DES (sizeof(struct mlx5_rxp_response_desc)) +#define MLX5_MAX_DB_SIZE (1u << 27u) +#define MLX5_MAX_SIZE_MATCH_RESP (254 * sizeof(struct mlx5_rxp_match_tuple)) +#define MLX5_RXP_SQ_NOT_BUSY false +#define MLX5_RXP_SQ_BUSY true + + +struct mlx5_rxp_ctl_hdr { + uint16_t cmd; + uint32_t len; +}; + +struct mlx5_rxp_rof_entry { + uint8_t type; + uint32_t addr; + uint64_t value; +}; + +struct mlx5_rxp_rof { + uint32_t rof_version; + char *timestamp; + char *rxp_compiler_version; + uint32_t rof_revision; + uint32_t number_of_entries; + struct mlx5_rxp_rof_entry *rof_entries; +}; + +struct mlx5_rxp_ctl_rules_pgm { + struct mlx5_rxp_ctl_hdr hdr; + uint32_t count; + struct mlx5_rxp_rof_entry rules[0]; +} __rte_packed; + +/* RXP programming mode setting. */ +enum mlx5_rxp_program_mode { + MLX5_RXP_MODE_NOT_DEFINED = 0, + MLX5_RXP_SHARED_PROG_MODE, + MLX5_RXP_PRIVATE_PROG_MODE, +}; + +#define MLX5_RXP_POLL_CSR_FOR_VALUE_TIMEOUT 3000 /* Poll timeout in ms. */ +#define MLX5_RXP_INITIALIZATION_TIMEOUT 60000 /* Initialize timeout in ms. */ +#define MLX5_RXP_MAX_ENGINES 2u /* Number of RXP engines. */ +#define MLX5_RXP_EM_COUNT 1u /* Extra External Memories to use. */ +#define MLX5_RXP_DB_NOT_ASSIGNED 0xFF + +struct mlx5_regex_umem { + struct mlx5dv_devx_umem *umem; + uint32_t id; + uint64_t offset; +}; + +#endif /* RTE_PMD_MLX5_REGEX_RXP_H_ */ From patchwork Mon Jul 20 06:26:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74468 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7485DA0540; Mon, 20 Jul 2020 08:28:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 96E361BF60; Mon, 20 Jul 2020 08:26:56 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 55D232BE5 for ; Mon, 20 Jul 2020 08:26:53 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:26:50 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKuk008177; Mon, 20 Jul 2020 09:26:49 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Mon, 20 Jul 2020 06:26:12 +0000 Message-Id: <1595226378-81144-9-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 08/13] regex/mlx5: add completion queue creation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the creation of CQ Signed-off-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/meson.build | 1 + drivers/regex/mlx5/mlx5_regex.c | 1 + drivers/regex/mlx5/mlx5_regex.h | 4 +- drivers/regex/mlx5/mlx5_regex_control.c | 195 ++++++++++++++++++++++++++++++++ drivers/regex/mlx5/mlx5_rxp.c | 1 + 6 files changed, 201 insertions(+), 2 deletions(-) create mode 100644 drivers/regex/mlx5/mlx5_regex_control.c diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 01853246..8918ebe 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -10,6 +10,7 @@ LIB = librte_pmd_mlx5_regex.a SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_rxp.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_devx.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_control.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index 9354145..4495fa7 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -13,6 +13,7 @@ sources = files( 'mlx5_regex.c', 'mlx5_rxp.c', 'mlx5_regex_devx.c', + 'mlx5_regex_control.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 72da3e2..309fde8 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -27,6 +27,7 @@ .dev_info_get = mlx5_regex_info_get, .dev_configure = mlx5_regex_configure, .dev_db_import = mlx5_regex_rules_db_import, + .dev_qp_setup = mlx5_regex_qp_setup, }; static struct ibv_device * diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index a2d2c9e..21770ca 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -19,7 +19,7 @@ struct mlx5_regex_sq { struct mlx5_devx_obj *obj; /* The SQ DevX object. */ int64_t dbr_offset; /* Door bell record offset. */ uint32_t dbr_umem; /* Door bell record umem id. */ - volatile struct mlx5_cqe *wqe; /* The SQ ring buffer. */ + uint8_t *wqe; /* The SQ ring buffer. */ struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ }; @@ -62,10 +62,10 @@ struct mlx5_regex_priv { struct mlx5_regex_db db[MLX5_RXP_MAX_ENGINES + MLX5_RXP_EM_COUNT]; uint32_t nb_engines; /* Number of RegEx engines. */ - struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ uint32_t eqn; /* EQ number. */ struct mlx5dv_devx_uar *uar; /* UAR object. */ struct ibv_pd *pd; + struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ }; /* mlx5_rxp.c */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c new file mode 100644 index 0000000..577965f --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -0,0 +1,195 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "mlx5_regex.h" +#include "mlx5_regex_utils.h" +#include "mlx5_rxp_csrs.h" +#include "mlx5_rxp.h" + +#define MLX5_REGEX_NUM_WQE_PER_PAGE (4096/64) + +/** + * Returns the number of qp obj to be created. + * + * @param nb_desc + * The number of descriptors for the queue. + * + * @return + * The number of obj to be created. + */ +static uint16_t +regex_ctrl_get_nb_obj(uint16_t nb_desc) +{ + return ((nb_desc / MLX5_REGEX_NUM_WQE_PER_PAGE) + + !!(nb_desc % MLX5_REGEX_NUM_WQE_PER_PAGE)); +} + +/** + * destroy CQ. + * + * @param priv + * Pointer to the priv object. + * @param cp + * Pointer to the CQ to be destroyed. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_destroy_cq(struct mlx5_regex_priv *priv, struct mlx5_regex_cq *cq) +{ + if (cq->cqe_umem) { + mlx5_glue->devx_umem_dereg(cq->cqe_umem); + cq->cqe_umem = NULL; + } + if (cq->cqe) { + rte_free((void *)(uintptr_t)cq->cqe); + cq->cqe = NULL; + } + if (cq->dbr_offset) { + mlx5_release_dbr(&priv->dbrpgs, cq->dbr_umem, cq->dbr_offset); + cq->dbr_offset = -1; + } + if (cq->obj) { + mlx5_devx_cmd_destroy(cq->obj); + cq->obj = NULL; + } + return 0; +} + +/** + * create the CQ object. + * + * @param priv + * Pointer to the priv object. + * @param cp + * Pointer to the CQ to be created. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_create_cq(struct mlx5_regex_priv *priv, struct mlx5_regex_cq *cq) +{ + struct mlx5_devx_cq_attr attr = { + .q_umem_valid = 1, + .db_umem_valid = 1, + .eqn = priv->eqn, + }; + struct mlx5_devx_dbr_page *dbr_page = NULL; + void *buf = NULL; + size_t pgsize = sysconf(_SC_PAGESIZE); + uint32_t cq_size = 1 << cq->log_nb_desc; + uint32_t i; + + cq->dbr_offset = mlx5_get_dbr(priv->ctx, &priv->dbrpgs, &dbr_page); + if (cq->dbr_offset < 0) { + DRV_LOG(ERR, "Can't allocate cq door bell record."); + rte_errno = ENOMEM; + goto error; + } + cq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); + buf = rte_calloc(NULL, 1, sizeof(struct mlx5_cqe) * cq_size, 4096); + if (!buf) { + DRV_LOG(ERR, "Can't allocate cqe buffer."); + rte_errno = ENOMEM; + goto error; + } + cq->cqe = buf; + for (i = 0; i < cq_size; i++) + cq->cqe[i].op_own = 0xff; + cq->cqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, + sizeof(struct mlx5_cqe) * + cq_size, 7); + if (!cq->cqe_umem) { + DRV_LOG(ERR, "Can't register cqe mem."); + rte_errno = ENOMEM; + goto error; + } + attr.db_umem_offset = cq->dbr_offset; + attr.db_umem_id = cq->dbr_umem; + attr.q_umem_id = mlx5_os_get_umem_id(cq->cqe_umem); + attr.log_cq_size = cq->log_nb_desc; + attr.uar_page_id = priv->uar->page_id; + attr.log_page_size = rte_log2_u32(pgsize); + cq->obj = mlx5_devx_cmd_create_cq(priv->ctx, &attr); + if (!cq->obj) { + DRV_LOG(ERR, "Can't create cq object."); + rte_errno = ENOMEM; + goto error; + } + return 0; +error: + if (cq->cqe_umem) + mlx5_glue->devx_umem_dereg(cq->cqe_umem); + if (buf) + rte_free(buf); + if (cq->dbr_offset) + mlx5_release_dbr(&priv->dbrpgs, cq->dbr_umem, cq->dbr_offset); + return -rte_errno; +} + +/** + * Setup the qp. + * + * @param dev + * Pointer to RegEx dev structure. + * @param qp_ind + * The queue index to setup. + * @param cfg + * The queue requested configuration. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, + const struct rte_regexdev_qp_conf *cfg) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_regex_qp *qp; + int ret; + + qp = &priv->qps[qp_ind]; + qp->flags = cfg->qp_conf_flags; + qp->cq.log_nb_desc = rte_log2_u32(cfg->nb_desc); + qp->nb_desc = 1 << qp->cq.log_nb_desc; + if (qp->flags & RTE_REGEX_QUEUE_PAIR_CFG_OOS_F) + qp->nb_obj = regex_ctrl_get_nb_obj(qp->nb_desc); + else + qp->nb_obj = 1; + qp->sqs = rte_malloc(NULL, + qp->nb_obj * sizeof(struct mlx5_regex_sq), 64); + if (!qp->sqs) { + DRV_LOG(ERR, "Can't allocate sq array memory."); + rte_errno = ENOMEM; + return -rte_errno; + } + ret = regex_ctrl_create_cq(priv, &qp->cq); + if (ret) { + DRV_LOG(ERR, "Can't create cq."); + goto error; + } + return 0; + +error: + regex_ctrl_destroy_cq(priv, &qp->cq); + return -rte_errno; + +} diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c index 5dfba26..b8fab79 100644 --- a/drivers/regex/mlx5/mlx5_rxp.c +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -118,6 +118,7 @@ info->max_queue_pairs = 1; info->regexdev_capa = RTE_REGEXDEV_SUPP_PCRE_GREEDY_F; info->rule_flags = 0; + info->max_queue_pairs = 10; return 0; } From patchwork Mon Jul 20 06:26:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74469 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4E4F3A0540; Mon, 20 Jul 2020 08:28:16 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E366D1BF92; Mon, 20 Jul 2020 08:27:07 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 5E4CF1BF92 for ; Mon, 20 Jul 2020 08:26:58 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:26:53 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKul008177; Mon, 20 Jul 2020 09:26:53 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Mon, 20 Jul 2020 06:26:13 +0000 Message-Id: <1595226378-81144-10-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 09/13] regex/mlx5: add send queue support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit introduce the SQ creation. The SQ is used for enqueuing a job. In order to support out of order matches, we create number os SQ per one application QP. Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.h | 2 + drivers/regex/mlx5/mlx5_regex_control.c | 179 ++++++++++++++++++++++++++++++++ 2 files changed, 181 insertions(+) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 21770ca..bf285a1 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -21,6 +21,7 @@ struct mlx5_regex_sq { uint32_t dbr_umem; /* Door bell record umem id. */ uint8_t *wqe; /* The SQ ring buffer. */ struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ + uint32_t *dbr; }; struct mlx5_regex_cq { @@ -30,6 +31,7 @@ struct mlx5_regex_cq { uint32_t dbr_umem; /* Door bell record umem id. */ volatile struct mlx5_cqe *cqe; /* The CQ ring buffer. */ struct mlx5dv_devx_umem *cqe_umem; /* CQ buffer umem. */ + uint32_t *dbr; }; struct mlx5_regex_qp { diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index 577965f..83d390a 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -105,6 +105,9 @@ goto error; } cq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); + cq->dbr = (uint32_t *)((uintptr_t)dbr_page->dbrs + + (uintptr_t)cq->dbr_offset); + buf = rte_calloc(NULL, 1, sizeof(struct mlx5_cqe) * cq_size, 4096); if (!buf) { DRV_LOG(ERR, "Can't allocate cqe buffer."); @@ -145,6 +148,170 @@ return -rte_errno; } +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +static int +regex_get_pdn(void *pd, uint32_t *pdn) +{ + struct mlx5dv_obj obj; + struct mlx5dv_pd pd_info; + int ret = 0; + + obj.pd.in = pd; + obj.pd.out = &pd_info; + ret = mlx5_glue->dv_init_obj(&obj, MLX5DV_OBJ_PD); + if (ret) { + DRV_LOG(DEBUG, "Fail to get PD object info"); + return ret; + } + *pdn = pd_info.pdn; + return 0; +} +#endif + +/** + * create the SQ object. + * + * @param priv + * Pointer to the priv object. + * @param qp + * Pointer to the QP element + * @param q_ind + * The index of the queue. + * @param log_nb_desc + * Log 2 of the number of descriptors to be used. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_create_sq(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, + uint16_t q_ind, uint16_t log_nb_desc) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT + struct mlx5_devx_create_sq_attr attr = { 0 }; + struct mlx5_devx_modify_sq_attr modify_attr = { 0 }; + struct mlx5_devx_wq_attr *wq_attr = &attr.wq_attr; + struct mlx5_devx_dbr_page *dbr_page = NULL; + struct mlx5_regex_sq *sq = &qp->sqs[q_ind]; + void *buf = NULL; + uint32_t sq_size; + uint32_t pd_num = 0; + int ret; + + sq->log_nb_desc = log_nb_desc; + sq_size = 1 << sq->log_nb_desc; + sq->dbr_offset = mlx5_get_dbr(priv->ctx, &priv->dbrpgs, &dbr_page); + if (sq->dbr_offset < 0) { + DRV_LOG(ERR, "Can't allocate sq door bell record."); + rte_errno = ENOMEM; + goto error; + } + sq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); + sq->dbr = (uint32_t *)((uintptr_t)dbr_page->dbrs + + (uintptr_t)sq->dbr_offset); + + buf = rte_calloc(NULL, 1, 64 * sq_size, 4096); + if (!buf) { + DRV_LOG(ERR, "Can't allocate wqe buffer."); + rte_errno = ENOMEM; + goto error; + } + sq->wqe = buf; + sq->wqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, 64 * sq_size, + 7); + if (!sq->wqe_umem) { + DRV_LOG(ERR, "Can't register wqe mem."); + rte_errno = ENOMEM; + goto error; + } + attr.state = MLX5_SQC_STATE_RST; + attr.tis_lst_sz = 0; + attr.tis_num = 0; + attr.user_index = q_ind; + attr.cqn = qp->cq.obj->id; + wq_attr->uar_page = priv->uar->page_id; + regex_get_pdn(priv->pd, &pd_num); + wq_attr->pd = pd_num; + wq_attr->wq_type = MLX5_WQ_TYPE_CYCLIC; + wq_attr->dbr_umem_id = sq->dbr_umem; + wq_attr->dbr_addr = sq->dbr_offset; + wq_attr->dbr_umem_valid = 1; + wq_attr->wq_umem_id = mlx5_os_get_umem_id(sq->wqe_umem); + wq_attr->wq_umem_offset = 0; + wq_attr->wq_umem_valid = 1; + wq_attr->log_wq_stride = 6; + wq_attr->log_wq_sz = sq->log_nb_desc; + sq->obj = mlx5_devx_cmd_create_sq(priv->ctx, &attr); + if (!sq->obj) { + DRV_LOG(ERR, "Can't create sq object."); + rte_errno = ENOMEM; + goto error; + } + modify_attr.state = MLX5_SQC_STATE_RDY; + ret = mlx5_devx_cmd_modify_sq(sq->obj, &modify_attr); + if (ret) { + DRV_LOG(ERR, "Can't change sq state to ready."); + rte_errno = ENOMEM; + goto error; + } + + return 0; +error: + if (sq->wqe_umem) + mlx5_glue->devx_umem_dereg(sq->wqe_umem); + if (buf) + rte_free(buf); + if (sq->dbr_offset) + mlx5_release_dbr(&priv->dbrpgs, sq->dbr_umem, sq->dbr_offset); + return -rte_errno; +#else + (void)priv; + (void)qp; + (void)q_ind; + (void)log_nb_desc; + DRV_LOG(ERR, "Cannot get pdn - no DV support."); + return -ENOTSUP; +#endif +} + +/** + * Destroy the SQ object. + * + * @param priv + * Pointer to the priv object. + * @param qp + * Pointer to the QP element + * @param q_ind + * The index of the queue. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_destroy_sq(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, + uint16_t q_ind) +{ + struct mlx5_regex_sq *sq = &qp->sqs[q_ind]; + + if (sq->wqe_umem) { + mlx5_glue->devx_umem_dereg(sq->wqe_umem); + sq->wqe_umem = NULL; + } + if (sq->wqe) { + rte_free((void *)(uintptr_t)sq->wqe); + sq->wqe = NULL; + } + if (sq->dbr_offset) { + mlx5_release_dbr(&priv->dbrpgs, sq->dbr_umem, sq->dbr_offset); + sq->dbr_offset = -1; + } + if (sq->obj) { + mlx5_devx_cmd_destroy(sq->obj); + sq->obj = NULL; + } + return 0; +} + /** * Setup the qp. * @@ -164,7 +331,9 @@ { struct mlx5_regex_priv *priv = dev->data->dev_private; struct mlx5_regex_qp *qp; + int i; int ret; + uint16_t log_desc; qp = &priv->qps[qp_ind]; qp->flags = cfg->qp_conf_flags; @@ -181,15 +350,25 @@ rte_errno = ENOMEM; return -rte_errno; } + log_desc = rte_log2_u32(qp->nb_desc / qp->nb_obj); ret = regex_ctrl_create_cq(priv, &qp->cq); if (ret) { DRV_LOG(ERR, "Can't create cq."); goto error; } + for (i = 0; i < qp->nb_obj; i++) { + ret = regex_ctrl_create_sq(priv, qp, i, log_desc); + if (ret) { + DRV_LOG(ERR, "Can't create sq."); + goto error; + } + } return 0; error: regex_ctrl_destroy_cq(priv, &qp->cq); + for (i = 0; i < qp->nb_obj; i++) + ret = regex_ctrl_destroy_sq(priv, qp, i); return -rte_errno; } From patchwork Mon Jul 20 06:26:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74470 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3113AA0540; Mon, 20 Jul 2020 08:28:33 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DF5BB1BFA9; Mon, 20 Jul 2020 08:27:09 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 67D311BF93 for ; Mon, 20 Jul 2020 08:26:58 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:26:57 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKum008177; Mon, 20 Jul 2020 09:26:56 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Mon, 20 Jul 2020 06:26:14 +0000 Message-Id: <1595226378-81144-11-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 10/13] regex/mlx5: fastpath setup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Allocated and register input/output buffers and metadata. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/common/mlx5/mlx5_prm.h | 36 ++++++ drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/meson.build | 1 + drivers/regex/mlx5/mlx5_regex.h | 8 ++ drivers/regex/mlx5/mlx5_regex_control.c | 2 + drivers/regex/mlx5/mlx5_regex_fastpath.c | 200 +++++++++++++++++++++++++++++++ 6 files changed, 248 insertions(+) create mode 100644 drivers/regex/mlx5/mlx5_regex_fastpath.c diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index a59acc7..6a3f22e 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -405,6 +405,42 @@ struct mlx5_cqe_ts { uint8_t op_own; }; +/* MMO metadata segment */ + +#define MLX5_OPCODE_MMO 0x2f +#define MLX5_OPC_MOD_MMO_REGEX 0x4 + +struct mlx5_wqe_metadata_seg { + uint32_t mmo_control_31_0; /* mmo_control_63_32 is in ctrl_seg.imm */ + uint32_t lkey; + uint64_t addr; +}; + +struct mlx5_ifc_regexp_mmo_control_bits { + uint8_t reserved_at_31[0x2]; + uint8_t le[0x1]; + uint8_t reserved_at_28[0x1]; + uint8_t subset_id_0[0xc]; + uint8_t reserved_at_16[0x4]; + uint8_t subset_id_1[0xc]; + uint8_t ctrl[0x4]; + uint8_t subset_id_2[0xc]; + uint8_t reserved_at_16_1[0x4]; + uint8_t subset_id_3[0xc]; +}; + +struct mlx5_ifc_regexp_metadata_bits { + uint8_t rof_version[0x10]; + uint8_t latency_count[0x10]; + uint8_t instruction_count[0x10]; + uint8_t primary_thread_count[0x10]; + uint8_t match_count[0x8]; + uint8_t detected_match_count[0x8]; + uint8_t status[0x10]; + uint8_t job_id[0x20]; + uint8_t reserved[0x80]; +}; + /* Adding direct verbs to data-path. */ /* CQ sequence number mask. */ diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 8918ebe..9e11926 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -11,6 +11,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_rxp.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_devx.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_control.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_fastpath.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index 4495fa7..d0f8fcf 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -14,6 +14,7 @@ sources = files( 'mlx5_rxp.c', 'mlx5_regex_devx.c', 'mlx5_regex_control.c', + 'mlx5_regex_fastpath.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index bf285a1..21bb02b 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -40,6 +40,11 @@ struct mlx5_regex_qp { struct mlx5_regex_sq *sqs; /* Pointer to sq array. */ uint16_t nb_obj; /* Number of sq objects. */ struct mlx5_regex_cq cq; /* CQ struct. */ + uint32_t free_sqs; + struct mlx5_regex_job *jobs; + struct ibv_mr *metadata; + struct ibv_mr *inputs; + struct ibv_mr *outputs; }; struct mlx5_regex_db { @@ -91,4 +96,7 @@ int mlx5_devx_regex_database_program(void *ctx, uint8_t engine, /* mlx5_regex_control.c */ int mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, const struct rte_regexdev_qp_conf *cfg); + +/* mlx5_regex_fastpath.c */ +int mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index 83d390a..ec57e24 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -363,6 +363,8 @@ goto error; } } + + mlx5_regexdev_setup_fastpath(priv, qp_ind); return 0; error: diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c new file mode 100644 index 0000000..8d55bbc --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -0,0 +1,200 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ +#include +#include + +#include +#include +#include +#include +#include +#include +#include + + +#include +#include +#include +#include +#include + +#include "mlx5_regex_utils.h" +#include "mlx5_rxp.h" +#include "mlx5_regex.h" + +/* Verbs header. */ +/* ISO C doesn't support unnamed structs/unions, disabling -pedantic. */ +#include + +#define MAX_WQE_INDEX 0xffff +#define MLX5_REGEX_METADATA_SIZE 64 +#define MLX5_REGEX_MAX_INPUT (1 << 14) +#define MLX5_REGEX_MAX_OUTPUT (1 << 11) + +#define MLX5_REGEX_WQE_METADATA_OFFSET 16 +#define MLX5_REGEX_WQE_GATHER_OFFSET 32 +#define MLX5_REGEX_WQE_SCATTER_OFFSET 48 + +static inline uint32_t +sq_size_get(struct mlx5_regex_sq *sq) +{ + return (1U << sq->log_nb_desc); +} +static inline uint32_t +cq_size_get(struct mlx5_regex_cq *cq) +{ + return (1U << cq->log_nb_desc); +} + +struct mlx5_regex_job { + uint64_t user_id; + uint8_t *input; + volatile uint8_t *output; + volatile uint8_t *metadata; +} __rte_cached_aligned; + +static inline void +set_data_seg(struct mlx5_wqe_data_seg *seg, + uint32_t length, uint32_t lkey, + uintptr_t address) +{ + seg->byte_count = rte_cpu_to_be_32(length); + seg->lkey = rte_cpu_to_be_32(lkey); + seg->addr = rte_cpu_to_be_64(address); +} + +static inline void +set_metadata_seg(struct mlx5_wqe_metadata_seg *seg, + uint32_t mmo_control_31_0, uint32_t lkey, + uintptr_t address) +{ + seg->mmo_control_31_0 = htobe32(mmo_control_31_0); + seg->lkey = rte_cpu_to_be_32(lkey); + seg->addr = rte_cpu_to_be_64(address); +} + +static void +setup_sqs(struct mlx5_regex_qp *queue) +{ + size_t sqid, entry; + uint32_t job_id; + for (sqid = 0; sqid < queue->nb_obj; sqid++) { + struct mlx5_regex_sq *sq = &queue->sqs[sqid]; + uint8_t *wqe = (uint8_t *)sq->wqe; + for (entry = 0 ; entry < sq_size_get(sq); entry++) { + job_id = sqid * sq_size_get(sq) + entry; + struct mlx5_regex_job *job = &queue->jobs[job_id]; + + set_metadata_seg((struct mlx5_wqe_metadata_seg *) + (wqe + MLX5_REGEX_WQE_METADATA_OFFSET), + 0, queue->metadata->lkey, + (uintptr_t)job->metadata); + set_data_seg((struct mlx5_wqe_data_seg *) + (wqe + MLX5_REGEX_WQE_GATHER_OFFSET), + 0, queue->inputs->lkey, + (uintptr_t)job->input); + set_data_seg((struct mlx5_wqe_data_seg *) + (wqe + MLX5_REGEX_WQE_SCATTER_OFFSET), + MLX5_REGEX_MAX_OUTPUT, + queue->outputs->lkey, + (uintptr_t)job->output); + wqe += 64; + } + queue->free_sqs |= 1 << sqid; + } +} + +static int +setup_buffers(struct mlx5_regex_qp *qp, struct ibv_pd *pd) +{ + uint32_t i; + int err; + + void *ptr = rte_calloc(__func__, qp->nb_desc, + MLX5_REGEX_METADATA_SIZE, + MLX5_REGEX_METADATA_SIZE); + if (!ptr) + return -ENOMEM; + + qp->metadata = mlx5_glue->reg_mr(pd, ptr, + MLX5_REGEX_METADATA_SIZE*qp->nb_desc, + IBV_ACCESS_LOCAL_WRITE); + if (!qp->metadata) { + rte_free(ptr); + return -EINVAL; + } + ptr = rte_calloc(__func__, qp->nb_desc, + MLX5_REGEX_MAX_INPUT, + MLX5_REGEX_MAX_INPUT); + + if (!ptr) { + err = -ENOMEM; + goto err_input; + } + qp->inputs = mlx5_glue->reg_mr(pd, ptr, + MLX5_REGEX_MAX_INPUT*qp->nb_desc, + IBV_ACCESS_LOCAL_WRITE); + if (!qp->inputs) { + rte_free(ptr); + err = -EINVAL; + goto err_input; + } + + ptr = rte_calloc(__func__, qp->nb_desc, + MLX5_REGEX_MAX_OUTPUT, + MLX5_REGEX_MAX_OUTPUT); + if (!ptr) { + err = -ENOMEM; + goto err_output; + } + qp->outputs = mlx5_glue->reg_mr(pd, ptr, + MLX5_REGEX_MAX_OUTPUT*qp->nb_desc, + IBV_ACCESS_LOCAL_WRITE); + if (!qp->outputs) { + rte_free(ptr); + err = -EINVAL; + goto err_output; + } + + /* distribute buffers to jobs */ + for (i = 0; i < qp->nb_desc; i++) { + qp->jobs[i].input = + (uint8_t *)qp->inputs->addr + + (i % qp->nb_desc) * MLX5_REGEX_MAX_INPUT; + qp->jobs[i].output = + (uint8_t *)qp->outputs->addr + + (i % qp->nb_desc) * MLX5_REGEX_MAX_OUTPUT; + qp->jobs[i].metadata = + (uint8_t *)qp->metadata->addr + + (i % qp->nb_desc) * MLX5_REGEX_METADATA_SIZE; + } + return 0; + +err_output: + ptr = qp->inputs->addr; + rte_free(ptr); + mlx5_glue->dereg_mr(qp->inputs); +err_input: + ptr = qp->metadata->addr; + rte_free(ptr); + mlx5_glue->dereg_mr(qp->metadata); + return err; +} + +int +mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id) +{ + struct mlx5_regex_qp *qp = &priv->qps[qp_id]; + int err; + + qp->jobs = rte_calloc(__func__, qp->nb_desc, sizeof(*qp->jobs), + sizeof(*qp->jobs)); + if (!qp->jobs) + return -ENOMEM; + err = setup_buffers(qp, priv->pd); + if (err) + return err; + setup_sqs(qp); + return 0; +} From patchwork Mon Jul 20 06:26:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74471 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 20B32A0540; Mon, 20 Jul 2020 08:28:43 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 42C9F1BFB2; Mon, 20 Jul 2020 08:27:11 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 6EC3B1BED3 for ; Mon, 20 Jul 2020 08:27:02 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:27:00 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKun008177; Mon, 20 Jul 2020 09:27:00 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Mon, 20 Jul 2020 06:26:15 +0000 Message-Id: <1595226378-81144-12-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 11/13] regex/mlx5: add enqueue implementation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Will look for a free SQ to send the job on. doorbell will be given when sq is full, or no more jobs on the burst. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 1 + drivers/regex/mlx5/mlx5_regex.h | 6 ++ drivers/regex/mlx5/mlx5_regex_control.c | 2 + drivers/regex/mlx5/mlx5_regex_fastpath.c | 122 ++++++++++++++++++++++++++++++- 4 files changed, 129 insertions(+), 2 deletions(-) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 309fde8..607ca4d 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -167,6 +167,7 @@ goto error; } priv->regexdev->dev_ops = &mlx5_regexdev_ops; + priv->regexdev->enqueue = mlx5_regexdev_enqueue; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; priv->regexdev->state = RTE_REGEXDEV_READY; diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 21bb02b..fb81cb8 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -21,6 +21,9 @@ struct mlx5_regex_sq { uint32_t dbr_umem; /* Door bell record umem id. */ uint8_t *wqe; /* The SQ ring buffer. */ struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ + size_t pi, db_pi; + size_t ci; + uint32_t sqn; uint32_t *dbr; }; @@ -45,6 +48,7 @@ struct mlx5_regex_qp { struct ibv_mr *metadata; struct ibv_mr *inputs; struct ibv_mr *outputs; + size_t ci, pi; }; struct mlx5_regex_db { @@ -99,4 +103,6 @@ int mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, /* mlx5_regex_fastpath.c */ int mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id); +uint16_t mlx5_regexdev_enqueue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index ec57e24..f144c69 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -219,6 +219,8 @@ sq->wqe = buf; sq->wqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, 64 * sq_size, 7); + sq->ci = 0; + sq->pi = 0; if (!sq->wqe_umem) { DRV_LOG(ERR, "Can't register wqe mem."); rte_errno = ENOMEM; diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c index 8d55bbc..ff6386b 100644 --- a/drivers/regex/mlx5/mlx5_regex_fastpath.c +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -27,10 +27,14 @@ /* ISO C doesn't support unnamed structs/unions, disabling -pedantic. */ #include -#define MAX_WQE_INDEX 0xffff +#define MLX5_REGEX_MAX_WQE_INDEX 0xffff #define MLX5_REGEX_METADATA_SIZE 64 #define MLX5_REGEX_MAX_INPUT (1 << 14) #define MLX5_REGEX_MAX_OUTPUT (1 << 11) +#define MLX5_REGEX_WQE_CTRL_OFFSET 12 +#define MLX5_REGEX_WQE_METADATA_OFFSET 16 +#define MLX5_REGEX_WQE_GATHER_OFFSET 32 +#define MLX5_REGEX_WQE_SCATTER_OFFSET 48 #define MLX5_REGEX_WQE_METADATA_OFFSET 16 #define MLX5_REGEX_WQE_GATHER_OFFSET 32 @@ -74,6 +78,120 @@ struct mlx5_regex_job { seg->addr = rte_cpu_to_be_64(address); } +static inline void +set_regex_ctrl_seg(void *seg, uint8_t le, uint16_t subset_id0, + uint16_t subset_id1, uint16_t subset_id2, + uint16_t subset_id3, uint8_t ctrl) +{ + MLX5_SET(regexp_mmo_control, seg, le, le); + MLX5_SET(regexp_mmo_control, seg, ctrl, ctrl); + MLX5_SET(regexp_mmo_control, seg, subset_id_0, subset_id0); + MLX5_SET(regexp_mmo_control, seg, subset_id_1, subset_id1); + MLX5_SET(regexp_mmo_control, seg, subset_id_2, subset_id2); + MLX5_SET(regexp_mmo_control, seg, subset_id_3, subset_id3); +} + +static inline void +set_wqe_ctrl_seg(struct mlx5_wqe_ctrl_seg *seg, uint16_t pi, uint8_t opcode, + uint8_t opmod, uint32_t qp_num, uint8_t fm_ce_se, uint8_t ds, + uint8_t signature, uint32_t imm) +{ + seg->opmod_idx_opcode = rte_cpu_to_be_32(((uint32_t)opmod << 24) | + ((uint32_t)pi << 8) | + opcode); + seg->qpn_ds = rte_cpu_to_be_32((qp_num << 8) | ds); + seg->fm_ce_se = fm_ce_se; + seg->signature = signature; + seg->imm = imm; +} + +static inline void +prep_one(struct mlx5_regex_sq *sq, struct rte_regex_ops *op, + struct mlx5_regex_job *job) +{ + size_t wqe_offset = (sq->pi & (sq_size_get(sq) - 1)) * MLX5_SEND_WQE_BB; + uint8_t *wqe = (uint8_t *)sq->wqe + wqe_offset; + int ds = 4; /* ctrl + meta + input + output */ + + memcpy(job->input, + rte_pktmbuf_mtod(op->mbuf, void *), + rte_pktmbuf_data_len(op->mbuf)); + set_wqe_ctrl_seg((struct mlx5_wqe_ctrl_seg *)wqe, sq->pi, + MLX5_OPCODE_MMO, MLX5_OPC_MOD_MMO_REGEX, sq->obj->id, + 0, ds, 0, 0); + set_regex_ctrl_seg(wqe + 12, 0, op->group_id0, op->group_id1, + op->group_id2, + op->group_id3, 0); + struct mlx5_wqe_data_seg *input_seg = + (struct mlx5_wqe_data_seg *)(wqe + + MLX5_REGEX_WQE_GATHER_OFFSET); + input_seg->byte_count = + rte_cpu_to_be_32(rte_pktmbuf_data_len(op->mbuf)); + job->user_id = op->user_id; + sq->db_pi = sq->pi; + sq->pi = (sq->pi + 1) & MLX5_REGEX_MAX_WQE_INDEX; +} + +static inline void +send_doorbell(struct mlx5dv_devx_uar *uar, struct mlx5_regex_sq *sq) +{ + size_t wqe_offset = (sq->db_pi & (sq_size_get(sq) - 1)) * + MLX5_SEND_WQE_BB; + uint8_t *wqe = (uint8_t *)sq->wqe + wqe_offset; + ((struct mlx5_wqe_ctrl_seg *)wqe)->fm_ce_se = MLX5_WQE_CTRL_CQ_UPDATE; + uint64_t *doorbell_addr = + (uint64_t *)((uint8_t *)uar->base_addr + 0x800); + rte_cio_wmb(); + sq->dbr[MLX5_SND_DBR] = rte_cpu_to_be_32((sq->db_pi + 1) & + MLX5_REGEX_MAX_WQE_INDEX); + rte_wmb(); + *doorbell_addr = *(volatile uint64_t *)wqe; + rte_wmb(); +} + +static inline int +can_send(struct mlx5_regex_sq *sq) { + return unlikely(sq->ci > sq->pi) ? + MLX5_REGEX_MAX_WQE_INDEX + sq->pi - sq->ci < + sq_size_get(sq) : + sq->pi - sq->ci < sq_size_get(sq); +} + +static inline uint32_t +job_id_get(uint32_t qid, size_t sq_size, size_t index) { + return qid*sq_size + index % sq_size; +} + +uint16_t +mlx5_regexdev_enqueue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_regex_qp *queue = &priv->qps[qp_id]; + struct mlx5_regex_sq *sq; + size_t sqid, job_id, i = 0; + + while ((sqid = ffs(queue->free_sqs))) { + sqid--; /* ffs returns 1 for bit 0 */ + sq = &queue->sqs[sqid]; + while (can_send(sq)) { + job_id = job_id_get(sqid, sq_size_get(sq), sq->pi); + prep_one(sq, ops[i], &queue->jobs[job_id]); + i++; + if (unlikely(i == nb_ops)) { + send_doorbell(priv->uar, sq); + goto out; + } + } + queue->free_sqs &= ~(1 << sqid); + send_doorbell(priv->uar, sq); + } + +out: + queue->pi += i; + return i; +} + static void setup_sqs(struct mlx5_regex_qp *queue) { @@ -149,7 +267,7 @@ struct mlx5_regex_job { goto err_output; } qp->outputs = mlx5_glue->reg_mr(pd, ptr, - MLX5_REGEX_MAX_OUTPUT*qp->nb_desc, + MLX5_REGEX_MAX_OUTPUT * qp->nb_desc, IBV_ACCESS_LOCAL_WRITE); if (!qp->outputs) { rte_free(ptr); From patchwork Mon Jul 20 06:26:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74472 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 02010A0540; Mon, 20 Jul 2020 08:29:01 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2F9252B9C; Mon, 20 Jul 2020 08:27:23 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 7A5582BE5 for ; Mon, 20 Jul 2020 08:27:07 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:27:04 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKuo008177; Mon, 20 Jul 2020 09:27:04 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Mon, 20 Jul 2020 06:26:16 +0000 Message-Id: <1595226378-81144-13-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 12/13] regex/mlx5: implement dequeue function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Implement dequeue function for the regex API. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/common/mlx5/mlx5_prm.h | 10 +++ drivers/regex/mlx5/mlx5_regex.c | 1 + drivers/regex/mlx5/mlx5_regex.h | 4 ++ drivers/regex/mlx5/mlx5_regex_control.c | 1 + drivers/regex/mlx5/mlx5_regex_fastpath.c | 108 ++++++++++++++++++++++++++++++- 5 files changed, 123 insertions(+), 1 deletion(-) diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 6a3f22e..8fab962 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -441,6 +441,12 @@ struct mlx5_ifc_regexp_metadata_bits { uint8_t reserved[0x80]; }; +struct mlx5_ifc_regexp_match_tuple_bits { + uint8_t length[0x10]; + uint8_t start_ptr[0x10]; + uint8_t rule_id[0x20]; +}; + /* Adding direct verbs to data-path. */ /* CQ sequence number mask. */ @@ -637,6 +643,10 @@ struct mlx5_modification_cmd { __mlx5_16_bit_off(typ, fld))); \ } while (0) +#define MLX5_GET_VOLATILE(typ, p, fld) \ + ((rte_be_to_cpu_32(*((volatile __be32 *)(p) +\ + __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ + __mlx5_mask(typ, fld)) #define MLX5_GET(typ, p, fld) \ ((rte_be_to_cpu_32(*((__be32 *)(p) +\ __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 607ca4d..fd457bf 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -168,6 +168,7 @@ } priv->regexdev->dev_ops = &mlx5_regexdev_ops; priv->regexdev->enqueue = mlx5_regexdev_enqueue; + priv->regexdev->dequeue = mlx5_regexdev_dequeue; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; priv->regexdev->state = RTE_REGEXDEV_READY; diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index fb81cb8..217a1a4 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -34,6 +34,7 @@ struct mlx5_regex_cq { uint32_t dbr_umem; /* Door bell record umem id. */ volatile struct mlx5_cqe *cqe; /* The CQ ring buffer. */ struct mlx5dv_devx_umem *cqe_umem; /* CQ buffer umem. */ + size_t ci; uint32_t *dbr; }; @@ -105,4 +106,7 @@ int mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, int mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id); uint16_t mlx5_regexdev_enqueue(struct rte_regexdev *dev, uint16_t qp_id, struct rte_regex_ops **ops, uint16_t nb_ops); +uint16_t mlx5_regexdev_dequeue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops); + #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index f144c69..faafb76 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -120,6 +120,7 @@ cq->cqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, sizeof(struct mlx5_cqe) * cq_size, 7); + cq->ci = 0; if (!cq->cqe_umem) { DRV_LOG(ERR, "Can't register cqe mem."); rte_errno = ENOMEM; diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c index ff6386b..fc5a7fc 100644 --- a/drivers/regex/mlx5/mlx5_regex_fastpath.c +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -35,6 +35,7 @@ #define MLX5_REGEX_WQE_METADATA_OFFSET 16 #define MLX5_REGEX_WQE_GATHER_OFFSET 32 #define MLX5_REGEX_WQE_SCATTER_OFFSET 48 +#define MLX5_REGEX_METADATA_OFF 32 #define MLX5_REGEX_WQE_METADATA_OFFSET 16 #define MLX5_REGEX_WQE_GATHER_OFFSET 32 @@ -159,7 +160,7 @@ struct mlx5_regex_job { static inline uint32_t job_id_get(uint32_t qid, size_t sq_size, size_t index) { - return qid*sq_size + index % sq_size; + return qid * sq_size + (index & (sq_size - 1)); } uint16_t @@ -192,6 +193,111 @@ struct mlx5_regex_job { return i; } +#define MLX5_REGEX_RESP_SZ 8 + +static inline void +extract_result(struct rte_regex_ops *op, struct mlx5_regex_job *job) +{ + size_t j, offset; + op->user_id = job->user_id; + op->nb_matches = MLX5_GET_VOLATILE(regexp_metadata, job->metadata + + MLX5_REGEX_METADATA_OFF, + match_count); + op->nb_actual_matches = MLX5_GET_VOLATILE(regexp_metadata, + job->metadata + + MLX5_REGEX_METADATA_OFF, + detected_match_count); + for (j = 0; j < op->nb_matches; j++) { + offset = MLX5_REGEX_RESP_SZ * j; + op->matches[j].rule_id = + MLX5_GET_VOLATILE(regexp_match_tuple, + (job->output + offset), rule_id); + op->matches[j].start_offset = + MLX5_GET_VOLATILE(regexp_match_tuple, + (job->output + offset), start_ptr); + op->matches[j].len = + MLX5_GET_VOLATILE(regexp_match_tuple, + (job->output + offset), length); + } +} + +static inline volatile struct mlx5_cqe * +poll_one(struct mlx5_regex_cq *cq) +{ + volatile struct mlx5_cqe *cqe; + size_t next_cqe_offset; + + next_cqe_offset = (cq->ci & (cq_size_get(cq) - 1)); + cqe = (volatile struct mlx5_cqe *)(cq->cqe + next_cqe_offset); + rte_cio_wmb(); + + int ret = check_cqe(cqe, cq_size_get(cq), cq->ci); + + if (unlikely(ret == MLX5_CQE_STATUS_ERR)) { + DRV_LOG(ERR, "Completion with error on qp 0x%x", 0); + return NULL; + } + + if (unlikely(ret != MLX5_CQE_STATUS_SW_OWN)) + return NULL; + + return cqe; +} + + +/** + * DPDK callback for dequeue. + * + * @param dev + * Pointer to the regex dev structure. + * @param qp_id + * The queue to enqueue the traffic to. + * @param ops + * List of regex ops to dequeue. + * @param nb_ops + * Number of ops in ops parameter. + * + * @return + * Number of packets successfully dequeued (<= pkts_n). + */ +uint16_t +mlx5_regexdev_dequeue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_regex_qp *queue = &priv->qps[qp_id]; + struct mlx5_regex_cq *cq = &queue->cq; + volatile struct mlx5_cqe *cqe; + size_t i = 0; + + while ((cqe = poll_one(cq))) { + uint16_t wq_counter + = (rte_be_to_cpu_16(cqe->wqe_counter) + 1) & + MLX5_REGEX_MAX_WQE_INDEX; + size_t sqid = cqe->rsvd3[2]; + struct mlx5_regex_sq *sq = &queue->sqs[sqid]; + while (sq->ci != wq_counter) { + if (unlikely(i == nb_ops)) { + /* Return without updating cq->ci */ + goto out; + } + uint32_t job_id = job_id_get(sqid, sq_size_get(sq), + sq->ci); + extract_result(ops[i], &queue->jobs[job_id]); + sq->ci = (sq->ci + 1) & MLX5_REGEX_MAX_WQE_INDEX; + i++; + } + cq->ci = (cq->ci + 1) & 0xffffff; + rte_wmb(); + cq->dbr[0] = rte_cpu_to_be_32(cq->ci); + queue->free_sqs |= (1 << sqid); + } + +out: + queue->ci += i; + return i; +} + static void setup_sqs(struct mlx5_regex_qp *queue) { From patchwork Mon Jul 20 06:26:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 74473 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 680F4A0540; Mon, 20 Jul 2020 08:29:12 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 71EB91BFA7; Mon, 20 Jul 2020 08:27:24 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 7D6231BFA3 for ; Mon, 20 Jul 2020 08:27:12 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 20 Jul 2020 09:27:08 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06K6QKup008177; Mon, 20 Jul 2020 09:27:08 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Mon, 20 Jul 2020 06:26:17 +0000 Message-Id: <1595226378-81144-14-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595226378-81144-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1595226378-81144-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v6 13/13] regex/mlx5: add start stop functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the start, stop and close functions. In current implementation they are empty functions and are only exists in order that when called from rte level, the function will return with success code. Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 21 +++++++++++++++++++++ drivers/regex/mlx5/mlx5_regex.h | 6 ++++++ 2 files changed, 27 insertions(+) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index fd457bf..d4b3219 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -28,8 +28,29 @@ .dev_configure = mlx5_regex_configure, .dev_db_import = mlx5_regex_rules_db_import, .dev_qp_setup = mlx5_regex_qp_setup, + .dev_start = mlx5_regex_start, + .dev_stop = mlx5_regex_stop, + .dev_close = mlx5_regex_close, }; +int +mlx5_regex_start(struct rte_regexdev *dev __rte_unused) +{ + return 0; +} + +int +mlx5_regex_stop(struct rte_regexdev *dev __rte_unused) +{ + return 0; +} + +int +mlx5_regex_close(struct rte_regexdev *dev __rte_unused) +{ + return 0; +} + static struct ibv_device * mlx5_regex_get_ib_device_match(struct rte_pci_addr *addr) { diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 217a1a4..0c6359a 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -80,6 +80,12 @@ struct mlx5_regex_priv { struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ }; +/* mlx5_regex.c */ + +int mlx5_regex_start(struct rte_regexdev *dev); +int mlx5_regex_stop(struct rte_regexdev *dev); +int mlx5_regex_close(struct rte_regexdev *dev); + /* mlx5_rxp.c */ int mlx5_regex_info_get(struct rte_regexdev *dev, struct rte_regexdev_info *info);