From patchwork Sun Jul 12 20:58:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73840 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E9AFA052A; Sun, 12 Jul 2020 22:59:39 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 18BFD1D516; Sun, 12 Jul 2020 22:59:29 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 36EDC1D508 for ; Sun, 12 Jul 2020 22:59:26 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:20 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFFu025997; Sun, 12 Jul 2020 23:59:20 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Thomas Monjalon , John McNamara , Marko Kovacevic , Shahaf Shuler , Ray Kinsella , Neil Horman Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 12 Jul 2020 20:58:42 +0000 Message-Id: <1594587541-110442-2-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 01/20] regex/mlx5: add RegEx PMD layer and mlx5 driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery This commit introduce the RegEx poll mode drivers class, and adds Mellanox RegEx PMD. Signed-off-by: Yuval Avnery Signed-off-by: Ori Kam --- v2: * Add documantion. --- MAINTAINERS | 12 +++ config/common_base | 5 + doc/guides/index.rst | 1 + doc/guides/regexdevs/features/default.ini | 17 ++++ doc/guides/regexdevs/features/mlx5.ini | 10 ++ doc/guides/regexdevs/features_overview.rst | 118 ++++++++++++++++++++++ doc/guides/regexdevs/index.rst | 15 +++ doc/guides/regexdevs/mlx5.rst | 95 +++++++++++++++++ doc/guides/regexdevs/overview_feature_table.txt | 105 +++++++++++++++++++ doc/guides/rel_notes/release_20_08.rst | 5 + drivers/Makefile | 2 + drivers/common/Makefile | 2 +- drivers/common/mlx5/Makefile | 4 +- drivers/meson.build | 3 +- drivers/regex/Makefile | 8 ++ drivers/regex/meson.build | 9 ++ drivers/regex/mlx5/Makefile | 34 +++++++ drivers/regex/mlx5/meson.build | 32 ++++++ drivers/regex/mlx5/mlx5_regex.c | 5 + drivers/regex/mlx5/mlx5_regex.h | 8 ++ drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map | 3 + mk/rte.app.mk | 5 +- 22 files changed, 492 insertions(+), 6 deletions(-) create mode 100644 doc/guides/regexdevs/features/default.ini create mode 100644 doc/guides/regexdevs/features/mlx5.ini create mode 100644 doc/guides/regexdevs/features_overview.rst create mode 100644 doc/guides/regexdevs/index.rst create mode 100644 doc/guides/regexdevs/mlx5.rst create mode 100644 doc/guides/regexdevs/overview_feature_table.txt create mode 100644 drivers/regex/Makefile create mode 100644 drivers/regex/meson.build create mode 100644 drivers/regex/mlx5/Makefile create mode 100644 drivers/regex/mlx5/meson.build create mode 100644 drivers/regex/mlx5/mlx5_regex.c create mode 100644 drivers/regex/mlx5/mlx5_regex.h create mode 100644 drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map diff --git a/MAINTAINERS b/MAINTAINERS index 3cd402b..272359f 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -453,7 +453,9 @@ F: doc/guides/compressdevs/features/default.ini RegEx API - EXPERIMENTAL M: Ori Kam F: lib/librte_regexdev/ +F: drivers/regex/ F: doc/guides/prog_guide/regexdev.rst +F: doc/guides/regexdevs/features/default.ini Eventdev API M: Jerin Jacob @@ -1128,6 +1130,16 @@ F: doc/guides/compressdevs/zlib.rst F: doc/guides/compressdevs/features/zlib.ini +RegEx Drivers +------------------ + +Mellanox MLX5 +M: Ori Kam +F: drivers/regex/mlx5/ +F: doc/guides/regexdevs/mlx5.rst +F: doc/guides/regexdevs/features/mlx5.ini + + vDPA Drivers ------------ T: git://dpdk.org/next/dpdk-next-virtio diff --git a/config/common_base b/config/common_base index f7a8824..f76585f 100644 --- a/config/common_base +++ b/config/common_base @@ -375,6 +375,11 @@ CONFIG_RTE_LIBRTE_MLX5_PMD=n CONFIG_RTE_LIBRTE_MLX5_DEBUG=n # +# Compile regex-oriented Mellanox PMD +# +CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD=n + +# # Compile vdpa-oriented Mellanox ConnectX-6 & BlueField (MLX5) PMD # CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD=n diff --git a/doc/guides/index.rst b/doc/guides/index.rst index 988c6ea..857f036 100644 --- a/doc/guides/index.rst +++ b/doc/guides/index.rst @@ -20,6 +20,7 @@ DPDK documentation cryptodevs/index compressdevs/index vdpadevs/index + regexdevs/index eventdevs/index rawdevs/index mempool/index diff --git a/doc/guides/regexdevs/features/default.ini b/doc/guides/regexdevs/features/default.ini new file mode 100644 index 0000000..4d284dd --- /dev/null +++ b/doc/guides/regexdevs/features/default.ini @@ -0,0 +1,17 @@ +; +; Features of a default RegEx driver. +; +; This file defines the features that are valid for inclusion in +; the other driver files and also the order that they appear in +; the features table in the documentation. The feature description +; string should not exceed feature_str_len defined in conf.py. +; +[Features] +ARMv7 = +ARMv8 = +Power8 = +x86-32 = +x86-64 = +Usage doc = +Design doc = +Perf doc = diff --git a/doc/guides/regexdevs/features/mlx5.ini b/doc/guides/regexdevs/features/mlx5.ini new file mode 100644 index 0000000..fa03d79 --- /dev/null +++ b/doc/guides/regexdevs/features/mlx5.ini @@ -0,0 +1,10 @@ +; +; Supported features of the 'mlx5' RegEx driver. +; +; Refer to default.ini for the full list of available driver features. +; +[Features] +ARMv8 = Y +x86-32 = Y +x86-64 = Y + diff --git a/doc/guides/regexdevs/features_overview.rst b/doc/guides/regexdevs/features_overview.rst new file mode 100644 index 0000000..39c8e96 --- /dev/null +++ b/doc/guides/regexdevs/features_overview.rst @@ -0,0 +1,118 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2020 Mellanox Technologies, Ltd + +Overview of RegEx Drivers Features +================================== + +This section explains the supported features that are listed in the table below. + +Cross buffer + Support cross buffer detection. + +PCRE start anchor + Support PCRE start anchor. + +PCRE atomic grouping + Support PCRE atomic grouping. + +PCRE back reference + Support PCRE back regerence. + +PCRE back tracking ctrl + Support PCRE back tracking ctrl. + +PCRE call outs + Support PCRE call outes. + +PCRE forward reference + Support Forward reference. + +PCRE greedy + Support PCRE greedy mode. + +PCRE match all + Support PCRE match all. + +PCRE match as end + Support match as end. + +PCRE match point rst + Support PCRE match point reset directive. + +PCRE New line conventions + Support new line conventions. + +PCRE new line SEQ + Support new line sequence. + +PCRE look around + Support PCRE look around. + +PCRE possessive qualifiers + Support PCRE possessive qualifiers. + +PCRE subroutine references + Support PCRE subroutine references. + +PCRE UTF 8 + Support UTF-8. + +PCRE UTF 16 + Support UTF-16. + +PCRE UTF 32 + Support UTF-32. + +PCRE word boundary + Support word boundaries. + +Run time compilation + Support compilation during run time. + +ARMv7 + Support armv7 architecture. + +ARMv8 + Support armv8a (64bit) architecture. + +Power8 + Support PowerPC architecture. + +x86-32 + Support 32bits x86 architecture. + +x86-64 + Support 64bits x86 architecture. + +Usage doc + Documentation describes usage, In ``doc/guides/regexdevs/``. + +Design doc + Documentation describes design. In ``doc/guides/regexdevs/``. + +Perf doc + Documentation describes performance values, In ``doc/perf/``. + +.. note:: + + Most of the features capabilities should be provided by the drivers via the + next vDPA operations: ``get_features`` and ``get_protocol_features``. + + +References +========== + + * `PCRE: PCRE patteren man page `_ + + +Features Table +============== + +.. _table_regex_pmd_features: + +.. include:: overview_feature_table.txt + +.. Note:: + + Features marked with "P" are partially supported. Refer to the appropriate + driver guide in the following sections for details. diff --git a/doc/guides/regexdevs/index.rst b/doc/guides/regexdevs/index.rst new file mode 100644 index 0000000..04d8723 --- /dev/null +++ b/doc/guides/regexdevs/index.rst @@ -0,0 +1,15 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2020 Mellanox Technologies, Ltd + +REGEX Device Drivers +==================== + +The following are a list of RegEx (Regular Expression) device drivers, +which can be used from an application through RegEx API. + +.. toctree:: + :maxdepth: 2 + :numbered: + + features_overview + mlx5 diff --git a/doc/guides/regexdevs/mlx5.rst b/doc/guides/regexdevs/mlx5.rst new file mode 100644 index 0000000..39f3067 --- /dev/null +++ b/doc/guides/regexdevs/mlx5.rst @@ -0,0 +1,95 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2020 Mellanox Technologies, Ltd + +.. include:: + +MLX5 RegEx driver +================= + +The MLX5 RegEx (Regular Expression) driver library +(**librte_pmd_mlx5_regex**) provides support for **Mellanox BlueField 2** +families of 25/50/100/200 Gb/s adapters. + +.. note:: + + Due to external dependencies, this driver is disabled in default + configuration of the "make" build. It can be enabled with + ``CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD=y`` or by using "meson" build system which + will detect dependencies. + + +Design +------ + +This PMD is configuring the RegEx HW engine. +For the PMD to work, the application must supply +a precompiled rule file in rof2 format. + +The PMD can use libibverbs and libmlx5 to access the device firmware +or directly the hardware components. +There are different levels of objects and bypassing abilities +to get the best performances: + +- Verbs is a complete high-level generic API +- Direct Verbs is a device-specific API +- DevX allows to access firmware objects +- Direct Rules manages flow steering at low-level hardware layer + +Enabling librte_pmd_mlx5_regex causes DPDK applications to be linked against +libibverbs. + +A Mellanox mlx5 PCI device can be probed by either net/mlx5 driver or regex/mlx5 +driver but not in parallel. Hence, the user should decide the driver by dissabling +the net device using ``CONFIG_RTE_LIBRTE_MLX5_PMD``. + +Supported NICs +-------------- + +* Mellanox\ |reg| BlueField 2 SmartNIC + +Prerequisites +------------- + +- Mellanox OFED version: **5.0** + see :doc:`../../nics/mlx5` guide for more Mellanox OFED details. +- Enable the RegEx caps using system call from the BlueField 2. + Contact Mellanox support for detail explanation. + +Compilation options +~~~~~~~~~~~~~~~~~~~ + +These options can be modified in the ``.config`` file. + +- ``CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD`` (default **n**) + + Toggle compilation of librte_pmd_mlx5 itself. + +- ``CONFIG_RTE_IBVERBS_LINK_DLOPEN`` (default **n**) + + Build PMD with additional code to make it loadable without hard + dependencies on **libibverbs** nor **libmlx5**, which may not be installed + on the target system. + + In this mode, their presence is still required for it to run properly, + however their absence won't prevent a DPDK application from starting (with + ``CONFIG_RTE_BUILD_SHARED_LIB`` disabled) and they won't show up as + missing with ``ldd(1)``. + + It works by moving these dependencies to a purpose-built rdma-core "glue" + plug-in which must either be installed in a directory whose name is based + on ``CONFIG_RTE_EAL_PMD_PATH`` suffixed with ``-glue`` if set, or in a + standard location for the dynamic linker (e.g. ``/lib``) if left to the + default empty string (``""``). + + This option has no performance impact. + +- ``CONFIG_RTE_IBVERBS_LINK_STATIC`` (default **n**) + + Embed static flavor of the dependencies **libibverbs** and **libmlx5** + in the PMD shared library or the executable static binary. + + +Run-time configuration +~~~~~~~~~~~~~~~~~~~~~~ + +- **ethtool** operations on related kernel interfaces also affect the PMD. diff --git a/doc/guides/regexdevs/overview_feature_table.txt b/doc/guides/regexdevs/overview_feature_table.txt new file mode 100644 index 0000000..38b580e --- /dev/null +++ b/doc/guides/regexdevs/overview_feature_table.txt @@ -0,0 +1,105 @@ + +.. raw:: html + + + +.. table:: Features availability in RegEx drivers + + ========================== = + Feature m + l + x + 5 + ========================== = + Cross buffer + PCRE start anchor + PCRE atomic grouping + PCRE back reference + PCRE back tracking ctrl + PCRE call outs + PCRE forward reference + PCRE greedy + PCRE match all + PCRE match as end + PCRE match point rst + PCRE new line conventions + PCRE new line SEQ + PCRE look around + PCRE possessive qualifiers + PCRE subroutine reference + PCRE UTF-8 + PCRE UTF-16 + PCRE UTF-32 + PCRE word boundary + Run time compilation + ARMv7 + ARMv8 Y + Power8 + x86-32 Y + x86-64 Y + Usage doc Y + Design doc Y + Perf doc + ========================== = diff --git a/doc/guides/rel_notes/release_20_08.rst b/doc/guides/rel_notes/release_20_08.rst index 988474c..3154272 100644 --- a/doc/guides/rel_notes/release_20_08.rst +++ b/doc/guides/rel_notes/release_20_08.rst @@ -81,6 +81,11 @@ New Features Added the RegEx library which provides an API for offload of regular expressions search operations to hardware or software accelerator devices. +* **Added the RegEx Library, a generic RegEx service library.** + + Added Mellanox MLX5 RegEx PMD driver, which implements the RegEx library + and allows to offload RegEx searches. + * **Updated PCAP driver.** Updated PCAP driver with new features and improvements, including: diff --git a/drivers/Makefile b/drivers/Makefile index c70bdf9..73a9a72 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -24,5 +24,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += event DEPDIRS-event := common bus mempool net crypto DIRS-$(CONFIG_RTE_LIBRTE_RAWDEV) += raw DEPDIRS-raw := common bus mempool net event +DIRS-$(CONFIG_RTE_LIBRTE_REGEXDEV) += regex +DEPDIRS-regex := common include $(RTE_SDK)/mk/rte.subdir.mk diff --git a/drivers/common/Makefile b/drivers/common/Makefile index df2e840..cbc7107 100644 --- a/drivers/common/Makefile +++ b/drivers/common/Makefile @@ -36,7 +36,7 @@ ifneq (,$(findstring y,$(IAVF-y))) DIRS-y += iavf endif -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) DIRS-y += mlx5 endif diff --git a/drivers/common/mlx5/Makefile b/drivers/common/mlx5/Makefile index f6c762b..5f12be3 100644 --- a/drivers/common/mlx5/Makefile +++ b/drivers/common/mlx5/Makefile @@ -10,7 +10,7 @@ LIB_GLUE_BASE = librte_pmd_mlx5_glue.so LIB_GLUE_VERSION = 20.02.0 # Sources. -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) ifneq ($(CONFIG_RTE_IBVERBS_LINK_DLOPEN),y) SRCS-y += linux/mlx5_glue.c endif @@ -344,7 +344,7 @@ mlx5_autoconf.h: mlx5_autoconf.h.new cmp '$<' '$@' $(AUTOCONF_OUTPUT) || \ mv '$<' '$@' -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) $(SRCS-y:.c=.o): mlx5_autoconf.h endif diff --git a/drivers/meson.build b/drivers/meson.build index 161cfda..cebf9cd 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -11,7 +11,8 @@ dpdk_driver_classes = ['common', 'compress', # depends on common, bus, mempool. 'vdpa', # depends on common, bus and mempool. 'event', # depends on common, bus, mempool and net. - 'baseband'] # depends on common and bus. + 'baseband', # depends on common and bus. + 'regex'] # depends on common, bus, regexdev. disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), ).stdout().split() diff --git a/drivers/regex/Makefile b/drivers/regex/Makefile new file mode 100644 index 0000000..906b205 --- /dev/null +++ b/drivers/regex/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +include $(RTE_SDK)/mk/rte.vars.mk + +DIRS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5 + +include $(RTE_SDK)/mk/rte.subdir.mk diff --git a/drivers/regex/meson.build b/drivers/regex/meson.build new file mode 100644 index 0000000..75522e3 --- /dev/null +++ b/drivers/regex/meson.build @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +drivers = ['mlx5'] +std_deps = ['ethdev', 'kvargs'] # 'ethdev' also pulls in mbuf, net, eal etc +std_deps += ['bus_pci'] # very many PMDs depend on PCI, so make std +std_deps += ['bus_vdev'] # same with vdev bus +config_flag_fmt = 'RTE_LIBRTE_@0@_PMD' +driver_name_fmt = 'rte_pmd_@0@' diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile new file mode 100644 index 0000000..1a16ab2 --- /dev/null +++ b/drivers/regex/mlx5/Makefile @@ -0,0 +1,34 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +include $(RTE_SDK)/mk/rte.vars.mk + +# Library name. +LIB = librte_pmd_mlx5_regex.a + +# Sources. +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c + +# Basic CFLAGS. +CFLAGS += -O3 +CFLAGS += -std=c11 -Wall -Wextra +CFLAGS += -g +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 +CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -Wno-strict-prototypes +LDLIBS += -lrte_common_mlx5 +LDLIBS += -lm +LDLIBS += -lrte_eal -lrte_mbuf +LDLIBS += -lrte_kvargs +LDLIBS += -lrte_bus_pci + +# A few warnings cannot be avoided in external headers. +CFLAGS += -Wno-error=cast-qual + +EXPORT_MAP := rte_pmd_mlx5_regex_version.map + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build new file mode 100644 index 0000000..e31b34a --- /dev/null +++ b/drivers/regex/mlx5/meson.build @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +if not is_linux + build = false + reason = 'only supported on Linux' + subdir_done() +endif + +fmt_name = 'mlx5_regex' +deps += ['common_mlx5', 'pci', 'bus_pci', 'eal', 'sched'] +sources = files( + 'mlx5_regex.c', +) +cflags_options = [ + '-std=c11', + '-Wno-strict-prototypes', + '-D_BSD_SOURCE', + '-D_DEFAULT_SOURCE', + '-D_XOPEN_SOURCE=600' +] +foreach option:cflags_options + if cc.has_argument(option) + cflags += option + endif +endforeach + +if get_option('buildtype').contains('debug') + cflags += [ '-pedantic', '-DPEDANTIC' ] +else + cflags += [ '-UPEDANTIC' ] +endif diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c new file mode 100644 index 0000000..b942a75 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -0,0 +1,5 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include "mlx5_regex.h" diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h new file mode 100644 index 0000000..0e0495c --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#ifndef MLX5_REGEX_H +#define MLX5_REGEX_H + +#endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map b/drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map new file mode 100644 index 0000000..4a76d1d --- /dev/null +++ b/drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map @@ -0,0 +1,3 @@ +DPDK_21 { + local: *; +}; diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 0ce8cf5..1b9551e 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -201,11 +201,12 @@ endif _LDLIBS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += -lrte_pmd_lio _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_MEMIF) += -lrte_pmd_memif _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += -lrte_pmd_mlx4 -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) _LDLIBS-y += -lrte_common_mlx5 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += -lrte_pmd_mlx5 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD) += -lrte_pmd_mlx5_vdpa +_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += -lrte_pmd_mlx5_regex ifeq ($(CONFIG_RTE_IBVERBS_LINK_DLOPEN),y) _LDLIBS-y += -ldl else ifeq ($(CONFIG_RTE_IBVERBS_LINK_STATIC),y) @@ -214,7 +215,7 @@ _LDLIBS-y += --no-whole-archive _LDLIBS-y += $(LIBS_IBVERBS_STATIC) _LDLIBS-y += --whole-archive else -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) _LDLIBS-y += -libverbs -lmlx5 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += -libverbs -lmlx4 From patchwork Sun Jul 12 20:58:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73839 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C48CAA052A; Sun, 12 Jul 2020 22:59:30 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C9C191D50A; Sun, 12 Jul 2020 22:59:27 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 3FC9D1D50A for ; Sun, 12 Jul 2020 22:59:26 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:25 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFFv025997; Sun, 12 Jul 2020 23:59:25 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 12 Jul 2020 20:58:43 +0000 Message-Id: <1594587541-110442-3-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 02/20] regex/mlx5: add log utils X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Add the DRV_LOG macro which should be used for error prints. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/mlx5_regex.c | 4 ++++ drivers/regex/mlx5/mlx5_regex_utils.h | 19 +++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 drivers/regex/mlx5/mlx5_regex_utils.h diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 1a16ab2..f495659 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -15,6 +15,7 @@ CFLAGS += -std=c11 -Wall -Wextra CFLAGS += -g CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux CFLAGS += -D_BSD_SOURCE CFLAGS += -D_DEFAULT_SOURCE CFLAGS += -D_XOPEN_SOURCE=600 diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index b942a75..06826a6 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -3,3 +3,7 @@ */ #include "mlx5_regex.h" +#include "mlx5_regex_utils.h" + +int mlx5_regex_logtype; + diff --git a/drivers/regex/mlx5/mlx5_regex_utils.h b/drivers/regex/mlx5/mlx5_regex_utils.h new file mode 100644 index 0000000..adca846 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex_utils.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#ifndef RTE_PMD_MLX5_REGEX_UTILS_H_ +#define RTE_PMD_MLX5_REGEX_UTILS_H_ + +#include + +extern int mlx5_regex_logtype; + +#define MLX5_REGEX_LOG_PREFIX "regex_mlx5" +/* Generic printf()-like logging macro with automatic line feed. */ +#define DRV_LOG(level, ...) \ + PMD_DRV_LOG_(level, mlx5_regex_logtype, MLX5_REGEX_LOG_PREFIX, \ + __VA_ARGS__ PMD_DRV_LOG_STRIP PMD_DRV_LOG_OPAREN, \ + PMD_DRV_LOG_CPAREN) + +#endif /* RTE_PMD_MLX5_REGEX_UTILS_H_ */ From patchwork Sun Jul 12 20:58:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73841 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5BCF2A052A; Sun, 12 Jul 2020 22:59:48 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 695BE1D51E; Sun, 12 Jul 2020 22:59:32 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 308C31D51D for ; Sun, 12 Jul 2020 22:59:31 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:29 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFFw025997; Sun, 12 Jul 2020 23:59:29 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 12 Jul 2020 20:58:44 +0000 Message-Id: <1594587541-110442-4-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 03/20] common/mlx5: add MMO and regexp structs/opcodes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Added General purpose PRM MMO structs, and regex specific structs. Signed-off-by: Yuval Avnery Signed-off-by: Ori Kam --- drivers/common/mlx5/mlx5_prm.h | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index c63795f..c2b9a20 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -373,6 +373,42 @@ struct mlx5_cqe { uint8_t op_own; }; +/* MMO metadata segment */ + +#define MLX5_OPCODE_MMO 0x2f +#define MLX5_OPC_MOD_MMO_REGEX 0x4 + +struct mlx5_wqe_metadata_seg { + uint32_t mmo_control_31_0; /* mmo_control_63_32 is in ctrl_seg.imm */ + uint32_t lkey; + uint64_t addr; +}; + +struct mlx5_ifc_regexp_mmo_control_bits { + uint8_t reserved_at_31[0x2]; + uint8_t le[0x1]; + uint8_t reserved_at_28[0x1]; + uint8_t subset_id_0[0xc]; + uint8_t reserved_at_16[0x4]; + uint8_t subset_id_1[0xc]; + uint8_t ctrl[0x4]; + uint8_t subset_id_2[0xc]; + uint8_t reserved_at_16_1[0x4]; + uint8_t subset_id_3[0xc]; +}; + +struct mlx5_ifc_regexp_metadata_bits { + uint8_t rof_version[0x10]; + uint8_t latency_count[0x10]; + uint8_t instruction_count[0x10]; + uint8_t primary_thread_count[0x10]; + uint8_t match_count[0x8]; + uint8_t detected_match_count[0x8]; + uint8_t status[0x10]; + uint8_t job_id[0x20]; + uint8_t reserved[0x80]; +}; + /* Adding direct verbs to data-path. */ /* CQ sequence number mask. */ @@ -759,6 +795,10 @@ enum { MLX5_CMD_OP_CREATE_GENERAL_OBJECT = 0xa00, MLX5_CMD_OP_MODIFY_GENERAL_OBJECT = 0xa01, MLX5_CMD_OP_QUERY_GENERAL_OBJECT = 0xa02, + MLX5_CMD_SET_REGEX_PARAM = 0xb04, + MLX5_CMD_QUERY_REGEX_PARAMS = 0xb05, + MLX5_CMD_SET_REGEX_REGISTERS = 0xb06, + MLX5_CMD_QUERY_REGEX_REGISTERS = 0xb07, }; enum { From patchwork Sun Jul 12 20:58:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73842 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4FB27A052A; Sun, 12 Jul 2020 22:59:59 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0BE9C1D51D; Sun, 12 Jul 2020 22:59:38 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 33A4F1D51D for ; Sun, 12 Jul 2020 22:59:36 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:32 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFFx025997; Sun, 12 Jul 2020 23:59:32 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 12 Jul 2020 20:58:45 +0000 Message-Id: <1594587541-110442-5-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 04/20] common/mlx5: add mlx5 regex command structs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Add regex commands structs to support regex. Signed-off-by: Yuval Avnery Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_prm.h | 89 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 88 insertions(+), 1 deletion(-) diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index c2b9a20..ede7810 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -795,7 +795,7 @@ enum { MLX5_CMD_OP_CREATE_GENERAL_OBJECT = 0xa00, MLX5_CMD_OP_MODIFY_GENERAL_OBJECT = 0xa01, MLX5_CMD_OP_QUERY_GENERAL_OBJECT = 0xa02, - MLX5_CMD_SET_REGEX_PARAM = 0xb04, + MLX5_CMD_SET_REGEX_PARAMS = 0xb04, MLX5_CMD_QUERY_REGEX_PARAMS = 0xb05, MLX5_CMD_SET_REGEX_REGISTERS = 0xb06, MLX5_CMD_QUERY_REGEX_REGISTERS = 0xb07, @@ -2526,6 +2526,93 @@ struct mlx5_ifc_query_qp_in_bits { u8 reserved_at_60[0x20]; }; +struct regexp_params_field_select_bits { + u8 reserved_at_0[0x1e]; + u8 stop_engine[0x1]; + u8 db_umem_id[0x1]; +}; + +struct mlx5_ifc_regexp_params_bits { + u8 reserved_at_0[0x1f]; + u8 stop_engine[0x1]; + u8 db_umem_id[0x20]; + u8 db_umem_offset[0x40]; + u8 reserved_at_80[0x100]; +}; + +struct mlx5_ifc_set_regexp_params_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + struct regexp_params_field_select_bits field_select; + struct mlx5_ifc_regexp_params_bits regexp_params; +}; + +struct mlx5_ifc_set_regexp_params_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved_at_18[0x40]; +}; + +struct mlx5_ifc_query_regexp_params_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + u8 reserved[0x20]; +}; + +struct mlx5_ifc_query_regexp_params_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved[0x40]; + struct mlx5_ifc_regexp_params_bits regexp_params; +}; + +struct mlx5_ifc_set_regexp_register_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + u8 register_address[0x20]; + u8 register_data[0x20]; + u8 reserved[0x40]; +}; + +struct mlx5_ifc_set_regexp_register_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved[0x40]; +}; + +struct mlx5_ifc_query_regexp_register_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + u8 register_address[0x20]; +}; + +struct mlx5_ifc_query_regexp_register_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved[0x20]; + u8 register_data[0x20]; +}; + /* CQE format mask. */ #define MLX5E_CQE_FORMAT_MASK 0xc From patchwork Sun Jul 12 20:58:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73843 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 56525A052A; Sun, 12 Jul 2020 23:00:08 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 62D8F1D534; Sun, 12 Jul 2020 22:59:42 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 3991C1D531 for ; Sun, 12 Jul 2020 22:59:41 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:37 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFG0025997; Sun, 12 Jul 2020 23:59:36 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 12 Jul 2020 20:58:46 +0000 Message-Id: <1594587541-110442-6-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 05/20] common/mlx5: add support for regex capability query X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Update hca cap struct and common query hca cap function. Signed-off-by: Yuval Avnery Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++ drivers/common/mlx5/mlx5_prm.h | 9 +++++++-- 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 2179a83..54b20a7 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -467,6 +467,9 @@ struct mlx5_devx_obj * attr->vdpa.queue_counters_valid = !!(MLX5_GET64(cmd_hca_cap, hcattr, general_obj_types) & MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS); + attr->regex = MLX5_GET(cmd_hca_cap, hcattr, regexp); + attr->regexp_num_of_engines = MLX5_GET(cmd_hca_cap, hcattr, + regexp_num_of_engines); if (attr->qos.sup) { MLX5_SET(query_hca_cap_in, in, op_mod, MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP | diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 25704ef..bb14ca5 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -90,6 +90,8 @@ struct mlx5_hca_attr { uint32_t vhca_id:16; uint32_t relaxed_ordering_write:1; uint32_t relaxed_ordering_read:1; + uint32_t regex:1; + uint32_t regexp_num_of_engines; struct mlx5_hca_qos_attr qos; struct mlx5_hca_vdpa_attr vdpa; }; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index ede7810..bfbc58b 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -1034,9 +1034,14 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 log_max_qp_sz[0x8]; u8 reserved_at_90[0xb]; u8 log_max_qp[0x5]; - u8 reserved_at_a0[0xb]; + u8 regexp[0x1]; + u8 reserved_at_a1[0x3]; + u8 regexp_num_of_engines[0x4]; + u8 reserved_at_a8[0x3]; u8 log_max_srq[0x5]; - u8 reserved_at_b0[0x10]; + u8 reserved_at_b0[0x3]; + u8 regexp_log_crspace_size[0x5]; + u8 reserved_at_b8[0x8]; u8 reserved_at_c0[0x8]; u8 log_max_cq_sz[0x8]; u8 reserved_at_d0[0xb]; From patchwork Sun Jul 12 20:58:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73844 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id AC255A052A; Sun, 12 Jul 2020 23:00:18 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A7A0F1D53B; Sun, 12 Jul 2020 22:59:43 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 436101D532 for ; Sun, 12 Jul 2020 22:59:41 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:39 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFG1025997; Sun, 12 Jul 2020 23:59:39 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Thomas Monjalon Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, orika@mellanox.com, rasland@mellanox.com, Parav Pandit Date: Sun, 12 Jul 2020 20:58:47 +0000 Message-Id: <1594587541-110442-7-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 06/20] regex/mlx5: add probe function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the probe function to the RegEx PMD. Signed-off-by: Parav Pandit Signed-off-by: Ori Kam --- drivers/regex/mlx5/Makefile | 3 + drivers/regex/mlx5/meson.build | 2 +- drivers/regex/mlx5/mlx5_regex.c | 215 ++++++++++++++++++++++++++++++++++++++++ drivers/regex/mlx5/mlx5_regex.h | 6 ++ mk/rte.app.mk | 1 + 5 files changed, 226 insertions(+), 1 deletion(-) diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index f495659..3b99570 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -16,6 +16,8 @@ CFLAGS += -g CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux +CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5/linux CFLAGS += -D_BSD_SOURCE CFLAGS += -D_DEFAULT_SOURCE CFLAGS += -D_XOPEN_SOURCE=600 @@ -26,6 +28,7 @@ LDLIBS += -lm LDLIBS += -lrte_eal -lrte_mbuf LDLIBS += -lrte_kvargs LDLIBS += -lrte_bus_pci +LDLIBS += -lrte_regexdev # A few warnings cannot be avoided in external headers. CFLAGS += -Wno-error=cast-qual diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index e31b34a..327c0ad 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -8,7 +8,7 @@ if not is_linux endif fmt_name = 'mlx5_regex' -deps += ['common_mlx5', 'pci', 'bus_pci', 'eal', 'sched'] +deps += ['common_mlx5', 'pci', 'bus_pci', 'eal', 'sched', 'regexdev'] sources = files( 'mlx5_regex.c', ) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 06826a6..d264ecd 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -2,8 +2,223 @@ * Copyright 2020 Mellanox Technologies, Ltd */ +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + #include "mlx5_regex.h" #include "mlx5_regex_utils.h" int mlx5_regex_logtype; +static const struct rte_regexdev_ops mlx5_regexdev_ops = {}; + +static struct ibv_device * +mlx5_regex_get_ib_device_match(struct rte_pci_addr *addr) +{ + int n; + struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n); + struct ibv_device *ibv_match = NULL; + + if (!ibv_list) { + rte_errno = ENOSYS; + return NULL; + } + while (n-- > 0) { + struct rte_pci_addr pci_addr; + + DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name); + if (mlx5_dev_to_pci_addr(ibv_list[n]->ibdev_path, &pci_addr)) + continue; + if (rte_pci_addr_cmp(addr, &pci_addr)) + continue; + ibv_match = ibv_list[n]; + break; + } + if (!ibv_match) + rte_errno = ENOENT; + mlx5_glue->free_device_list(ibv_list); + return ibv_match; +} + +static void +mlx5_regex_get_name(char *name, struct rte_pci_device *pci_dev __rte_unused) +{ + sprintf(name, "mlx5_regex_%02x:%02x.%02x", pci_dev->addr.bus, + pci_dev->addr.devid, pci_dev->addr.function); +} + +/** + * DPDK callback to register a PCI device. + * + * This function spawns RegEx device out of a given PCI device. + * + * @param[in] pci_drv + * PCI driver structure (mlx5_regex_driver). + * @param[in] pci_dev + * PCI device information. + * + * @return + * 0 on success, 1 to skip this driver, a negative errno value otherwise + * and rte_errno is set. + */ +static int +mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) +{ + struct ibv_device *ibv; + struct mlx5_regex_priv *priv = NULL; + struct ibv_context *ctx = NULL; + struct mlx5_hca_attr attr; + char name[RTE_REGEXDEV_NAME_MAX_LEN]; + int ret; + + ibv = mlx5_regex_get_ib_device_match(&pci_dev->addr); + if (!ibv) { + DRV_LOG(ERR, "No matching IB device for PCI slot " + PCI_PRI_FMT ".", pci_dev->addr.domain, + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + return -rte_errno; + } + DRV_LOG(INFO, "PCI information matches for device \"%s\".", + ibv->name); + ctx = mlx5_glue->dv_open_device(ibv); + if (!ctx) { + DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); + rte_errno = ENODEV; + return -rte_errno; + } + ret = mlx5_devx_cmd_query_hca_attr(ctx, &attr); + if (ret) { + DRV_LOG(ERR, "Unable to read HCA capabilities."); + rte_errno = ENOTSUP; + goto error; + } else if (!attr.regex || attr.regexp_num_of_engines == 0) { + DRV_LOG(ERR, "Not enough capabilities to support RegEx, maybe " + "old FW/OFED version?"); + rte_errno = ENOTSUP; + goto error; + } + priv = rte_zmalloc("mlx5 regex device private", sizeof(*priv), + RTE_CACHE_LINE_SIZE); + if (!priv) { + DRV_LOG(ERR, "Failed to allocate private memory."); + rte_errno = ENOMEM; + goto error; + } + priv->ctx = ctx; + mlx5_regex_get_name(name, pci_dev); + priv->regexdev = rte_regexdev_register(name); + if (priv->regexdev == NULL) { + DRV_LOG(ERR, "Failed to register RegEx device."); + rte_errno = rte_errno ? rte_errno : EINVAL; + goto error; + } + priv->regexdev->dev_ops = &mlx5_regexdev_ops; + priv->regexdev->device = (struct rte_device *)pci_dev; + priv->regexdev->data->dev_private = priv; + return 0; + +error: + if (ctx) + mlx5_glue->close_device(ctx); + if (priv) + rte_free(priv); + return -rte_errno; +} + +/** + * DPDK callback to remove a PCI device. + * + * This function removes all RegEx devices belong to a given PCI device. + * + * @param[in] pci_dev + * Pointer to the PCI device. + * + * @return + * 0 on success, the function cannot fail. + */ +static int +mlx5_regex_pci_remove(struct rte_pci_device *pci_dev) +{ + char name[RTE_REGEXDEV_NAME_MAX_LEN]; + struct rte_regexdev *dev; + struct mlx5_regex_priv *priv = NULL; + + mlx5_regex_get_name(name, pci_dev); + dev = rte_regexdev_get_device_by_name(name); + if (!dev) + return 0; + priv = dev->data->dev_private; + if (priv) { + if (priv->ctx) + mlx5_glue->close_device(priv->ctx); + if (priv->regexdev) + rte_regexdev_unregister(priv->regexdev); + rte_free(priv); + } + return 0; +} + +static const struct rte_pci_id mlx5_regex_pci_id_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6) + }, + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6VF) + }, + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6DX) + }, + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6DXVF) + }, + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6DXBF) + }, + { + .vendor_id = 0 + } +}; + +static struct rte_pci_driver mlx5_regex_driver = { + .driver = { + .name = "mlx5_regex", + }, + .id_table = mlx5_regex_pci_id_map, + .probe = mlx5_regex_pci_probe, + .remove = mlx5_regex_pci_remove, + .drv_flags = 0, +}; + +/** + * Driver initialization routine. + */ +RTE_INIT(rte_mlx5_regex_init) +{ + /* Initialize common log type. */ + mlx5_regex_logtype = rte_log_register("pmd.regex.mlx5"); + if (mlx5_regex_logtype >= 0) + rte_log_set_level(mlx5_regex_logtype, RTE_LOG_NOTICE); + if (mlx5_glue) + rte_pci_register(&mlx5_regex_driver); +} + +RTE_PMD_EXPORT_NAME(net_mlx5_regex, __COUNTER__); +RTE_PMD_REGISTER_PCI_TABLE(net_mlx5_regex, mlx5_regex_pci_id_map); +RTE_PMD_REGISTER_KMOD_DEP(net_mlx5_regex, "* ib_uverbs & mlx5_core & mlx5_ib"); diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 0e0495c..0ce1e4d 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -5,4 +5,10 @@ #ifndef MLX5_REGEX_H #define MLX5_REGEX_H +struct mlx5_regex_priv { + TAILQ_ENTRY(mlx5_regex_priv) next; + struct ibv_context *ctx; /* Device context. */ + struct rte_pci_device *pci_dev; + struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ +}; #endif /* MLX5_REGEX_H */ diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 1b9551e..6efd7ae 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -101,6 +101,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched _LDLIBS-$(CONFIG_RTE_LIBRTE_RCU) += -lrte_rcu _LDLIBS-$(CONFIG_RTE_LIBRTE_GRAPH) += -lrte_graph _LDLIBS-$(CONFIG_RTE_LIBRTE_NODE) += -lrte_node +_LDLIBS-$(CONFIG_RTE_LIBRTE_REGEXDEV) += -lrte_regexdev ifeq ($(CONFIG_RTE_EXEC_ENV_LINUX),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_KNI) += -lrte_kni From patchwork Sun Jul 12 20:58:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73845 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5C1C8A052A; Sun, 12 Jul 2020 23:00:29 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0E3FA1D542; Sun, 12 Jul 2020 22:59:49 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 48C2F1D541 for ; Sun, 12 Jul 2020 22:59:46 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:43 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFG2025997; Sun, 12 Jul 2020 23:59:43 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler , Ray Kinsella , Neil Horman Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 12 Jul 2020 20:58:48 +0000 Message-Id: <1594587541-110442-8-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 07/20] common/mlx5: add rxp database set cmd X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the database set command for the RXP engine. Signed-off-by: Ori Kam Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_devx_cmds.c | 104 ++++++++++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 8 +- drivers/common/mlx5/rte_common_mlx5_version.map | 3 + 3 files changed, 114 insertions(+), 1 deletion(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 54b20a7..fe781c7 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1616,3 +1616,107 @@ struct mlx5_devx_obj * invalid_buffer); return ret; } + +/** + * Issue the RXP stop database command. + * + * @param[in] ctx + * Context returned from mlx5 open_device() glue function. + * @param[in] engine + * The engine to stop. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_regex_database_stop(void *ctx, uint8_t engine) +{ + uint32_t out[MLX5_ST_SZ_DW(set_regexp_params_out)] = {0}; + uint32_t in[MLX5_ST_SZ_DW(set_regexp_params_in)] = {0}; + int ret; + + MLX5_SET(set_regexp_params_in, in, opcode, MLX5_CMD_SET_REGEX_PARAMS); + MLX5_SET(set_regexp_params_in, in, engine_id, engine); + MLX5_SET(set_regexp_params_in, in, regexp_params.stop_engine, 1); + MLX5_SET(set_regexp_params_in, in, field_select.stop_engine, 1); + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Database stop failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + +/** + * Issue the RXP resume database command. + * + * @param[in] ctx + * Context returned from mlx5 open_device() glue function. + * @param[in] engine + * The engine to start. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_regex_database_resume(void *ctx, uint8_t engine) +{ + uint32_t out[MLX5_ST_SZ_DW(set_regexp_params_out)] = {0}; + uint32_t in[MLX5_ST_SZ_DW(set_regexp_params_in)] = {0}; + int ret; + + MLX5_SET(set_regexp_params_in, in, opcode, MLX5_CMD_SET_REGEX_PARAMS); + MLX5_SET(set_regexp_params_in, in, engine_id, engine); + MLX5_SET(set_regexp_params_in, in, regexp_params.stop_engine, 0); + MLX5_SET(set_regexp_params_in, in, field_select.stop_engine, 1); + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Database start failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + +/** + * Issue the RXP program database command. + * + * @param[in] ctx + * Context returned from mlx5 open_device() glue function. + * @param[in] engine + * The engine to program. + * @param[in] umem_id + * The umem id to use. + * @param[in] umem_offset + * The offset in the umem to start copying from. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_regex_database_program(void *ctx, uint8_t engine, uint32_t umem_id, + uint64_t umem_offset) +{ + uint32_t out[MLX5_ST_SZ_DW(set_regexp_params_out)] = {0}; + uint32_t in[MLX5_ST_SZ_DW(set_regexp_params_in)] = {0}; + int ret; + + MLX5_SET(set_regexp_params_in, in, opcode, MLX5_CMD_SET_REGEX_PARAMS); + MLX5_SET(set_regexp_params_in, in, engine_id, engine); + MLX5_SET(set_regexp_params_in, in, regexp_params.db_umem_id, umem_id); + MLX5_SET64(set_regexp_params_in, in, regexp_params.db_umem_offset, + umem_offset); + MLX5_SET(set_regexp_params_in, in, field_select.db_umem_id, 1); + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Database program failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index bb14ca5..655e31f 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -401,5 +401,11 @@ int mlx5_devx_cmd_modify_rqt(struct mlx5_devx_obj *rqt, __rte_internal int mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj, struct mlx5_devx_virtio_q_couners_attr *attr); - +__rte_internal +int mlx5_devx_regex_database_stop(void *ctx, uint8_t engine); +__rte_internal +int mlx5_devx_regex_database_resume(void *ctx, uint8_t engine); +__rte_internal +int mlx5_devx_regex_database_program(void *ctx, uint8_t engine, + uint32_t umem_id, uint64_t umem_offset); #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */ diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map index ae57ebd..6054d39 100644 --- a/drivers/common/mlx5/rte_common_mlx5_version.map +++ b/drivers/common/mlx5/rte_common_mlx5_version.map @@ -35,6 +35,9 @@ INTERNAL { mlx5_devx_cmd_query_virtio_q_counters; mlx5_devx_cmd_query_virtq; mlx5_devx_get_out_command_status; + mlx5_devx_regex_database_program; + mlx5_devx_regex_database_resume; + mlx5_devx_regex_database_stop; mlx5_get_ifname_sysfs; mlx5_get_dbr; From patchwork Sun Jul 12 20:58:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73846 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4ABEDA052A; Sun, 12 Jul 2020 23:00:42 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0DD221D558; Sun, 12 Jul 2020 22:59:53 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 49B6C1D539 for ; Sun, 12 Jul 2020 22:59:51 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:48 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFG3025997; Sun, 12 Jul 2020 23:59:47 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Parav Pandit Date: Sun, 12 Jul 2020 20:58:49 +0000 Message-Id: <1594587541-110442-9-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 08/20] regex/mlx5: add RXP register definitions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Parav Pandit This commit indroduce the mlx5_rxp_csrs.h file. This file holds all the relevant defines for the RXP engine. Signed-off-by: Parav Pandit Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_rxp_csrs.h | 338 +++++++++++++++++++++++++++++++++++++ 1 file changed, 338 insertions(+) create mode 100644 drivers/regex/mlx5/mlx5_rxp_csrs.h diff --git a/drivers/regex/mlx5/mlx5_rxp_csrs.h b/drivers/regex/mlx5/mlx5_rxp_csrs.h new file mode 100644 index 0000000..bab2946 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_rxp_csrs.h @@ -0,0 +1,338 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ +#ifndef _MLX5_RXP_CSRS_H_ +#define _MLX5_RXP_CSRS_H_ + +/* + * Common to all RXP implementations + */ +#define MLX5_RXP_CSR_BASE_ADDRESS 0x0000ul +#define MLX5_RXP_RTRU_CSR_BASE_ADDRESS 0x0100ul +#define MLX5_RXP_STATS_CSR_BASE_ADDRESS 0x0200ul +#define MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS 0x0600ul + +#define MLX5_RXP_CSR_WIDTH 4 + +/* This is the identifier we expect to see in the first RXP CSR */ +#define MLX5_RXP_IDENTIFER 0x5254 + +/* Hyperion specific BAR0 offsets */ +#define MLX5_RXP_FPGA_BASE_ADDRESS 0x0000ul +#define MLX5_RXP_PCIE_BASE_ADDRESS 0x1000ul +#define MLX5_RXP_IDMA_BASE_ADDRESS 0x2000ul +#define MLX5_RXP_EDMA_BASE_ADDRESS 0x3000ul +#define MLX5_RXP_SYSMON_BASE_ADDRESS 0xf300ul +#define MLX5_RXP_ISP_CSR_BASE_ADDRESS 0xf400ul + +/* Offset to the RXP common 4K CSR space */ +#define MLX5_RXP_PCIE_CSR_BASE_ADDRESS 0xf000ul + +/* FPGA CSRs */ + +#define MLX5_RXP_FPGA_VERSION (MLX5_RXP_FPGA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) + +/* PCIe CSRs */ +#define MLX5_RXP_PCIE_INIT_ISR (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_PCIE_INIT_IMR (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_PCIE_INIT_CFG_STAT (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +#define MLX5_RXP_PCIE_INIT_FLR (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 3) +#define MLX5_RXP_PCIE_INIT_CTRL (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 4) + +/* IDMA CSRs */ +#define MLX5_RXP_IDMA_ISR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_IDMA_IMR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_IDMA_CSR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 4) +#define MLX5_RXP_IDMA_CSR_RST_MSK 0x0001 +#define MLX5_RXP_IDMA_CSR_PDONE_MSK 0x0002 +#define MLX5_RXP_IDMA_CSR_INIT_MSK 0x0004 +#define MLX5_RXP_IDMA_CSR_EN_MSK 0x0008 +#define MLX5_RXP_IDMA_QCR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 5) +#define MLX5_RXP_IDMA_QCR_QAVAIL_MSK 0x00FF +#define MLX5_RXP_IDMA_QCR_QEN_MSK 0xFF00 +#define MLX5_RXP_IDMA_DCR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 6) +#define MLX5_RXP_IDMA_DWCTR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 7) +#define MLX5_RXP_IDMA_DWTOR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 8) +#define MLX5_RXP_IDMA_PADCR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 9) +#define MLX5_RXP_IDMA_DFCR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_IDMA_FOFLR0 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_IDMA_FOFLR1 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_IDMA_FOFLR2 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +#define MLX5_RXP_IDMA_FUFLR0 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_IDMA_FUFLR1 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_IDMA_FUFLR2 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) + +#define MLX5_RXP_IDMA_QCSR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 128) +#define MLX5_RXP_IDMA_QCSR_RST_MSK 0x0001 +#define MLX5_RXP_IDMA_QCSR_PDONE_MSK 0x0002 +#define MLX5_RXP_IDMA_QCSR_INIT_MSK 0x0004 +#define MLX5_RXP_IDMA_QCSR_EN_MSK 0x0008 +#define MLX5_RXP_IDMA_QDPTR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 192) +#define MLX5_RXP_IDMA_QTPTR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 256) +#define MLX5_RXP_IDMA_QDRPTR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 320) +#define MLX5_RXP_IDMA_QDRALR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 384) +#define MLX5_RXP_IDMA_QDRAHR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 385) + +/* EDMA CSRs */ +#define MLX5_RXP_EDMA_ISR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_EDMA_IMR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_EDMA_CSR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 4) +#define MLX5_RXP_EDMA_CSR_RST_MSK 0x0001 +#define MLX5_RXP_EDMA_CSR_PDONE_MSK 0x0002 +#define MLX5_RXP_EDMA_CSR_INIT_MSK 0x0004 +#define MLX5_RXP_EDMA_CSR_EN_MSK 0x0008 +#define MLX5_RXP_EDMA_QCR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 5) +#define MLX5_RXP_EDMA_QCR_QAVAIL_MSK 0x00FF +#define MLX5_RXP_EDMA_QCR_QEN_MSK 0xFF00 +#define MLX5_RXP_EDMA_DCR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 6) +#define MLX5_RXP_EDMA_DWCTR (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 7) +#define MLX5_RXP_EDMA_DWTOR (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 8) +#define MLX5_RXP_EDMA_DFCR (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_EDMA_FOFLR0 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_EDMA_FOFLR1 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_EDMA_FOFLR2 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +#define MLX5_RXP_EDMA_FUFLR0 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_EDMA_FUFLR1 (MLX5_RXP_EDMA_BASE_ADDRESS +\ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_EDMA_FUFLR2 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) + +#define MLX5_RXP_EDMA_QCSR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 128) +#define MLX5_RXP_EDMA_QCSR_RST_MSK 0x0001 +#define MLX5_RXP_EDMA_QCSR_PDONE_MSK 0x0002 +#define MLX5_RXP_EDMA_QCSR_INIT_MSK 0x0004 +#define MLX5_RXP_EDMA_QCSR_EN_MSK 0x0008 +#define MLX5_RXP_EDMA_QTPTR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 256) +#define MLX5_RXP_EDMA_QDRPTR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 320) +#define MLX5_RXP_EDMA_QDRALR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 384) +#define MLX5_RXP_EDMA_QDRAHR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 385) + +/* Main CSRs */ +#define MLX5_RXP_CSR_IDENTIFIER (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_CSR_REVISION (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_CSR_CAPABILITY_0 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +#define MLX5_RXP_CSR_CAPABILITY_1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 3) +#define MLX5_RXP_CSR_CAPABILITY_2 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 4) +#define MLX5_RXP_CSR_CAPABILITY_3 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 5) +#define MLX5_RXP_CSR_CAPABILITY_4 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 6) +#define MLX5_RXP_CSR_CAPABILITY_5 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 7) +#define MLX5_RXP_CSR_CAPABILITY_6 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 8) +#define MLX5_RXP_CSR_CAPABILITY_7 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 9) +#define MLX5_RXP_CSR_STATUS (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_CSR_STATUS_INIT_DONE 0x0001 +#define MLX5_RXP_CSR_STATUS_GOING 0x0008 +#define MLX5_RXP_CSR_STATUS_IDLE 0x0040 +#define MLX5_RXP_CSR_STATUS_TRACKER_OK 0x0080 +#define MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT 0x0100 +#define MLX5_RXP_CSR_FIFO_STATUS_0 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 11) +#define MLX5_RXP_CSR_FIFO_STATUS_1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 12) +#define MLX5_RXP_CSR_JOB_DDOS_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 13) +/* 14 + 15 reserved */ +#define MLX5_RXP_CSR_CORE_CLK_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_CSR_WRITE_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_CSR_JOB_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +#define MLX5_RXP_CSR_JOB_ERROR_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 19) +#define MLX5_RXP_CSR_JOB_BYTE_COUNT0 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 20) +#define MLX5_RXP_CSR_JOB_BYTE_COUNT1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 21) +#define MLX5_RXP_CSR_RESPONSE_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 22) +#define MLX5_RXP_CSR_MATCH_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 23) +#define MLX5_RXP_CSR_CTRL (MLX5_RXP_CSR_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_CSR_CTRL_INIT 0x0001 +#define MLX5_RXP_CSR_CTRL_GO 0x0008 +#define MLX5_RXP_CSR_MAX_MATCH (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_CSR_MAX_PREFIX (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) +#define MLX5_RXP_CSR_MAX_PRI_THREAD (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 27) +#define MLX5_RXP_CSR_MAX_LATENCY (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 28) +#define MLX5_RXP_CSR_SCRATCH_1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 29) +#define MLX5_RXP_CSR_CLUSTER_MASK (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 30) +#define MLX5_RXP_CSR_INTRA_CLUSTER_MASK (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 31) + +/* Runtime Rule Update CSRs */ +/* 0 + 1 reserved */ +#define MLX5_RXP_RTRU_CSR_CAPABILITY (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +/* 3-9 reserved */ +#define MLX5_RXP_RTRU_CSR_STATUS (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_RTRU_CSR_STATUS_UPDATE_DONE 0x0002 +#define MLX5_RXP_RTRU_CSR_STATUS_IM_INIT_DONE 0x0010 +#define MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE 0x0020 +#define MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE 0x0040 +#define MLX5_RXP_RTRU_CSR_STATUS_EM_INIT_DONE 0x0080 +#define MLX5_RXP_RTRU_CSR_FIFO_STAT (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 11) +/* 12-15 reserved */ +#define MLX5_RXP_RTRU_CSR_CHECKSUM_0 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_RTRU_CSR_CHECKSUM_1 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_RTRU_CSR_CHECKSUM_2 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +/* 19 + 20 reserved */ +#define MLX5_RXP_RTRU_CSR_RTRU_COUNT (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 21) +#define MLX5_RXP_RTRU_CSR_ROF_REV (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 22) +/* 23 reserved */ +#define MLX5_RXP_RTRU_CSR_CTRL (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT 0x0001 +#define MLX5_RXP_RTRU_CSR_CTRL_GO 0x0002 +#define MLX5_RXP_RTRU_CSR_CTRL_SIP 0x0004 +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_MASK (3 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2_EM (0 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2 (1 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2 (2 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_EM (3 << 4) +#define MLX5_RXP_RTRU_CSR_ADDR (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_RTRU_CSR_DATA_0 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) +#define MLX5_RXP_RTRU_CSR_DATA_1 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 27) +/* 28-31 reserved */ + +/* Statistics CSRs */ +#define MLX5_RXP_STATS_CSR_CLUSTER (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_STATS_CSR_L2_CACHE (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_STATS_CSR_MPFE_FIFO (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_STATS_CSR_PE (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 28) +#define MLX5_RXP_STATS_CSR_CP (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 30) +#define MLX5_RXP_STATS_CSR_DP (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 31) + +/* Sysmon Stats CSRs */ +#define MLX5_RXP_SYSMON_CSR_T_FPGA (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_SYSMON_CSR_V_VCCINT (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_SYSMON_CSR_V_VCCAUX (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +#define MLX5_RXP_SYSMON_CSR_T_U1 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 20) +#define MLX5_RXP_SYSMON_CSR_I_EDG12V (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 21) +#define MLX5_RXP_SYSMON_CSR_I_VCC3V3 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 22) +#define MLX5_RXP_SYSMON_CSR_I_VCC2V5 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 23) +#define MLX5_RXP_SYSMON_CSR_T_U2 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 28) +#define MLX5_RXP_SYSMON_CSR_I_AUX12V (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 29) +#define MLX5_RXP_SYSMON_CSR_I_VCC1V8 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 30) +#define MLX5_RXP_SYSMON_CSR_I_VDDR3 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 31) + +/* In Service Programming CSRs */ + +/* RXP-F1 and RXP-ZYNQ specific CSRs */ +#define MLX5_RXP_MQ_CP_BASE (0x0500ul) +#define MLX5_RXP_MQ_CP_CAPABILITY_BASE (MLX5_RXP_MQ_CP_BASE + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_0 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 0 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_1 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 1 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_2 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_3 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 3 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_BASE (MLX5_RXP_MQ_CP_BASE + \ + 11 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C0 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 0 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C1 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 1 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C2 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C3 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 3 * MLX5_RXP_CSR_WIDTH) + +/* Royalty tracker / licensing related CSRs */ +#define MLX5_RXPL__CSR_IDENT (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 0 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__IDENTIFIER 0x4c505852 /* MLX5_RXPL_ */ +#define MLX5_RXPL__CSR_CAPABILITY (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__TYPE_MASK 0xFF +#define MLX5_RXPL__TYPE_NONE 0 +#define MLX5_RXPL__TYPE_MAXIM 1 +#define MLX5_RXPL__TYPE_XILINX_DNA 2 +#define MLX5_RXPL__CSR_STATUS (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 10 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__CSR_IDENT_0 (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 16 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__CSR_KEY_0 (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 24 * MLX5_RXP_CSR_WIDTH) + +#endif /* _MLX5_RXP_CSRS_H_ */ From patchwork Sun Jul 12 20:58:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73847 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CADE4A052A; Sun, 12 Jul 2020 23:00:50 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 663781D54D; Sun, 12 Jul 2020 22:59:57 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 534EF1D541 for ; Sun, 12 Jul 2020 22:59:56 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:51 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFG4025997; Sun, 12 Jul 2020 23:59:50 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler , Ray Kinsella , Neil Horman Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 12 Jul 2020 20:58:50 +0000 Message-Id: <1594587541-110442-10-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 09/20] common/mlx5: add write and read RXP registers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commits add the write and read RXP registers functionality. Signed-off-by: Ori Kam Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_devx_cmds.c | 78 +++++++++++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 10 ++++ drivers/common/mlx5/rte_common_mlx5_version.map | 2 + 3 files changed, 90 insertions(+) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index fe781c7..9094012 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1720,3 +1720,81 @@ struct mlx5_devx_obj * return 0; } +/** + * Write to RXP registers. + * + * @param ctx + * ibv device handle + * @param engine_id + * Chooses on which engine the register will be written.. + * @param addr + * Register address. + * @param data + * Data to be written to the register. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_regex_register_write(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t data) +{ + uint32_t out[MLX5_ST_SZ_DW(set_regexp_register_out)] = {0}; + uint32_t in[MLX5_ST_SZ_DW(set_regexp_register_in)] = {0}; + int ret; + + MLX5_SET(set_regexp_register_in, in, opcode, + MLX5_CMD_SET_REGEX_REGISTERS); + MLX5_SET(set_regexp_register_in, in, engine_id, engine_id); + MLX5_SET(set_regexp_register_in, in, register_address, addr); + MLX5_SET(set_regexp_register_in, in, register_data, data); + + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Set regexp register failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + + +/** + * Read from RXP registers + * + * @param ctx + * ibv device handle + * @param engine_id + * Chooses from which engine to read. + * @param addr + * Register address. + * @param data + * Output containing the pointer to the data.. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_regex_register_read(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t *data) +{ + uint32_t out[MLX5_ST_SZ_DW(query_regexp_register_out)] = {0}; + uint32_t in[MLX5_ST_SZ_DW(query_regexp_register_in)] = {0}; + int ret; + + MLX5_SET(query_regexp_register_in, in, opcode, + MLX5_CMD_QUERY_REGEX_REGISTERS); + MLX5_SET(query_regexp_register_in, in, engine_id, engine_id); + MLX5_SET(query_regexp_register_in, in, register_address, addr); + + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Query regexp register failed %d", ret); + rte_errno = errno; + return -errno; + } + *data = MLX5_GET(query_regexp_register_out, out, register_data); + return 0; +} diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 655e31f..a2a9045 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -374,6 +374,10 @@ int mlx5_devx_cmd_modify_qp_state(struct mlx5_devx_obj *qp, __rte_internal int mlx5_devx_cmd_modify_rqt(struct mlx5_devx_obj *rqt, struct mlx5_devx_rqt_attr *rqt_attr); +int mlx5_devx_regex_register_write(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t data); +int mlx5_devx_regex_register_read(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t *data); /** * Create virtio queue counters object DevX API. @@ -408,4 +412,10 @@ int mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj, __rte_internal int mlx5_devx_regex_database_program(void *ctx, uint8_t engine, uint32_t umem_id, uint64_t umem_offset); +__rte_internal +int mlx5_devx_regex_register_read(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t *data); +__rte_internal +int mlx5_devx_regex_register_write(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t data); #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */ diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map index 6054d39..138719d 100644 --- a/drivers/common/mlx5/rte_common_mlx5_version.map +++ b/drivers/common/mlx5/rte_common_mlx5_version.map @@ -38,6 +38,8 @@ INTERNAL { mlx5_devx_regex_database_program; mlx5_devx_regex_database_resume; mlx5_devx_regex_database_stop; + mlx5_devx_regex_register_read; + mlx5_devx_regex_register_write; mlx5_get_ifname_sysfs; mlx5_get_dbr; From patchwork Sun Jul 12 20:58:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73848 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8E1BDA052A; Sun, 12 Jul 2020 23:00:58 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AB5C21D561; Sun, 12 Jul 2020 22:59:58 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 594A21D55C for ; Sun, 12 Jul 2020 22:59:56 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:53 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFG5025997; Sun, 12 Jul 2020 23:59:53 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 12 Jul 2020 20:58:51 +0000 Message-Id: <1594587541-110442-11-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 10/20] regex/mlx5: add engine status check X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit checks the engine status. Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index d264ecd..c469a10 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -17,6 +17,7 @@ #include "mlx5_regex.h" #include "mlx5_regex_utils.h" +#include "mlx5_rxp_csrs.h" int mlx5_regex_logtype; @@ -49,6 +50,28 @@ mlx5_glue->free_device_list(ibv_list); return ibv_match; } +static int +mlx5_regex_engines_status(struct ibv_context *ctx, int num_engines) +{ + uint32_t fpga_ident = 0; + int err; + int i; + + for (i = 0; i < num_engines; i++) { + err = mlx5_devx_regex_register_read(ctx, i, + MLX5_RXP_CSR_IDENTIFIER, + &fpga_ident); + fpga_ident = (fpga_ident & (0x0000FFFF)); + if (err || fpga_ident != MLX5_RXP_IDENTIFER) { + DRV_LOG(ERR, "Failed setup RXP %d err %d database " + "memory 0x%x", i, err, fpga_ident); + if (!err) + err = EINVAL; + return err; + } + } + return 0; +} static void mlx5_regex_get_name(char *name, struct rte_pci_device *pci_dev __rte_unused) @@ -109,6 +132,11 @@ rte_errno = ENOTSUP; goto error; } + if (mlx5_regex_engines_status(ctx, 2)) { + DRV_LOG(ERR, "RegEx engine error."); + rte_errno = ENOMEM; + goto error; + } priv = rte_zmalloc("mlx5 regex device private", sizeof(*priv), RTE_CACHE_LINE_SIZE); if (!priv) { From patchwork Sun Jul 12 20:58:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73849 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 105F5A052A; Sun, 12 Jul 2020 23:01:09 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8CE3E1D56E; Sun, 12 Jul 2020 23:00:04 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 6A8D71D56E for ; Sun, 12 Jul 2020 23:00:00 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 12 Jul 2020 23:59:56 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFG6025997; Sun, 12 Jul 2020 23:59:56 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 12 Jul 2020 20:58:52 +0000 Message-Id: <1594587541-110442-12-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 11/20] regex/mlx5: add get info function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the get info function. Signed-off-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/meson.build | 1 + drivers/regex/mlx5/mlx5_regex.c | 5 ++++- drivers/regex/mlx5/mlx5_regex.h | 5 +++++ drivers/regex/mlx5/mlx5_rxp.c | 41 +++++++++++++++++++++++++++++++++++++++++ 5 files changed, 52 insertions(+), 1 deletion(-) create mode 100644 drivers/regex/mlx5/mlx5_rxp.c diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 3b99570..be23b5a 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -8,6 +8,7 @@ LIB = librte_pmd_mlx5_regex.a # Sources. SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_rxp.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index 327c0ad..d8d35c3 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -11,6 +11,7 @@ fmt_name = 'mlx5_regex' deps += ['common_mlx5', 'pci', 'bus_pci', 'eal', 'sched', 'regexdev'] sources = files( 'mlx5_regex.c', + 'mlx5_rxp.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index c469a10..2c4b7ce 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -21,7 +21,10 @@ int mlx5_regex_logtype; -static const struct rte_regexdev_ops mlx5_regexdev_ops = {}; +const struct rte_regexdev_ops mlx5_regexdev_ops = { + .dev_info_get = mlx5_regex_info_get, +}; + static struct ibv_device * mlx5_regex_get_ib_device_match(struct rte_pci_addr *addr) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 0ce1e4d..9d0fc16 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -11,4 +11,9 @@ struct mlx5_regex_priv { struct rte_pci_device *pci_dev; struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ }; + +/* mlx5_rxp.c */ +int mlx5_regex_info_get(struct rte_regexdev *dev, + struct rte_regexdev_info *info); + #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c new file mode 100644 index 0000000..12d55ed --- /dev/null +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include +#include +#include +#include +#include + +#include "mlx5_regex.h" + +#define MLX5_REGEX_MAX_MATCHES 255 +#define MLX5_REGEX_MAX_PAYLOAD_SIZE UINT16_MAX +#define MLX5_REGEX_MAX_RULES_PER_GROUP UINT16_MAX +#define MLX5_REGEX_MAX_GROUPS UINT16_MAX + +/** + * DPDK callback for reading device info. + * + * @param dev + * Pointer to RegEx device structure. + * @param[out] info + * Pointer to the regexdev info structure to be filled with the contextual + * information of the device. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_regex_info_get(struct rte_regexdev *dev __rte_unused, + struct rte_regexdev_info *info) +{ + info->max_matches = MLX5_REGEX_MAX_MATCHES; + info->max_payload_size = MLX5_REGEX_MAX_PAYLOAD_SIZE; + info->max_rules_per_group = MLX5_REGEX_MAX_RULES_PER_GROUP; + info->max_groups = MLX5_REGEX_MAX_GROUPS; + info->regexdev_capa = RTE_REGEXDEV_SUPP_PCRE_GREEDY_F; + info->rule_flags = 0; + return 0; +} From patchwork Sun Jul 12 20:58:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73850 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CB2AFA052A; Sun, 12 Jul 2020 23:01:20 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AFEFD1D57F; Sun, 12 Jul 2020 23:00:09 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 66BDB1D570 for ; Sun, 12 Jul 2020 23:00:05 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 13 Jul 2020 00:00:00 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFG7025997; Sun, 12 Jul 2020 23:59:59 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 12 Jul 2020 20:58:53 +0000 Message-Id: <1594587541-110442-13-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 12/20] regex/mlx5: add configure function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit implements the configure function. This function is responsible to configure the RegEx engine. Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 2 + drivers/regex/mlx5/mlx5_regex.h | 15 +++ drivers/regex/mlx5/mlx5_rxp.c | 279 +++++++++++++++++++++++++++++++++++++++- 3 files changed, 295 insertions(+), 1 deletion(-) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 2c4b7ce..94e4352 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -23,6 +23,7 @@ const struct rte_regexdev_ops mlx5_regexdev_ops = { .dev_info_get = mlx5_regex_info_get, + .dev_configure = mlx5_regex_configure, }; @@ -158,6 +159,7 @@ priv->regexdev->dev_ops = &mlx5_regexdev_ops; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; + priv->regexdev->state = RTE_REGEXDEV_READY; return 0; error: diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 9d0fc16..5238f24 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -5,15 +5,30 @@ #ifndef MLX5_REGEX_H #define MLX5_REGEX_H +struct mlx5_regex_sq { + uint32_t nb_desc; /* Number of desc for this object. */ +}; + +struct mlx5_regex_qp { + uint32_t flags; /* QP user flags. */ + uint32_t nb_desc; /* Total number of desc for thsi qp. */ + struct mlx5_regex_sq *sqs; /* Pointer to sq array. */ +}; + struct mlx5_regex_priv { TAILQ_ENTRY(mlx5_regex_priv) next; struct ibv_context *ctx; /* Device context. */ struct rte_pci_device *pci_dev; struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ + uint16_t nb_queues; /* Number of queues. */ + struct mlx5_regex_qp *qps; /* Pointer to the qp array. */ + uint16_t nb_max_matches; /* Max number of matches. */ }; /* mlx5_rxp.c */ int mlx5_regex_info_get(struct rte_regexdev *dev, struct rte_regexdev_info *info); +int mlx5_regex_configure(struct rte_regexdev *dev, + const struct rte_regexdev_config *cfg); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c index 12d55ed..60a4640 100644 --- a/drivers/regex/mlx5/mlx5_rxp.c +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -2,13 +2,22 @@ * Copyright 2020 Mellanox Technologies, Ltd */ +#include + #include #include +#include #include #include #include +#include +#include +#include + #include "mlx5_regex.h" +#include "mlx5_regex_utils.h" +#include "mlx5_rxp_csrs.h" #define MLX5_REGEX_MAX_MATCHES 255 #define MLX5_REGEX_MAX_PAYLOAD_SIZE UINT16_MAX @@ -29,7 +38,7 @@ */ int mlx5_regex_info_get(struct rte_regexdev *dev __rte_unused, - struct rte_regexdev_info *info) + struct rte_regexdev_info *info) { info->max_matches = MLX5_REGEX_MAX_MATCHES; info->max_payload_size = MLX5_REGEX_MAX_PAYLOAD_SIZE; @@ -39,3 +48,271 @@ info->rule_flags = 0; return 0; } + +static int +rxp_poll_csr_for_value(struct ibv_context *ctx, uint32_t *value, + uint32_t address, uint32_t expected_value, + uint32_t expected_mask, uint32_t timeout_ms, uint8_t id) +{ + unsigned int i; + int ret; + + ret = -EBUSY; + for (i = 0; i < timeout_ms; i++) { + if (mlx5_devx_regex_register_read(ctx, id, address, value)) + return -1; + + if ((*value & expected_mask) == expected_value) { + ret = 0; + break; + } + rte_delay_us(1000); + } + return ret; +} + +/** + * Start the selected engine. + * + * @param ctx + * The IBV context. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +rxp_start_engine(struct ibv_context *ctx, uint8_t id) +{ + uint32_t ctrl; + int ret; + + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + ctrl |= MLX5_RXP_CSR_CTRL_GO; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + return ret; +} + +/** + * Stop the selected engine. + * + * @param ctx + * The IBV context. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +rxp_stop_engine(struct ibv_context *ctx, uint8_t id) +{ + uint32_t ctrl; + int ret; + + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + ctrl &= ~MLX5_RXP_CSR_CTRL_GO; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + return ret; +} + +static int +rxp_init_rtru(struct ibv_context *ctx, uint8_t id, uint32_t init_bits) +{ + uint32_t ctrl_value; + uint32_t poll_value; + uint32_t expected_value; + uint32_t expected_mask; + int ret = 0; + + /* Read the rtru ctrl CSR */ + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + &ctrl_value); + if (ret) + return -1; + /* Clear any previous init modes */ + ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_MASK); + if (ctrl_value & MLX5_RXP_RTRU_CSR_CTRL_INIT) { + ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + } + /* Set the init_mode bits in the rtru ctrl CSR */ + ctrl_value |= init_bits; + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + /* Need to sleep for a short period after pulsing the rtru init bit. */ + rte_delay_us(20000); + /* Poll the rtru status CSR until all the init done bits are set. */ + DRV_LOG(DEBUG, "waiting for RXP rule memory to complete init"); + /* Set the init bit in the rtru ctrl CSR. */ + ctrl_value |= MLX5_RXP_RTRU_CSR_CTRL_INIT; + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + /* Clear the init bit in the rtru ctrl CSR */ + ctrl_value &= ~MLX5_RXP_RTRU_CSR_CTRL_INIT; + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + /* Check that the following bits are set in the RTRU_CSR. */ + if (init_bits == MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2) { + /* Must be incremental mode */ + expected_value = MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + } else { + expected_value = MLX5_RXP_RTRU_CSR_STATUS_IM_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + } + expected_mask = expected_value; + ret = rxp_poll_csr_for_value(ctx, &poll_value, + MLX5_RXP_RTRU_CSR_STATUS, + expected_value, expected_mask, + MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT, id); + if (ret) + return ret; + DRV_LOG(DEBUG, "rule Memory initialise: 0x%08X", poll_value); + /* Clear the init bit in the rtru ctrl CSR */ + ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + return 0; +} + +/** + * Init the engine. + * + * @param ctx + * The IBV context. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +rxp_init(struct mlx5_regex_priv *priv, uint8_t id) +{ + uint32_t ctrl; + uint32_t reg; + struct ibv_context *ctx = priv->ctx; + int ret; + + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + if (ctrl & MLX5_RXP_CSR_CTRL_INIT) { + ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, + ctrl); + if (ret) + return ret; + } + ctrl |= MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + if (ret) + return ret; + ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + rte_delay_us(20000); + + ret = rxp_poll_csr_for_value(ctx, &ctrl, MLX5_RXP_CSR_STATUS, + MLX5_RXP_CSR_STATUS_INIT_DONE, + MLX5_RXP_CSR_STATUS_INIT_DONE, + MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT, id); + if (ret) + return ret; + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, + ctrl); + if (ret) + return ret; + rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); + ret = rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); + if (ret) + return ret; + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CAPABILITY_5, + ®); + if (ret) + return ret; + DRV_LOG(DEBUG, "max matches: %d, DDOS threshold: %d", reg >> 16, + reg & 0xffff); + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_MATCH, + priv->nb_max_matches); + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_LATENCY, 0); + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_PRI_THREAD, 0); + return ret; +} + +/** + * DPDK callback for reading device info. + * + * @param dev + * Pointer to RegEx device structure. + * @param[in] cfg + * Pointer to the regexdev device configuration structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_regex_configure(struct rte_regexdev *dev, + const struct rte_regexdev_config *cfg) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + int ret; + uint8_t id; + + priv->nb_queues = cfg->nb_queue_pairs; + priv->qps = rte_zmalloc(NULL, sizeof(struct mlx5_regex_qp) * + priv->nb_queues, 0); + if (!priv->nb_queues) { + DRV_LOG(ERR, "can't allocate qps memory"); + rte_errno = ENOMEM; + return -rte_errno; + } + priv->nb_max_matches = cfg->nb_max_matches; + for (id = 0; id < 2; id++) { + ret = rxp_stop_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "can't stop engine."); + rte_errno = ENODEV; + return -rte_errno; + } + ret = rxp_init(priv, id); + if (ret) { + DRV_LOG(ERR, "can't init engine."); + rte_errno = ENODEV; + return -rte_errno; + } + ret = mlx5_devx_regex_register_write(priv->ctx, id, + MLX5_RXP_CSR_MAX_MATCH, + priv->nb_max_matches); + if (ret) { + DRV_LOG(ERR, "can't update number of matches."); + rte_errno = ENODEV; + goto configure_error; + } + ret = rxp_start_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "can't start engine."); + rte_errno = ENODEV; + goto configure_error; + } + + } + return 0; +configure_error: + if (priv->qps) + rte_free(priv->qps); + return -rte_errno; +} From patchwork Sun Jul 12 20:58:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73851 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A266A052A; Sun, 12 Jul 2020 23:01:32 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 486261D511; Sun, 12 Jul 2020 23:00:28 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 965381D455 for ; Sun, 12 Jul 2020 23:00:26 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 13 Jul 2020 00:00:21 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFG8025997; Mon, 13 Jul 2020 00:00:20 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Francis Kelly Date: Sun, 12 Jul 2020 20:58:54 +0000 Message-Id: <1594587541-110442-14-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 13/20] regex/mlx5: add program rules support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Francis Kelly This commit introduce the ability to program rules to the RegEx engine. Signed-off-by: Francis Kelly --- drivers/regex/mlx5/mlx5_regex.c | 34 ++ drivers/regex/mlx5/mlx5_regex.h | 56 ++- drivers/regex/mlx5/mlx5_rxp.c | 1015 +++++++++++++++++++++++++++++++++++++-- drivers/regex/mlx5/mlx5_rxp.h | 138 ++++++ 4 files changed, 1191 insertions(+), 52 deletions(-) create mode 100644 drivers/regex/mlx5/mlx5_rxp.h diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 94e4352..d5b33ad 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -24,6 +24,7 @@ const struct rte_regexdev_ops mlx5_regexdev_ops = { .dev_info_get = mlx5_regex_info_get, .dev_configure = mlx5_regex_configure, + .dev_db_import = mlx5_regex_rules_db_import, }; @@ -149,6 +150,9 @@ goto error; } priv->ctx = ctx; + priv->nb_engines = 2; /* attr.regexp_num_of_engines */ + /* Default RXP programming mode to Shared. */ + priv->prog_mode = MLX5_RXP_SHARED_PROG_MODE; mlx5_regex_get_name(name, pci_dev); priv->regexdev = rte_regexdev_register(name); if (priv->regexdev == NULL) { @@ -156,6 +160,24 @@ rte_errno = rte_errno ? rte_errno : EINVAL; goto error; } + ret = mlx5_glue->devx_query_eqn(ctx, 0, &priv->eqn); + if (ret) { + DRV_LOG(ERR, "can't query event queue number."); + rte_errno = ENOMEM; + goto error; + } + priv->uar = mlx5_glue->devx_alloc_uar(ctx, 0); + if (!priv->uar) { + DRV_LOG(ERR, "can't allocate uar."); + rte_errno = ENOMEM; + goto error; + } + priv->pd = mlx5_glue->alloc_pd(ctx); + if (!priv->pd) { + DRV_LOG(ERR, "can't allocate pd."); + rte_errno = ENOMEM; + goto error; + } priv->regexdev->dev_ops = &mlx5_regexdev_ops; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; @@ -163,6 +185,12 @@ return 0; error: + if (priv->pd) + mlx5_glue->dealloc_pd(priv->pd); + if (priv->uar) + mlx5_glue->devx_free_uar(priv->uar); + if (priv->regexdev) + rte_regexdev_unregister(priv->regexdev); if (ctx) mlx5_glue->close_device(ctx); if (priv) @@ -194,6 +222,12 @@ return 0; priv = dev->data->dev_private; if (priv) { + if (priv->pd) + mlx5_glue->dealloc_pd(priv->pd); + if (priv->uar) + mlx5_glue->devx_free_uar(priv->uar); + if (priv->regexdev) + rte_regexdev_unregister(priv->regexdev); if (priv->ctx) mlx5_glue->close_device(priv->ctx); if (priv->regexdev) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 5238f24..a3a0e55 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -5,14 +5,53 @@ #ifndef MLX5_REGEX_H #define MLX5_REGEX_H +#ifdef PEDANTIC +#pragma GCC diagnostic ignored "-Wpedantic" +#endif +#include +#include +#ifdef PEDANTIC +#pragma GCC diagnostic error "-Wpedantic" +#endif + +#include + +#include "mlx5_rxp.h" + struct mlx5_regex_sq { - uint32_t nb_desc; /* Number of desc for this object. */ + uint16_t log_nb_desc; /* Log 2 number of desc for this object. */ + struct mlx5_devx_obj *obj; /* The SQ DevX object. */ + int64_t dbr_offset; /* Door bell record offset. */ + uint32_t dbr_umem; /* Door bell record umem id. */ + volatile struct mlx5_cqe *wqe; /* The SQ ring buffer. */ + struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ +}; + +struct mlx5_regex_cq { + uint32_t log_nb_desc; /* Log 2 number of desc for this object. */ + struct mlx5_devx_obj *obj; /* The CQ DevX object. */ + int64_t dbr_offset; /* Door bell record offset. */ + uint32_t dbr_umem; /* Door bell record umem id. */ + volatile struct mlx5_cqe *cqe; /* The CQ ring buffer. */ + struct mlx5dv_devx_umem *cqe_umem; /* CQ buffer umem. */ }; struct mlx5_regex_qp { uint32_t flags; /* QP user flags. */ - uint32_t nb_desc; /* Total number of desc for thsi qp. */ + uint16_t nb_desc; /* Total number of desc for this qp. */ struct mlx5_regex_sq *sqs; /* Pointer to sq array. */ + uint16_t nb_obj; /* Number of sq objects. */ + struct mlx5_regex_cq cq; /* CQ struct. */ +}; + +struct mlx5_regex_db { + void *ptr; /* Pointer to the db memory. */ + uint32_t len; /* The memory len. */ + bool active; /* Active flag. */ + uint8_t db_assigned_to_eng_num; + /**< To which engine the db is connected. */ + struct mlx5_regex_umem umem; + /**< The umem struct. */ }; struct mlx5_regex_priv { @@ -23,6 +62,14 @@ struct mlx5_regex_priv { uint16_t nb_queues; /* Number of queues. */ struct mlx5_regex_qp *qps; /* Pointer to the qp array. */ uint16_t nb_max_matches; /* Max number of matches. */ + enum mlx5_rxp_program_mode prog_mode; + struct mlx5_regex_db db[MLX5_RXP_MAX_ENGINES + + MLX5_RXP_EM_COUNT]; + uint32_t nb_engines; /* Number of RegEx engines. */ + struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ + uint32_t eqn; /* EQ number. */ + struct mlx5dv_devx_uar *uar; /* UAR object. */ + struct ibv_pd *pd; }; /* mlx5_rxp.c */ @@ -30,5 +77,8 @@ int mlx5_regex_info_get(struct rte_regexdev *dev, struct rte_regexdev_info *info); int mlx5_regex_configure(struct rte_regexdev *dev, const struct rte_regexdev_config *cfg); - +int mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, + const struct rte_regexdev_qp_conf *cfg); +int mlx5_regex_rules_db_import(struct rte_regexdev *dev, + const char *rule_db, uint32_t rule_db_len); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c index 60a4640..a6d4dbc 100644 --- a/drivers/regex/mlx5/mlx5_rxp.c +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -2,8 +2,6 @@ * Copyright 2020 Mellanox Technologies, Ltd */ -#include - #include #include #include @@ -14,15 +12,110 @@ #include #include #include +#include #include "mlx5_regex.h" #include "mlx5_regex_utils.h" #include "mlx5_rxp_csrs.h" +#include "mlx5_rxp.h" + +#define MLX5_REGEX_MAX_MATCHES MLX5_RXP_MAX_MATCHES +#define MLX5_REGEX_MAX_PAYLOAD_SIZE MLX5_RXP_MAX_JOB_LENGTH +#define MLX5_REGEX_MAX_RULES_PER_GROUP UINT32_MAX +#define MLX5_REGEX_MAX_GROUPS MLX5_RXP_MAX_SUBSETS + +/* Private Declarations */ +static int +rxp_poll_csr_for_value(struct ibv_context *ctx, uint32_t *value, + uint32_t address, uint32_t expected_value, + uint32_t expected_mask, uint32_t timeout_ms, uint8_t id); +static int +mlnx_set_database(struct mlx5_regex_priv *priv, uint8_t id, uint8_t db_to_use); +static int +mlnx_resume_database(struct mlx5_regex_priv *priv, uint8_t id); +static int +mlnx_update_database(struct mlx5_regex_priv *priv, uint8_t id); +static int +program_rxp_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint8_t id); +static int +rxp_init_eng(struct mlx5_regex_priv *priv, uint8_t id); +static int +write_private_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, + uint8_t id); +static int +write_shared_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint32_t count, + uint8_t db_to_program); +static int +rxp_db_setup(struct mlx5_regex_priv *priv); +static void +rxp_dump_csrs(struct ibv_context *ctx, uint8_t id); +static int +rxp_write_rules_via_cp(struct ibv_context *ctx, + struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id); +static int +rxp_flush_rules(struct ibv_context *ctx, struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id); +static int +rxp_start_engine(struct ibv_context *ctx, uint8_t id); +static int +rxp_stop_engine(struct ibv_context *ctx, uint8_t id); + +/** + * RXP Register display function + * + * @param ctx + * The IBV context. + * @param id + * The selected engine to read back registers. + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static void __rte_unused +rxp_dump_csrs(struct ibv_context *ctx __rte_unused, uint8_t id __rte_unused) +{ + uint32_t reg, i; -#define MLX5_REGEX_MAX_MATCHES 255 -#define MLX5_REGEX_MAX_PAYLOAD_SIZE UINT16_MAX -#define MLX5_REGEX_MAX_RULES_PER_GROUP UINT16_MAX -#define MLX5_REGEX_MAX_GROUPS UINT16_MAX + /* Main CSRs*/ + for (i = 0; i < MLX5_RXP_CSR_NUM_ENTRIES; i++) { + if (mlx5_devx_regex_register_read(ctx, id, + (MLX5_RXP_CSR_WIDTH * i) + + MLX5_RXP_CSR_BASE_ADDRESS, + ®)) { + DRV_LOG(ERR, "Failed to read Main CSRs Engine %d!", id); + return; + } + DRV_LOG(DEBUG, "RXP Main CSRs (Eng%d) register (%d): %08x", + id, i, reg); + } + /* RTRU CSRs*/ + for (i = 0; i < MLX5_RXP_CSR_NUM_ENTRIES; i++) { + if (mlx5_devx_regex_register_read(ctx, id, + (MLX5_RXP_CSR_WIDTH * i) + + MLX5_RXP_RTRU_CSR_BASE_ADDRESS, + ®)) { + DRV_LOG(ERR, "Failed to read RTRU CSRs Engine %d!", id); + return; + } + DRV_LOG(DEBUG, "RXP RTRU CSRs (Eng%d) register (%d): %08x", + id, i, reg); + } + /* STAT CSRs */ + for (i = 0; i < MLX5_RXP_CSR_NUM_ENTRIES; i++) { + if (mlx5_devx_regex_register_read(ctx, id, + (MLX5_RXP_CSR_WIDTH * i) + + MLX5_RXP_STATS_CSR_BASE_ADDRESS, + ®)) { + DRV_LOG(ERR, "Failed to read STAT CSRs Engine %d!", id); + return; + } + DRV_LOG(DEBUG, "RXP STAT CSRs (Eng%d) register (%d): %08x", + id, i, reg); + } +} /** * DPDK callback for reading device info. @@ -44,24 +137,168 @@ info->max_payload_size = MLX5_REGEX_MAX_PAYLOAD_SIZE; info->max_rules_per_group = MLX5_REGEX_MAX_RULES_PER_GROUP; info->max_groups = MLX5_REGEX_MAX_GROUPS; + info->max_queue_pairs = 1; info->regexdev_capa = RTE_REGEXDEV_SUPP_PCRE_GREEDY_F; info->rule_flags = 0; return 0; } +/** + * Actual writing of RXP instructions to RXP via CSRs. + * + * @param ctx + * The IBV context. + * @param rules + * RXP instructions to be written. + * @param count + * The number of instructions to be written. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +rxp_write_rules_via_cp(struct ibv_context *ctx, + struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id) +{ + int i, ret = 0; + uint32_t tmp; + + for (i = 0; i < count; i++) { + tmp = (uint32_t)rules[i].value; + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_RTRU_CSR_DATA_0, + tmp); + tmp = (uint32_t)(rules[i].value >> 32); + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_RTRU_CSR_DATA_0 + + MLX5_RXP_CSR_WIDTH, tmp); + tmp = rules[i].addr; + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_RTRU_CSR_ADDR, + tmp); + if (ret) { + DRV_LOG(ERR, "Failed to copy instructions to RXP."); + return -1; + } + } + DRV_LOG(DEBUG, "Written %d instructions", count); + return 0; +} + +/** + * Flushing the programmed instructions to RXP. + * + * @param ctx + * The IBV context. + * @param rules + * RXP instructions to be written. + * @param count + * The number of instructions to be written. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +rxp_flush_rules(struct ibv_context *ctx, struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id) +{ + uint32_t val, fifo_depth; + int ret; + + ret = rxp_write_rules_via_cp(ctx, rules, count, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to write rules via CSRs."); + return -1; + } + ret = mlx5_devx_regex_register_read(ctx, id, + MLX5_RXP_RTRU_CSR_CAPABILITY, + &fifo_depth); + if (ret) { + DRV_LOG(ERR, "CSR read failed!"); + return -1; + } + ret = rxp_poll_csr_for_value(ctx, &val, MLX5_RXP_RTRU_CSR_FIFO_STAT, + count, ~0, + MLX5_RXP_POLL_CSR_FOR_VALUE_TIMEOUT, id); + if (ret < 0) { + DRV_LOG(ERR, "Rules not rx by RXP: credit: %d, depth: %d", val, + fifo_depth); + return ret; + } + DRV_LOG(DEBUG, "RTRU FIFO depth: 0x%x", fifo_depth); + DRV_LOG(DEBUG, "Rules flush took %d cycles.", ret); + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + &val); + if (ret) { + DRV_LOG(ERR, "CSR read failed!"); + return -1; + } + val |= MLX5_RXP_RTRU_CSR_CTRL_GO; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + val); + ret = rxp_poll_csr_for_value(ctx, &val, MLX5_RXP_RTRU_CSR_STATUS, + MLX5_RXP_RTRU_CSR_STATUS_UPDATE_DONE, + MLX5_RXP_RTRU_CSR_STATUS_UPDATE_DONE, + MLX5_RXP_POLL_CSR_FOR_VALUE_TIMEOUT, id); + if (ret < 0) { + DRV_LOG(ERR, "Rules update timeout: 0x%08X", val); + return ret; + } + DRV_LOG(DEBUG, "Rules update took %d cycles", ret); + if (mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + &val)) { + DRV_LOG(ERR, "CSR read failed!"); + return -1; + } + val &= ~(MLX5_RXP_RTRU_CSR_CTRL_GO); + if (mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + val)) { + DRV_LOG(ERR, "CSR write write failed!"); + return -1; + } + + DRV_LOG(DEBUG, "RXP Flush rules finished."); + return 0; +} + +/** + * Poll RXP CSRs for expected value. + * + * @param ctx + * The IBV context. + * @param value + * The returned value from a CSR read. + * @param address + * The RXP CSR register to read from. + * @param expected_value + * The expected value to compare read value against. + * @param expected_mask + * A mask used to only compare particular bits of read value. + * @param timeout_ms + * The number of poll interations. + * @param id + * The selected engine to poll. + * + * @return + * 0 on success, a negative errno value otherwise. + */ static int rxp_poll_csr_for_value(struct ibv_context *ctx, uint32_t *value, uint32_t address, uint32_t expected_value, uint32_t expected_mask, uint32_t timeout_ms, uint8_t id) { unsigned int i; - int ret; + int ret = 0; ret = -EBUSY; for (i = 0; i < timeout_ms; i++) { if (mlx5_devx_regex_register_read(ctx, id, address, value)) return -1; - if ((*value & expected_mask) == expected_value) { ret = 0; break; @@ -92,6 +329,7 @@ if (ret) return ret; ctrl |= MLX5_RXP_CSR_CTRL_GO; + ctrl |= MLX5_RXP_CSR_CTRL_DISABLE_L2C; ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); return ret; } @@ -121,6 +359,18 @@ return ret; } +/** + * Initialise the selected RXP engine to specified init mode. + * + * @param ctx + * The IBV context. + * @param id + * The selected engine. + * @param init_bits + * The RXP initialisation modes. + * @return + * 0 on success, a negative errno value otherwise. + */ static int rxp_init_rtru(struct ibv_context *ctx, uint8_t id, uint32_t init_bits) { @@ -130,23 +380,23 @@ uint32_t expected_mask; int ret = 0; - /* Read the rtru ctrl CSR */ + /* Read the rtru ctrl CSR. */ ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, &ctrl_value); if (ret) return -1; - /* Clear any previous init modes */ + /* Clear any previous init modes. */ ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_MASK); if (ctrl_value & MLX5_RXP_RTRU_CSR_CTRL_INIT) { ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, ctrl_value); } - /* Set the init_mode bits in the rtru ctrl CSR */ + /* Set the init_mode bits in the rtru ctrl CSR. */ ctrl_value |= init_bits; mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, ctrl_value); - /* Need to sleep for a short period after pulsing the rtru init bit. */ + /* Need to sleep for a short period after pulsing the rtru init bit. */ rte_delay_us(20000); /* Poll the rtru status CSR until all the init done bits are set. */ DRV_LOG(DEBUG, "waiting for RXP rule memory to complete init"); @@ -162,11 +412,11 @@ if (init_bits == MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2) { /* Must be incremental mode */ expected_value = MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | - MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; } else { expected_value = MLX5_RXP_RTRU_CSR_STATUS_IM_INIT_DONE | - MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | - MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; } expected_mask = expected_value; ret = rxp_poll_csr_for_value(ctx, &poll_value, @@ -175,7 +425,7 @@ MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT, id); if (ret) return ret; - DRV_LOG(DEBUG, "rule Memory initialise: 0x%08X", poll_value); + DRV_LOG(DEBUG, "rule memory initialise: 0x%08X", poll_value); /* Clear the init bit in the rtru ctrl CSR */ ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, @@ -184,6 +434,257 @@ } /** + * Extract instructions from buffer into rules format ready for RXP programming. + * + * @param buf + * The buffer holding RXP instructions. + * @param len + * The actual length of the buffer. + * @param init_bits + * The RXP initialisation modes. + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +rxp_parse_rof(const char *buf, uint32_t len, + struct mlx5_rxp_ctl_rules_pgm **rules) +{ + static const char del[] = "\n\r"; + char *line; + char *tmp; + char *cur_pos; + uint32_t lines = 0; + uint32_t entries; + struct mlx5_rxp_rof_entry *curentry; + + tmp = rte_malloc("", len, 0); + if (!tmp) + return -ENOMEM; + memcpy(tmp, buf, len); + line = strtok(tmp, del); + while (line) { + if (line[0] != '#' && line[0] != '\0') + lines++; + line = strtok(NULL, del); + } + *rules = rte_malloc("", lines * sizeof(*curentry) + sizeof(**rules), 0); + if (!(*rules)) { + rte_free(tmp); + return -ENOMEM; + } + memset(*rules, 0, lines * sizeof(curentry) + sizeof(**rules)); + curentry = (*rules)->rules; + (*rules)->hdr.cmd = MLX5_RXP_CTL_RULES_PGM; + entries = 0; + memcpy(tmp, buf, len); + line = strtok(tmp, del); + while (line) { + if (line[0] == '#' || line[0] == '\0') { + line = strtok(NULL, del); + continue; + } + curentry->type = strtoul(line, &cur_pos, 10); + if (cur_pos == line || cur_pos[0] != ',') + goto parse_error; + cur_pos++; + curentry->addr = strtoul(cur_pos, &cur_pos, 16); + if (cur_pos[0] != ',') + goto parse_error; + cur_pos++; + curentry->value = strtoull(cur_pos, &cur_pos, 16); + if (cur_pos[0] != '\0' && cur_pos[0] != '\n') + goto parse_error; + curentry++; + entries++; + if (entries > lines) + goto parse_error; + line = strtok(NULL, del); + } + (*rules)->count = entries; + (*rules)->hdr.len = entries * sizeof(*curentry) + sizeof(**rules); + rte_free(tmp); + return 0; +parse_error: + rte_free(tmp); + if (*rules) + rte_free(*rules); + return -EINVAL; +} + +/** + * Provide db pointer to EM and ensure idle. + * + * @param priv + * Pointer to the private device data structure. + * @param id + * The RXP engine to setup. + * @param db_to_use + * Index to which db pointer allocated for RXP engine. + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +mlnx_set_database(struct mlx5_regex_priv *priv, uint8_t id, uint8_t db_to_use) +{ + int ret; + uint32_t umem_id; + + ret = mlx5_devx_regex_database_stop(priv->ctx, id); + if (ret < 0) { + DRV_LOG(ERR, "stop engine failed!"); + return ret; + } + umem_id = mlx5_os_get_umem_id(priv->db[db_to_use].umem.umem); + ret = mlx5_devx_regex_database_program(priv->ctx, id, umem_id, 0); + if (ret < 0) { + DRV_LOG(ERR, "program db failed!"); + return ret; + } + return 0; +} + +/** + * Resume engine. + * + * @param priv + * Pointer to the private device data structure. + * @param id + * The RXP engine to resume. + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +mlnx_resume_database(struct mlx5_regex_priv *priv, uint8_t id) +{ + mlx5_devx_regex_database_resume(priv->ctx, id); + return 0; +} + +/** + * Assign db memory for RXP programming. + * + * @param priv + * Pointer to the private device data structure. + * @param id + * The RXP engine to assign db. + * @return + * Index to allocated db buffer on success, a negative errno value otherwise. + */ +static int +mlnx_update_database(struct mlx5_regex_priv *priv, uint8_t id) +{ + unsigned int i; + uint8_t db_free = MLX5_RXP_DB_NOT_ASSIGNED; + uint8_t eng_assigned = MLX5_RXP_DB_NOT_ASSIGNED; + + /* Check which database rxp_eng is currently located if any? */ + for (i = 0; i < (priv->nb_engines + MLX5_RXP_EM_COUNT); + i++) { + if (priv->db[i].db_assigned_to_eng_num == id) { + eng_assigned = i; + break; + } + } + /* + * If private mode then, we can keep the same db ptr as RXP will be + * programming EM itself if necessary, however need to see if + * programmed yet. + */ + if ((priv->prog_mode == MLX5_RXP_PRIVATE_PROG_MODE) && + (eng_assigned != MLX5_RXP_DB_NOT_ASSIGNED)) + return eng_assigned; + /* Check for inactive db memory to use. */ + for (i = 0; i < (priv->nb_engines + MLX5_RXP_EM_COUNT); + i++) { + if (priv->db[i].active == true) + continue; /* Already in use, so skip db. */ + /* Set this db to active now as free to use. */ + priv->db[i].active = true; + /* Now unassign last db index in use by RXP Eng. */ + if (eng_assigned != MLX5_RXP_DB_NOT_ASSIGNED) { + priv->db[eng_assigned].active = false; + priv->db[eng_assigned].db_assigned_to_eng_num = + MLX5_RXP_DB_NOT_ASSIGNED; + + /* Set all DB memory to 0's before setting up DB. */ + memset(priv->db[i].ptr, 0x00, MLX5_MAX_DB_SIZE); + } + /* Now reassign new db index with RXP Engine. */ + priv->db[i].db_assigned_to_eng_num = id; + db_free = i; + break; + } + if (db_free == MLX5_RXP_DB_NOT_ASSIGNED) + return -1; + return db_free; +} + +/** + * Program RXP instruction db to RXP engine/s. + * + * @param priv + * Pointer to the private device data structure. + * @param rule_buf + * Pointer to buffer that will be holding RXP instructions. + * @param len + * The length of the rule buffer. + * @param id + * The selected engine. + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +program_rxp_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint8_t id) +{ + int ret, db_free; + uint32_t rule_cnt; + + rule_cnt = rules->count; + db_free = mlnx_update_database(priv, id); + if (db_free < 0) { + DRV_LOG(ERR, "Failed to setup db memory!"); + return db_free; + } + if (priv->prog_mode == MLX5_RXP_PRIVATE_PROG_MODE) { + /* Register early to ensure RXP writes to EM use valid addr. */ + ret = mlnx_set_database(priv, id, db_free); + if (ret < 0) { + DRV_LOG(ERR, "Failed to register db memory!"); + return ret; + } + } + ret = write_private_rules(priv, rules, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to write rules!"); + return ret; + } + if (priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE) { + /* Write external rules directly to EM. */ + rules->count = rule_cnt; + /* Now write external instructions to EM. */ + ret = write_shared_rules(priv, rules, rules->hdr.len, db_free); + if (ret < 0) { + DRV_LOG(ERR, "Failed to write EM rules!"); + return ret; + } + ret = mlnx_set_database(priv, id, db_free); + if (ret < 0) { + DRV_LOG(ERR, "Failed to register db memory!"); + return ret; + } + } + ret = mlnx_resume_database(priv, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to resume engine!"); + return ret; + } + DRV_LOG(DEBUG, "Programmed RXP Engine %d\n", id); + rules->count = rule_cnt; + return 0; +} + +/** * Init the engine. * * @param ctx @@ -192,10 +693,10 @@ * The selected engine. * * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. + * 0 on success, a negative errno value otherwise. */ static int -rxp_init(struct mlx5_regex_priv *priv, uint8_t id) +rxp_init_eng(struct mlx5_regex_priv *priv, uint8_t id) { uint32_t ctrl; uint32_t reg; @@ -219,7 +720,6 @@ ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); rte_delay_us(20000); - ret = rxp_poll_csr_for_value(ctx, &ctrl, MLX5_RXP_CSR_STATUS, MLX5_RXP_CSR_STATUS_INIT_DONE, MLX5_RXP_CSR_STATUS_INIT_DONE, @@ -234,7 +734,6 @@ ctrl); if (ret) return ret; - rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); ret = rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); if (ret) return ret; @@ -244,8 +743,16 @@ return ret; DRV_LOG(DEBUG, "max matches: %d, DDOS threshold: %d", reg >> 16, reg & 0xffff); - ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_MATCH, - priv->nb_max_matches); + if ((reg >> 16) >= priv->nb_max_matches) + ret = mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_MATCH, + priv->nb_max_matches); + else + ret = mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_MATCH, + (reg >> 16)); + ret |= mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_PREFIX, + (reg & 0xFFFF)); ret |= mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_LATENCY, 0); ret |= mlx5_devx_regex_register_write(ctx, id, @@ -254,6 +761,427 @@ } /** + * Private programming of RXP, here all private/internal instructions will + * be written to RXP private memories, and all external instructions will be + * written to EM via RXP and not host. + * + * @param priv + * Pointer to private regex structure. + * @param rules + * Pointer to actual rules buffer containing instructions. + * @param count + * The number of instructions to program. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +write_private_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, + uint8_t id) +{ + unsigned int pending; + uint32_t block, reg, val, rule_cnt, rule_offset, rtru_max_num_entries; + int ret = 1; + + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) + return -EINVAL; + if (rules->hdr.len == 0 || rules->hdr.cmd < MLX5_RXP_CTL_RULES_PGM || + rules->hdr.cmd > MLX5_RXP_CTL_RULES_PGM_INCR) + return -EINVAL; + /* For a non-incremental rules program, re-init the RXP. */ + if (rules->hdr.cmd == MLX5_RXP_CTL_RULES_PGM) { + ret = rxp_init_eng(priv, id); + if (ret < 0) + return ret; + } else if (rules->hdr.cmd == MLX5_RXP_CTL_RULES_PGM_INCR) { + /* Flush RXP L1 and L2 cache by using MODE_L1_L2. */ + ret = rxp_init_rtru(priv->ctx, id, + MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2); + if (ret < 0) + return ret; + } + if (rules->count == 0) + return -EINVAL; + /* Confirm the RXP is initialised. */ + if (mlx5_devx_regex_register_read(priv->ctx, id, + MLX5_RXP_CSR_STATUS, &val)) { + DRV_LOG(ERR, "Failed to read from RXP!"); + return -ENODEV; + } + if (!(val & MLX5_RXP_CSR_STATUS_INIT_DONE)) { + DRV_LOG(ERR, "RXP not initialised..."); + return -EBUSY; + } + /* Get the RTRU maximum number of entries allowed. */ + if (mlx5_devx_regex_register_read(priv->ctx, id, + MLX5_RXP_RTRU_CSR_CAPABILITY, &rtru_max_num_entries)) { + DRV_LOG(ERR, "Failed to read RTRU capability!"); + return -ENODEV; + } + rtru_max_num_entries = (rtru_max_num_entries & 0x00FF); + rule_cnt = 0; + pending = 0; + while (rules->count > 0) { + if ((rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_INST) || + (rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_IM) || + (rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_EM)) { + if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_EM) && + (priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE)) { + /* Skip EM rules programming. */ + if (pending > 0) { + /* Flush any rules that are pending. */ + rule_offset = (rule_cnt - pending); + ret = rxp_flush_rules(priv->ctx, + &rules->rules[rule_offset], + pending, id); + if (ret < 0) { + DRV_LOG(ERR, "Flushing rules."); + return -ENODEV; + } + pending = 0; + } + rule_cnt++; + } else { + pending++; + rule_cnt++; + /* + * If parsing the last rule, or if reached the + * maximum number of rules for this batch, then + * flush the rules batch to the RXP. + */ + if ((rules->count == 1) || + (pending == rtru_max_num_entries)) { + rule_offset = (rule_cnt - pending); + ret = rxp_flush_rules(priv->ctx, + &rules->rules[rule_offset], + pending, id); + if (ret < 0) { + DRV_LOG(ERR, "Flushing rules."); + return -ENODEV; + } + pending = 0; + } + } + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_EQ) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_GTE) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_LTE) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM_EX_EM)) { + if (pending) { + /* Flush rules before checking reg values. */ + rule_offset = (rule_cnt - pending); + ret = rxp_flush_rules(priv->ctx, + &rules->rules[rule_offset], + pending, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to flush rules."); + return -ENODEV; + } + } + block = (rules->rules[rule_cnt].addr >> 16) & 0xFFFF; + if (block == 0) + reg = MLX5_RXP_CSR_BASE_ADDRESS; + else if (block == 1) + reg = MLX5_RXP_RTRU_CSR_BASE_ADDRESS; + else { + DRV_LOG(ERR, "Invalid ROF register 0x%08X!", + rules->rules[rule_cnt].addr); + return -EINVAL; + } + reg += (rules->rules[rule_cnt].addr & 0xFFFF) * + MLX5_RXP_CSR_WIDTH; + ret = mlx5_devx_regex_register_read(priv->ctx, id, + reg, &val); + if (ret) { + DRV_LOG(ERR, "RXP CSR read failed!"); + return ret; + } + if ((priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE) && + ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM_EX_EM) && + (val != rules->rules[rule_cnt].value))) { + DRV_LOG(ERR, "Unexpected value for register:"); + DRV_LOG(ERR, "reg %x" PRIu32 " got %x" PRIu32, + rules->rules[rule_cnt].addr, val); + DRV_LOG(ERR, "expected %" PRIx64 ".", + rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((priv->prog_mode == + MLX5_RXP_PRIVATE_PROG_MODE) && + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM) && + (val != rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value for register:"); + DRV_LOG(ERR, "reg %x" PRIu32 " got %x" PRIu32, + rules->rules[rule_cnt].addr, val); + DRV_LOG(ERR, "expected %" PRIx64 ".", + rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_EQ) && + (val != rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value for register:"); + DRV_LOG(ERR, "reg %x" PRIu32 " got %x" PRIu32, + rules->rules[rule_cnt].addr, val); + DRV_LOG(ERR, "expected %" PRIx64 ".", + rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_GTE) && + (val < rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value reg 0x%08X,", + rules->rules[rule_cnt].addr); + DRV_LOG(ERR, "got %X, expected >= %" PRIx64 ".", + val, rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_LTE) && + (val > rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value reg 0x%08X,", + rules->rules[rule_cnt].addr); + DRV_LOG(ERR, "got %08X expected <= %" PRIx64, + val, rules->rules[rule_cnt].value); + return -EINVAL; + } + rule_cnt++; + pending = 0; + } else { + DRV_LOG(ERR, "Error: Invalid rule type %d!", + rules->rules[rule_cnt].type); + return -EINVAL; + } + rules->count--; + } + return ret; +} + +/** + * Shared memory programming mode, here all external db instructions are written + * to EM via the host. + * + * @param priv + * Pointer to private regex structure. + * @param rules + * Pointer to actual rules buffer containing instructions. + * @param count + * The number of instructions to program. + * @param db_to_program + * The selected db memory to write too. + * + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +write_shared_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint32_t count, + uint8_t db_to_program) +{ + uint32_t rule_cnt, rof_rule_addr; + uint64_t tmp_write_swap[4]; + + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) + return -EINVAL; + if ((rules->count == 0) || (count == 0)) + return -EINVAL; + rule_cnt = 0; + /* + * Note the following section of code carries out a 32byte swap of + * instruction to coincide with HW 32byte swap. This may need removed + * in new variants of this programming function! + */ + while (rule_cnt < rules->count) { + if ((rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_EM) && + (priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE)) { + /* + * Note there are always blocks of 8 instructions for + * 7's written sequentially. However there is no + * guarantee that all blocks are sequential! + */ + if (count >= (rule_cnt + MLX5_RXP_INST_BLOCK_SIZE)) { + /* + * Ensure memory write not exceeding boundary + * Check essential to ensure 0x10000 offset + * accounted for! + */ + if ((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + ((rules->rules[rule_cnt + 7].addr << + MLX5_RXP_INST_OFFSET))) >= + ((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + MLX5_MAX_DB_SIZE))) { + DRV_LOG(ERR, "DB exceeded memory!"); + return -ENODEV; + } + /* + * Rule address Offset to align with RXP + * external instruction offset. + */ + rof_rule_addr = (rules->rules[rule_cnt].addr << + MLX5_RXP_INST_OFFSET); + /* 32 byte instruction swap (sw work around)! */ + tmp_write_swap[0] = le64toh( + rules->rules[(rule_cnt + 4)].value); + tmp_write_swap[1] = le64toh( + rules->rules[(rule_cnt + 5)].value); + tmp_write_swap[2] = le64toh( + rules->rules[(rule_cnt + 6)].value); + tmp_write_swap[3] = le64toh( + rules->rules[(rule_cnt + 7)].value); + /* Write only 4 of the 8 instructions. */ + memcpy((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + rof_rule_addr), &tmp_write_swap, + (sizeof(uint64_t) * 4)); + /* Write 1st 4 rules of block after last 4. */ + rof_rule_addr = (rules->rules[ + (rule_cnt + 4)].addr << + MLX5_RXP_INST_OFFSET); + tmp_write_swap[0] = le64toh( + rules->rules[(rule_cnt + 0)].value); + tmp_write_swap[1] = le64toh( + rules->rules[(rule_cnt + 1)].value); + tmp_write_swap[2] = le64toh( + rules->rules[(rule_cnt + 2)].value); + tmp_write_swap[3] = le64toh( + rules->rules[(rule_cnt + 3)].value); + memcpy((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + rof_rule_addr), &tmp_write_swap, + (sizeof(uint64_t) * 4)); + } else + return -1; + /* Fast forward as already handled block of 8. */ + rule_cnt += MLX5_RXP_INST_BLOCK_SIZE; + } else + rule_cnt++; /* Must be something other than EM rule. */ + } + return 0; +} + +/** + * RXP initial db setup function for EM instructions + * + * @param priv + * Pointer to private RegEx structure. + * + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +rxp_db_setup(struct mlx5_regex_priv *priv) +{ + int ret; + uint8_t i; + + /* Setup database memories for both RXP engines + reprogram memory. */ + for (i = 0; i < (priv->nb_engines + MLX5_RXP_EM_COUNT); i++) { + priv->db[i].ptr = rte_malloc("", MLX5_MAX_DB_SIZE, 0); + if (!priv->db[i].ptr) { + DRV_LOG(ERR, "Failed to alloc db memory!"); + ret = ENODEV; + goto tidyup_error; + } + /* Register the memory. */ + priv->db[i].umem.umem = mlx5_glue->devx_umem_reg(priv->ctx, + priv->db[i].ptr, + MLX5_MAX_DB_SIZE, 7); + if (!priv->db[i].umem.umem) { + DRV_LOG(ERR, "Failed to register memory!"); + ret = ENODEV; + goto tidyup_error; + } + /* Ensure set all DB memory to 0's before setting up DB. */ + memset(priv->db[i].ptr, 0x00, MLX5_MAX_DB_SIZE); + /* No data currently in database. */ + priv->db[i].len = 0; + priv->db[i].active = false; + priv->db[i].db_assigned_to_eng_num = MLX5_RXP_DB_NOT_ASSIGNED; + } + return 0; +tidyup_error: + for (i = 0; i < (priv->nb_engines + MLX5_RXP_EM_COUNT); i++) { + if (priv->db[i].ptr) + rte_free(priv->db[i].ptr); + if (priv->db[i].umem.umem) + mlx5_glue->devx_umem_dereg(priv->db[i].umem.umem); + } + return -ret; +} + +/** + * DPDK callback for loading RXP rules. + * + * @param dev + * Pointer to RegEx device structure. + * @param[in] cfg + * Pointer to the regexdev device configuration structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_regex_rules_db_import(struct rte_regexdev *dev, + const char *rule_db, uint32_t rule_db_len) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_rxp_ctl_rules_pgm *rules = NULL; + uint8_t id; + int ret; + + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) { + DRV_LOG(ERR, "RXP programming mode not set!"); + return -1; + } + if (rule_db == NULL) { + DRV_LOG(ERR, "Database empty!"); + return -ENODEV; + } + if (rule_db_len == 0) + return -EINVAL; + ret = rxp_parse_rof(rule_db, rule_db_len, &rules); + if (ret) { + DRV_LOG(ERR, "Can't parse ROF file."); + return ret; + } + /* Need to ensure RXP not busy before stop! */ + for (id = 0; id < priv->nb_engines; id++) { + ret = rxp_stop_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "Can't stop engine."); + ret = -ENODEV; + goto tidyup_error; + } + ret = program_rxp_rules(priv, rules, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to program rxp rules."); + ret = -ENODEV; + goto tidyup_error; + } + ret = rxp_start_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "Can't start engine."); + ret = -ENODEV; + goto tidyup_error; + } + } + rte_free(rules); + return 0; +tidyup_error: + rte_free(rules); + return ret; +} + +/** * DPDK callback for reading device info. * * @param dev @@ -270,9 +1198,11 @@ { struct mlx5_regex_priv *priv = dev->data->dev_private; int ret; - uint8_t id; + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) + return -1; priv->nb_queues = cfg->nb_queue_pairs; + dev->data->dev_conf.nb_queue_pairs = priv->nb_queues; priv->qps = rte_zmalloc(NULL, sizeof(struct mlx5_regex_qp) * priv->nb_queues, 0); if (!priv->nb_queues) { @@ -281,35 +1211,22 @@ return -rte_errno; } priv->nb_max_matches = cfg->nb_max_matches; - for (id = 0; id < 2; id++) { - ret = rxp_stop_engine(priv->ctx, id); - if (ret) { - DRV_LOG(ERR, "can't stop engine."); - rte_errno = ENODEV; - return -rte_errno; - } - ret = rxp_init(priv, id); - if (ret) { - DRV_LOG(ERR, "can't init engine."); - rte_errno = ENODEV; - return -rte_errno; - } - ret = mlx5_devx_regex_register_write(priv->ctx, id, - MLX5_RXP_CSR_MAX_MATCH, - priv->nb_max_matches); - if (ret) { - DRV_LOG(ERR, "can't update number of matches."); - rte_errno = ENODEV; - goto configure_error; - } - ret = rxp_start_engine(priv->ctx, id); - if (ret) { - DRV_LOG(ERR, "can't start engine."); + /* Setup rxp db memories. */ + if (rxp_db_setup(priv)) { + DRV_LOG(ERR, "Failed to setup RXP db memory"); + rte_errno = ENOMEM; + return -rte_errno; + } + if (cfg->rule_db != NULL) { + ret = mlx5_regex_rules_db_import(dev, cfg->rule_db, + cfg->rule_db_len); + if (ret < 0) { + DRV_LOG(ERR, "Failed to program rxp rules."); rte_errno = ENODEV; goto configure_error; } - - } + } else + DRV_LOG(DEBUG, "Regex config without rules programming!"); return 0; configure_error: if (priv->qps) diff --git a/drivers/regex/mlx5/mlx5_rxp.h b/drivers/regex/mlx5/mlx5_rxp.h new file mode 100644 index 0000000..9686e24 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_rxp.h @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#ifndef RTE_PMD_MLX5_REGEX_RXP_H_ +#define RTE_PMD_MLX5_REGEX_RXP_H_ + +#define MLX5_RXP_MAX_JOB_LENGTH 16384 +#define MLX5_RXP_MAX_SUBSETS 4095 +#define MLX5_RXP_CSR_NUM_ENTRIES 31 + +#define MLX5_RXP_CTRL_TYPE_MASK 7 +#define MLX5_RXP_CTRL_TYPE_JOB_DESCRIPTOR 0 +#define MLX5_RXP_CTRL_TYPE_RESPONSE_DESCRIPTOR 1 +#define MLX5_RXP_CTRL_TYPE_MEMORY_WRITE 4 +#define MLX5_RXP_CSR_CTRL_DISABLE_L2C (1 << 7) + +#define MLX5_RXP_CTRL_JOB_DESC_SOF 0x0010 +#define MLX5_RXP_CTRL_JOB_DESC_EOF 0x0020 +#define MLX5_RXP_CTRL_JOB_DESC_HPM_ENABLE 0x0100 +#define MLX5_RXP_CTRL_JOB_DESC_ANYMATCH_ENABLE 0x0200 +#define MLX5_RXP_CTRL_JOB_DESC_FLAGS (MLX5_RXP_CTRL_JOB_DESC_SOF | \ + MLX5_RXP_CTRL_JOB_DESC_EOF | \ + MLX5_RXP_CTRL_JOB_DESC_HPM_ENABLE | \ + MLX5_RXP_CTRL_JOB_DESC_ANYMATCH_ENABLE) + +#define MLX5_RXP_CTRL_VALID 0x8000 + +#define MLX5_RXP_RESP_STATUS_MAX_PRI_THREADS (1 << 3) +#define MLX5_RXP_RESP_STATUS_MAX_SEC_THREADS (1 << 4) +#define MLX5_RXP_RESP_STATUS_MAX_LATENCY (1 << 5) +#define MLX5_RXP_RESP_STATUS_MAX_MATCH (1 << 6) +#define MLX5_RXP_RESP_STATUS_MAX_PREFIX (1 << 7) +#define MLX5_RXP_RESP_STATUS_HPM (1 << 8) +#define MLX5_RXP_RESP_STATUS_ANYMATCH (1 << 9) +#define MLX5_RXP_RESP_STATUS_PMI_SOJ (1 << 13) +#define MLX5_RXP_RESP_STATUS_PMI_EOJ (1 << 14) + +/* This describes the header the RXP expects for any search data. */ +struct mlx5_rxp_job_desc { + uint32_t job_id; + uint16_t ctrl; + uint16_t len; + uint16_t subset[4]; +} __rte_packed; + +struct mlx5_rxp_response_desc { + uint32_t job_id; + uint16_t status; + uint8_t detected_match_count; + uint8_t match_count; + uint16_t primary_thread_count; + uint16_t instruction_count; + uint16_t latency_count; + uint16_t pmi_min_byte_ptr; +} __rte_packed; + +struct mlx5_rxp_match_tuple { + uint32_t rule_id; + uint16_t start_ptr; + uint16_t length; +} __rte_packed; + +struct mlx5_rxp_response { + struct mlx5_rxp_response_desc header; + struct mlx5_rxp_match_tuple matches[0]; +}; + +#define MLX5_RXP_MAX_MATCHES 254 + +#define MLX5_RXP_CTL_RULES_PGM 1 +#define MLX5_RXP_CTL_RULES_PGM_INCR 2 + +#define MLX5_RXP_ROF_ENTRY_INST 0 +#define MLX5_RXP_ROF_ENTRY_EQ 1 +#define MLX5_RXP_ROF_ENTRY_GTE 2 +#define MLX5_RXP_ROF_ENTRY_LTE 3 +#define MLX5_RXP_ROF_ENTRY_CHECKSUM 4 +#define MLX5_RXP_ROF_ENTRY_CHECKSUM_EX_EM 5 +#define MLX5_RXP_ROF_ENTRY_IM 6 +#define MLX5_RXP_ROF_ENTRY_EM 7 +#define MLX5_RXP_ROF_ENTRY_TYPE_MAX 7 + +#define MLX5_RXP_INST_OFFSET 3 +#define MLX5_RXP_INST_BLOCK_SIZE 8 +#define MLX5_MAX_SIZE_RES_DES (sizeof(struct mlx5_rxp_response_desc)) +#define MLX5_MAX_DB_SIZE (1u << 27u) +#define MLX5_MAX_SIZE_MATCH_RESP (254 * sizeof(struct mlx5_rxp_match_tuple)) +#define MLX5_RXP_SQ_NOT_BUSY false +#define MLX5_RXP_SQ_BUSY true + + +struct mlx5_rxp_ctl_hdr { + uint16_t cmd; + uint32_t len; +}; + +struct mlx5_rxp_rof_entry { + uint8_t type; + uint32_t addr; + uint64_t value; +}; + +struct mlx5_rxp_rof { + uint32_t rof_version; + char *timestamp; + char *rxp_compiler_version; + uint32_t rof_revision; + uint32_t number_of_entries; + struct mlx5_rxp_rof_entry *rof_entries; +}; + +struct mlx5_rxp_ctl_rules_pgm { + struct mlx5_rxp_ctl_hdr hdr; + uint32_t count; + struct mlx5_rxp_rof_entry rules[0]; +} __rte_packed; + +/* RXP programming mode setting. */ +enum mlx5_rxp_program_mode { + MLX5_RXP_MODE_NOT_DEFINED = 0, + MLX5_RXP_SHARED_PROG_MODE, + MLX5_RXP_PRIVATE_PROG_MODE, +}; + +#define MLX5_RXP_POLL_CSR_FOR_VALUE_TIMEOUT 3000 /* Poll timeout in ms. */ +#define MLX5_RXP_INITIALIZATION_TIMEOUT 60000 /* Initialize timeout in ms. */ +#define MLX5_RXP_MAX_ENGINES 2u /* Number of RXP engines. */ +#define MLX5_RXP_EM_COUNT 1u /* Extra External Memories to use. */ +#define MLX5_RXP_DB_NOT_ASSIGNED 0xFF + +struct mlx5_regex_umem { + struct mlx5dv_devx_umem *umem; + uint32_t id; + uint64_t offset; +}; + +#endif /* RTE_PMD_MLX5_REGEX_RXP_H_ */ From patchwork Sun Jul 12 20:58:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73852 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B3163A052A; Sun, 12 Jul 2020 23:01:43 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A2F641D560; Sun, 12 Jul 2020 23:00:29 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id CBE731D50C for ; Sun, 12 Jul 2020 23:00:26 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 13 Jul 2020 00:00:23 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFG9025997; Mon, 13 Jul 2020 00:00:22 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 12 Jul 2020 20:58:55 +0000 Message-Id: <1594587541-110442-15-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 14/20] regex/mlx5: add completion queue creation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the creation of CQ Signed-off-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/meson.build | 1 + drivers/regex/mlx5/mlx5_regex.c | 1 + drivers/regex/mlx5/mlx5_regex.h | 4 +- drivers/regex/mlx5/mlx5_regex_control.c | 195 ++++++++++++++++++++++++++++++++ drivers/regex/mlx5/mlx5_rxp.c | 1 + 6 files changed, 201 insertions(+), 2 deletions(-) create mode 100644 drivers/regex/mlx5/mlx5_regex_control.c diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index be23b5a..3d3fc5d 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -9,6 +9,7 @@ LIB = librte_pmd_mlx5_regex.a # Sources. SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_rxp.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_control.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index d8d35c3..c7406fe 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -12,6 +12,7 @@ deps += ['common_mlx5', 'pci', 'bus_pci', 'eal', 'sched', 'regexdev'] sources = files( 'mlx5_regex.c', 'mlx5_rxp.c', + 'mlx5_regex_control.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index d5b33ad..f06f817 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -25,6 +25,7 @@ .dev_info_get = mlx5_regex_info_get, .dev_configure = mlx5_regex_configure, .dev_db_import = mlx5_regex_rules_db_import, + .dev_qp_setup = mlx5_regex_qp_setup, }; diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index a3a0e55..515c9f4 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -23,7 +23,7 @@ struct mlx5_regex_sq { struct mlx5_devx_obj *obj; /* The SQ DevX object. */ int64_t dbr_offset; /* Door bell record offset. */ uint32_t dbr_umem; /* Door bell record umem id. */ - volatile struct mlx5_cqe *wqe; /* The SQ ring buffer. */ + uint8_t *wqe; /* The SQ ring buffer. */ struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ }; @@ -66,10 +66,10 @@ struct mlx5_regex_priv { struct mlx5_regex_db db[MLX5_RXP_MAX_ENGINES + MLX5_RXP_EM_COUNT]; uint32_t nb_engines; /* Number of RegEx engines. */ - struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ uint32_t eqn; /* EQ number. */ struct mlx5dv_devx_uar *uar; /* UAR object. */ struct ibv_pd *pd; + struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ }; /* mlx5_rxp.c */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c new file mode 100644 index 0000000..577965f --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -0,0 +1,195 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "mlx5_regex.h" +#include "mlx5_regex_utils.h" +#include "mlx5_rxp_csrs.h" +#include "mlx5_rxp.h" + +#define MLX5_REGEX_NUM_WQE_PER_PAGE (4096/64) + +/** + * Returns the number of qp obj to be created. + * + * @param nb_desc + * The number of descriptors for the queue. + * + * @return + * The number of obj to be created. + */ +static uint16_t +regex_ctrl_get_nb_obj(uint16_t nb_desc) +{ + return ((nb_desc / MLX5_REGEX_NUM_WQE_PER_PAGE) + + !!(nb_desc % MLX5_REGEX_NUM_WQE_PER_PAGE)); +} + +/** + * destroy CQ. + * + * @param priv + * Pointer to the priv object. + * @param cp + * Pointer to the CQ to be destroyed. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_destroy_cq(struct mlx5_regex_priv *priv, struct mlx5_regex_cq *cq) +{ + if (cq->cqe_umem) { + mlx5_glue->devx_umem_dereg(cq->cqe_umem); + cq->cqe_umem = NULL; + } + if (cq->cqe) { + rte_free((void *)(uintptr_t)cq->cqe); + cq->cqe = NULL; + } + if (cq->dbr_offset) { + mlx5_release_dbr(&priv->dbrpgs, cq->dbr_umem, cq->dbr_offset); + cq->dbr_offset = -1; + } + if (cq->obj) { + mlx5_devx_cmd_destroy(cq->obj); + cq->obj = NULL; + } + return 0; +} + +/** + * create the CQ object. + * + * @param priv + * Pointer to the priv object. + * @param cp + * Pointer to the CQ to be created. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_create_cq(struct mlx5_regex_priv *priv, struct mlx5_regex_cq *cq) +{ + struct mlx5_devx_cq_attr attr = { + .q_umem_valid = 1, + .db_umem_valid = 1, + .eqn = priv->eqn, + }; + struct mlx5_devx_dbr_page *dbr_page = NULL; + void *buf = NULL; + size_t pgsize = sysconf(_SC_PAGESIZE); + uint32_t cq_size = 1 << cq->log_nb_desc; + uint32_t i; + + cq->dbr_offset = mlx5_get_dbr(priv->ctx, &priv->dbrpgs, &dbr_page); + if (cq->dbr_offset < 0) { + DRV_LOG(ERR, "Can't allocate cq door bell record."); + rte_errno = ENOMEM; + goto error; + } + cq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); + buf = rte_calloc(NULL, 1, sizeof(struct mlx5_cqe) * cq_size, 4096); + if (!buf) { + DRV_LOG(ERR, "Can't allocate cqe buffer."); + rte_errno = ENOMEM; + goto error; + } + cq->cqe = buf; + for (i = 0; i < cq_size; i++) + cq->cqe[i].op_own = 0xff; + cq->cqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, + sizeof(struct mlx5_cqe) * + cq_size, 7); + if (!cq->cqe_umem) { + DRV_LOG(ERR, "Can't register cqe mem."); + rte_errno = ENOMEM; + goto error; + } + attr.db_umem_offset = cq->dbr_offset; + attr.db_umem_id = cq->dbr_umem; + attr.q_umem_id = mlx5_os_get_umem_id(cq->cqe_umem); + attr.log_cq_size = cq->log_nb_desc; + attr.uar_page_id = priv->uar->page_id; + attr.log_page_size = rte_log2_u32(pgsize); + cq->obj = mlx5_devx_cmd_create_cq(priv->ctx, &attr); + if (!cq->obj) { + DRV_LOG(ERR, "Can't create cq object."); + rte_errno = ENOMEM; + goto error; + } + return 0; +error: + if (cq->cqe_umem) + mlx5_glue->devx_umem_dereg(cq->cqe_umem); + if (buf) + rte_free(buf); + if (cq->dbr_offset) + mlx5_release_dbr(&priv->dbrpgs, cq->dbr_umem, cq->dbr_offset); + return -rte_errno; +} + +/** + * Setup the qp. + * + * @param dev + * Pointer to RegEx dev structure. + * @param qp_ind + * The queue index to setup. + * @param cfg + * The queue requested configuration. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, + const struct rte_regexdev_qp_conf *cfg) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_regex_qp *qp; + int ret; + + qp = &priv->qps[qp_ind]; + qp->flags = cfg->qp_conf_flags; + qp->cq.log_nb_desc = rte_log2_u32(cfg->nb_desc); + qp->nb_desc = 1 << qp->cq.log_nb_desc; + if (qp->flags & RTE_REGEX_QUEUE_PAIR_CFG_OOS_F) + qp->nb_obj = regex_ctrl_get_nb_obj(qp->nb_desc); + else + qp->nb_obj = 1; + qp->sqs = rte_malloc(NULL, + qp->nb_obj * sizeof(struct mlx5_regex_sq), 64); + if (!qp->sqs) { + DRV_LOG(ERR, "Can't allocate sq array memory."); + rte_errno = ENOMEM; + return -rte_errno; + } + ret = regex_ctrl_create_cq(priv, &qp->cq); + if (ret) { + DRV_LOG(ERR, "Can't create cq."); + goto error; + } + return 0; + +error: + regex_ctrl_destroy_cq(priv, &qp->cq); + return -rte_errno; + +} diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c index a6d4dbc..694f852 100644 --- a/drivers/regex/mlx5/mlx5_rxp.c +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -140,6 +140,7 @@ info->max_queue_pairs = 1; info->regexdev_capa = RTE_REGEXDEV_SUPP_PCRE_GREEDY_F; info->rule_flags = 0; + info->max_queue_pairs = 10; return 0; } From patchwork Sun Jul 12 20:58:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73853 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7A8FBA052A; Sun, 12 Jul 2020 23:01:56 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A37DA1D58B; Sun, 12 Jul 2020 23:00:34 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id E5EE31D58A for ; Sun, 12 Jul 2020 23:00:30 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 13 Jul 2020 00:00:26 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFGA025997; Mon, 13 Jul 2020 00:00:26 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 12 Jul 2020 20:58:56 +0000 Message-Id: <1594587541-110442-16-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 15/20] regex/mlx5: add send queue support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit introduce the SQ creation. The SQ is used for enqueuing a job. In order to support out of order matches, we create number os SQ per one applicaiton QP. Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.h | 2 + drivers/regex/mlx5/mlx5_regex_control.c | 168 ++++++++++++++++++++++++++++++++ 2 files changed, 170 insertions(+) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 515c9f4..12033e8 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -25,6 +25,7 @@ struct mlx5_regex_sq { uint32_t dbr_umem; /* Door bell record umem id. */ uint8_t *wqe; /* The SQ ring buffer. */ struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ + uint32_t *dbr; }; struct mlx5_regex_cq { @@ -34,6 +35,7 @@ struct mlx5_regex_cq { uint32_t dbr_umem; /* Door bell record umem id. */ volatile struct mlx5_cqe *cqe; /* The CQ ring buffer. */ struct mlx5dv_devx_umem *cqe_umem; /* CQ buffer umem. */ + uint32_t *dbr; }; struct mlx5_regex_qp { diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index 577965f..d378f48 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -105,6 +105,9 @@ goto error; } cq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); + cq->dbr = (uint32_t *)((uintptr_t)dbr_page->dbrs + + (uintptr_t)cq->dbr_offset); + buf = rte_calloc(NULL, 1, sizeof(struct mlx5_cqe) * cq_size, 4096); if (!buf) { DRV_LOG(ERR, "Can't allocate cqe buffer."); @@ -145,6 +148,159 @@ return -rte_errno; } +static int +regex_get_pdn(void *pd, uint32_t *pdn) +{ + struct mlx5dv_obj obj; + struct mlx5dv_pd pd_info; + int ret = 0; + + obj.pd.in = pd; + obj.pd.out = &pd_info; + ret = mlx5_glue->dv_init_obj(&obj, MLX5DV_OBJ_PD); + if (ret) { + DRV_LOG(DEBUG, "Fail to get PD object info"); + return ret; + } + *pdn = pd_info.pdn; + return 0; +} + +/** + * create the SQ object. + * + * @param priv + * Pointer to the priv object. + * @param qp + * Pointer to the QP element + * @param q_ind + * The index of the queue. + * @param log_nb_desc + * Log 2 of the number of descriptors to be used. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_create_sq(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, + uint16_t q_ind, uint16_t log_nb_desc) +{ + struct mlx5_devx_create_sq_attr attr = { 0 }; + struct mlx5_devx_modify_sq_attr modify_attr = { 0 }; + struct mlx5_devx_wq_attr *wq_attr = &attr.wq_attr; + struct mlx5_devx_dbr_page *dbr_page = NULL; + struct mlx5_regex_sq *sq = &qp->sqs[q_ind]; + void *buf = NULL; + uint32_t sq_size; + uint32_t pd_num = 0; + int ret; + + sq->log_nb_desc = log_nb_desc; + sq_size = 1 << sq->log_nb_desc; + sq->dbr_offset = mlx5_get_dbr(priv->ctx, &priv->dbrpgs, &dbr_page); + if (sq->dbr_offset < 0) { + DRV_LOG(ERR, "Can't allocate sq door bell record."); + rte_errno = ENOMEM; + goto error; + } + sq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); + sq->dbr = (uint32_t *)((uintptr_t)dbr_page->dbrs + + (uintptr_t)sq->dbr_offset); + + buf = rte_calloc(NULL, 1, 64 * sq_size, 4096); + if (!buf) { + DRV_LOG(ERR, "Can't allocate wqe buffer."); + rte_errno = ENOMEM; + goto error; + } + sq->wqe = buf; + sq->wqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, 64 * sq_size, + 7); + if (!sq->wqe_umem) { + DRV_LOG(ERR, "Can't register wqe mem."); + rte_errno = ENOMEM; + goto error; + } + attr.state = MLX5_SQC_STATE_RST; + attr.tis_lst_sz = 0; + attr.tis_num = 0; + attr.user_index = q_ind; + attr.cqn = qp->cq.obj->id; + wq_attr->uar_page = priv->uar->page_id; + regex_get_pdn(priv->pd, &pd_num); + wq_attr->pd = pd_num; + wq_attr->wq_type = MLX5_WQ_TYPE_CYCLIC; + wq_attr->dbr_umem_id = sq->dbr_umem; + wq_attr->dbr_addr = sq->dbr_offset; + wq_attr->dbr_umem_valid = 1; + wq_attr->wq_umem_id = mlx5_os_get_umem_id(sq->wqe_umem); + wq_attr->wq_umem_offset = 0; + wq_attr->wq_umem_valid = 1; + wq_attr->log_wq_stride = 6; + wq_attr->log_wq_sz = sq->log_nb_desc; + sq->obj = mlx5_devx_cmd_create_sq(priv->ctx, &attr); + if (!sq->obj) { + DRV_LOG(ERR, "Can't create sq object."); + rte_errno = ENOMEM; + goto error; + } + modify_attr.state = MLX5_SQC_STATE_RDY; + ret = mlx5_devx_cmd_modify_sq(sq->obj, &modify_attr); + if (ret) { + DRV_LOG(ERR, "Can't change sq state to ready."); + rte_errno = ENOMEM; + goto error; + } + + return 0; +error: + if (sq->wqe_umem) + mlx5_glue->devx_umem_dereg(sq->wqe_umem); + if (buf) + rte_free(buf); + if (sq->dbr_offset) + mlx5_release_dbr(&priv->dbrpgs, sq->dbr_umem, sq->dbr_offset); + return -rte_errno; +} + +/** + * Destroy the SQ object. + * + * @param priv + * Pointer to the priv object. + * @param qp + * Pointer to the QP element + * @param q_ind + * The index of the queue. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_destroy_sq(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, + uint16_t q_ind) +{ + struct mlx5_regex_sq *sq = &qp->sqs[q_ind]; + + if (sq->wqe_umem) { + mlx5_glue->devx_umem_dereg(sq->wqe_umem); + sq->wqe_umem = NULL; + } + if (sq->wqe) { + rte_free((void *)(uintptr_t)sq->wqe); + sq->wqe = NULL; + } + if (sq->dbr_offset) { + mlx5_release_dbr(&priv->dbrpgs, sq->dbr_umem, sq->dbr_offset); + sq->dbr_offset = -1; + } + if (sq->obj) { + mlx5_devx_cmd_destroy(sq->obj); + sq->obj = NULL; + } + return 0; +} + /** * Setup the qp. * @@ -164,7 +320,9 @@ { struct mlx5_regex_priv *priv = dev->data->dev_private; struct mlx5_regex_qp *qp; + int i; int ret; + uint16_t log_desc; qp = &priv->qps[qp_ind]; qp->flags = cfg->qp_conf_flags; @@ -181,15 +339,25 @@ rte_errno = ENOMEM; return -rte_errno; } + log_desc = rte_log2_u32(qp->nb_desc / qp->nb_obj); ret = regex_ctrl_create_cq(priv, &qp->cq); if (ret) { DRV_LOG(ERR, "Can't create cq."); goto error; } + for (i = 0; i < qp->nb_obj; i++) { + ret = regex_ctrl_create_sq(priv, qp, i, log_desc); + if (ret) { + DRV_LOG(ERR, "Can't create sq."); + goto error; + } + } return 0; error: regex_ctrl_destroy_cq(priv, &qp->cq); + for (i = 0; i < qp->nb_obj; i++) + ret = regex_ctrl_destroy_sq(priv, qp, i); return -rte_errno; } From patchwork Sun Jul 12 20:58:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73856 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1D497A052A; Sun, 12 Jul 2020 23:02:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7CF6A1D5AC; Sun, 12 Jul 2020 23:01:25 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id EFD7B1D529 for ; Sun, 12 Jul 2020 23:01:19 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 13 Jul 2020 00:00:29 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFGB025997; Mon, 13 Jul 2020 00:00:29 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 12 Jul 2020 20:58:57 +0000 Message-Id: <1594587541-110442-17-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 16/20] common/mlx5: add match tuple hw layout X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Add the found match tuple. Signed-off-by: Yuval Avnery Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_prm.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index bfbc58b..874dde6 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -409,6 +409,12 @@ struct mlx5_ifc_regexp_metadata_bits { uint8_t reserved[0x80]; }; +struct mlx5_ifc_regexp_match_tuple_bits { + uint8_t length[0x10]; + uint8_t start_ptr[0x10]; + uint8_t rule_id[0x20]; +}; + /* Adding direct verbs to data-path. */ /* CQ sequence number mask. */ From patchwork Sun Jul 12 20:58:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73858 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5F324A052A; Sun, 12 Jul 2020 23:02:46 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EC50E1D5C4; Sun, 12 Jul 2020 23:01:27 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 190191D53B for ; Sun, 12 Jul 2020 23:01:20 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 13 Jul 2020 00:00:31 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFGC025997; Mon, 13 Jul 2020 00:00:31 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 12 Jul 2020 20:58:58 +0000 Message-Id: <1594587541-110442-18-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 17/20] regex/mlx5: fastpath setup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Allocated and register input/output buffers and metadata. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/meson.build | 1 + drivers/regex/mlx5/mlx5_regex.h | 8 ++ drivers/regex/mlx5/mlx5_regex_control.c | 2 + drivers/regex/mlx5/mlx5_regex_fastpath.c | 198 +++++++++++++++++++++++++++++++ 5 files changed, 210 insertions(+) create mode 100644 drivers/regex/mlx5/mlx5_regex_fastpath.c diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 3d3fc5d..79bc0b8 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -10,6 +10,7 @@ LIB = librte_pmd_mlx5_regex.a SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_rxp.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_control.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_fastpath.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index c7406fe..a459f78 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -13,6 +13,7 @@ sources = files( 'mlx5_regex.c', 'mlx5_rxp.c', 'mlx5_regex_control.c', + 'mlx5_regex_fastpath.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 12033e8..cf8863f 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -44,6 +44,11 @@ struct mlx5_regex_qp { struct mlx5_regex_sq *sqs; /* Pointer to sq array. */ uint16_t nb_obj; /* Number of sq objects. */ struct mlx5_regex_cq cq; /* CQ struct. */ + uint32_t free_sqs; + struct mlx5_regex_job *jobs; + struct ibv_mr *metadata; + struct ibv_mr *inputs; + struct ibv_mr *outputs; }; struct mlx5_regex_db { @@ -83,4 +88,7 @@ int mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, const struct rte_regexdev_qp_conf *cfg); int mlx5_regex_rules_db_import(struct rte_regexdev *dev, const char *rule_db, uint32_t rule_db_len); + +/* mlx5_regex_fastpath.c */ +int mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index d378f48..9b3f39e 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -352,6 +352,8 @@ goto error; } } + + mlx5_regexdev_setup_fastpath(priv, qp_ind); return 0; error: diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c new file mode 100644 index 0000000..b5147ce --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -0,0 +1,198 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ +#include +#include + +#include +#include +#include +#include +#include +#include +#include + + +#include +#include +#include +#include +#include + +#include "mlx5_regex_utils.h" +#include "mlx5_rxp.h" +#include "mlx5_regex.h" + +/* Verbs header. */ +/* ISO C doesn't support unnamed structs/unions, disabling -pedantic. */ +#ifdef PEDANTIC +#pragma GCC diagnostic ignored "-Wpedantic" +#endif +#include +#ifdef PEDANTIC +#pragma GCC diagnostic error "-Wpedantic" +#endif + +#define MAX_WQE_INDEX 0xffff +#define MLX5_REGEX_METADATA_SIZE 64 +#define MLX5_REGEX_MAX_INPUT (1<<14) +#define MLX5_REGEX_MAX_OUTPUT (1<<11) + + +static inline uint32_t +sq_size_get(struct mlx5_regex_sq *sq) +{ + return (1U << sq->log_nb_desc); +} +static inline uint32_t +cq_size_get(struct mlx5_regex_cq *cq) +{ + return (1U << cq->log_nb_desc); +} + +struct mlx5_regex_job { + uint64_t user_id; + uint8_t *input; + volatile uint8_t *output; + volatile uint8_t *metadata; +} __rte_cached_aligned; + +static inline void +set_data_seg(struct mlx5_wqe_data_seg *seg, + uint32_t length, uint32_t lkey, + uintptr_t address) +{ + seg->byte_count = rte_cpu_to_be_32(length); + seg->lkey = rte_cpu_to_be_32(lkey); + seg->addr = rte_cpu_to_be_64(address); +} + +static inline void +set_metadata_seg(struct mlx5_wqe_metadata_seg *seg, + uint32_t mmo_control_31_0, uint32_t lkey, + uintptr_t address) +{ + seg->mmo_control_31_0 = htobe32(mmo_control_31_0); + seg->lkey = rte_cpu_to_be_32(lkey); + seg->addr = rte_cpu_to_be_64(address); +} + +static void +setup_sqs(struct mlx5_regex_qp *queue) +{ + size_t sqid, entry; + uint32_t job_id; + for (sqid = 0; sqid < queue->nb_obj; sqid++) { + struct mlx5_regex_sq *sq = &queue->sqs[sqid]; + uint8_t *wqe = (uint8_t *)sq->wqe; + for (entry = 0 ; entry < sq_size_get(sq); entry++) { + job_id = sqid * sq_size_get(sq) + entry; + struct mlx5_regex_job *job = &queue->jobs[job_id]; + + set_metadata_seg((struct mlx5_wqe_metadata_seg *) + (wqe + 16), 0, queue->metadata->lkey, + (uintptr_t)job->metadata); + set_data_seg((struct mlx5_wqe_data_seg *)(wqe + 32), + 0, queue->inputs->lkey, + (uintptr_t)job->input); + set_data_seg((struct mlx5_wqe_data_seg *)(wqe + 48), + 1 << 11, queue->outputs->lkey, + (uintptr_t)job->output); + wqe += 64; + } + queue->free_sqs |= 1 << sqid; + } +} + +static int +setup_buffers(struct mlx5_regex_qp *qp, struct ibv_pd *pd) +{ + int i, err; + + void *ptr = rte_calloc(__func__, qp->nb_desc, + MLX5_REGEX_METADATA_SIZE, + MLX5_REGEX_METADATA_SIZE); + if (!ptr) + return -ENOMEM; + + qp->metadata = mlx5_glue->reg_mr(pd, ptr, + MLX5_REGEX_METADATA_SIZE*qp->nb_desc, + IBV_ACCESS_LOCAL_WRITE); + if (!qp->metadata) { + rte_free(ptr); + return -EINVAL; + } + ptr = rte_calloc(__func__, qp->nb_desc, + MLX5_REGEX_MAX_INPUT, + MLX5_REGEX_MAX_INPUT); + + if (!ptr) { + err = -ENOMEM; + goto err_input; + } + qp->inputs = mlx5_glue->reg_mr(pd, ptr, + MLX5_REGEX_MAX_INPUT*qp->nb_desc, + IBV_ACCESS_LOCAL_WRITE); + if (!qp->inputs) { + rte_free(ptr); + err = -EINVAL; + goto err_input; + } + + ptr = rte_calloc(__func__, qp->nb_desc, + MLX5_REGEX_MAX_OUTPUT, + MLX5_REGEX_MAX_OUTPUT); + if (!ptr) { + err = -ENOMEM; + goto err_output; + } + qp->outputs = mlx5_glue->reg_mr(pd, ptr, + MLX5_REGEX_MAX_OUTPUT*qp->nb_desc, + IBV_ACCESS_LOCAL_WRITE); + if (!qp->outputs) { + rte_free(ptr); + err = -EINVAL; + goto err_output; + } + + /* distribute buffers to jobs */ + for (i = 0; i < qp->nb_desc; i++) { + qp->jobs[i].input = + (uint8_t *)qp->inputs->addr + + (i%qp->nb_desc)*MLX5_REGEX_MAX_INPUT; + qp->jobs[i].output = + (uint8_t *)qp->outputs->addr + + (i%qp->nb_desc)*MLX5_REGEX_MAX_OUTPUT; + qp->jobs[i].metadata = + (uint8_t *)qp->metadata->addr + + (i%qp->nb_desc)*MLX5_REGEX_METADATA_SIZE; + } + return 0; + +err_output: + ptr = qp->inputs->addr; + rte_free(ptr); + mlx5_glue->dereg_mr(qp->inputs); +err_input: + ptr = qp->metadata->addr; + rte_free(ptr); + mlx5_glue->dereg_mr(qp->metadata); + return err; +} + +int +mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id) +{ + struct mlx5_regex_qp *qp = &priv->qps[qp_id]; + int err; + + qp->jobs = rte_calloc(__func__, qp->nb_desc, sizeof(*qp->jobs), + sizeof(*qp->jobs)); + if (!qp->jobs) + return -ENOMEM; + err = setup_buffers(qp, priv->pd); + if (err) + return err; + setup_sqs(qp); + return 0; +} From patchwork Sun Jul 12 20:58:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73855 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6741AA052A; Sun, 12 Jul 2020 23:02:17 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4EAB31D57A; Sun, 12 Jul 2020 23:01:24 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id F06211D535 for ; Sun, 12 Jul 2020 23:01:19 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 13 Jul 2020 00:00:35 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFGD025997; Mon, 13 Jul 2020 00:00:34 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 12 Jul 2020 20:58:59 +0000 Message-Id: <1594587541-110442-19-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 18/20] regex/mlx5: add enqueue implementation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Will look for a free SQ to send the job on. doorbell will be given when sq is full, or no more jobs on the burst. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 1 + drivers/regex/mlx5/mlx5_regex.h | 6 ++ drivers/regex/mlx5/mlx5_regex_control.c | 2 + drivers/regex/mlx5/mlx5_regex_fastpath.c | 146 +++++++++++++++++++++++++++++-- 4 files changed, 148 insertions(+), 7 deletions(-) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index f06f817..d823e17 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -180,6 +180,7 @@ goto error; } priv->regexdev->dev_ops = &mlx5_regexdev_ops; + priv->regexdev->enqueue = mlx5_regexdev_enqueue; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; priv->regexdev->state = RTE_REGEXDEV_READY; diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index cf8863f..468772c 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -25,6 +25,9 @@ struct mlx5_regex_sq { uint32_t dbr_umem; /* Door bell record umem id. */ uint8_t *wqe; /* The SQ ring buffer. */ struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ + size_t pi, db_pi; + size_t ci; + uint32_t sqn; uint32_t *dbr; }; @@ -49,6 +52,7 @@ struct mlx5_regex_qp { struct ibv_mr *metadata; struct ibv_mr *inputs; struct ibv_mr *outputs; + size_t ci, pi; }; struct mlx5_regex_db { @@ -91,4 +95,6 @@ int mlx5_regex_rules_db_import(struct rte_regexdev *dev, /* mlx5_regex_fastpath.c */ int mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id); +uint16_t mlx5_regexdev_enqueue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index 9b3f39e..c2d080f 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -216,6 +216,8 @@ sq->wqe = buf; sq->wqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, 64 * sq_size, 7); + sq->ci = 0; + sq->pi = 0; if (!sq->wqe_umem) { DRV_LOG(ERR, "Can't register wqe mem."); rte_errno = ENOMEM; diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c index b5147ce..1823353 100644 --- a/drivers/regex/mlx5/mlx5_regex_fastpath.c +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -33,10 +33,14 @@ #pragma GCC diagnostic error "-Wpedantic" #endif -#define MAX_WQE_INDEX 0xffff +#define MLX5_REGEX_MAX_WQE_INDEX 0xffff #define MLX5_REGEX_METADATA_SIZE 64 -#define MLX5_REGEX_MAX_INPUT (1<<14) -#define MLX5_REGEX_MAX_OUTPUT (1<<11) +#define MLX5_REGEX_MAX_INPUT (1 << 14) +#define MLX5_REGEX_MAX_OUTPUT (1 << 11) +#define MLX5_REGEX_WQE_CTRL_OFFSET 12 +#define MLX5_REGEX_WQE_METADATA_OFFSET 16 +#define MLX5_REGEX_WQE_GATHER_OFFSET 32 +#define MLX5_REGEX_WQE_SCATTER_OFFSET 48 static inline uint32_t @@ -77,6 +81,134 @@ struct mlx5_regex_job { seg->addr = rte_cpu_to_be_64(address); } +static inline void +set_regex_ctrl_seg(void *seg, uint8_t le, uint16_t subset_id0, + uint16_t subset_id1, uint16_t subset_id2, + uint16_t subset_id3, uint8_t ctrl) +{ + DEVX_SET(regexp_mmo_control, seg, le, le); + DEVX_SET(regexp_mmo_control, seg, ctrl, ctrl); + DEVX_SET(regexp_mmo_control, seg, subset_id_0, subset_id0); + DEVX_SET(regexp_mmo_control, seg, subset_id_1, subset_id1); + DEVX_SET(regexp_mmo_control, seg, subset_id_2, subset_id2); + DEVX_SET(regexp_mmo_control, seg, subset_id_3, subset_id3); +} + +static inline void +set_wqe_ctrl_seg(struct mlx5_wqe_ctrl_seg *seg, uint16_t pi, uint8_t opcode, + uint8_t opmod, uint32_t qp_num, uint8_t fm_ce_se, uint8_t ds, + uint8_t signature, uint32_t imm) +{ + seg->opmod_idx_opcode = rte_cpu_to_be_32(((uint32_t)opmod << 24) | + ((uint32_t)pi << 8) | + opcode); + seg->qpn_ds = rte_cpu_to_be_32((qp_num << 8) | ds); + seg->fm_ce_se = fm_ce_se; + seg->signature = signature; + seg->imm = imm; +} + +static inline void +prep_one(struct mlx5_regex_sq *sq, struct rte_regex_ops *op, + struct mlx5_regex_job *job) +{ + size_t wqe_offset = (sq->pi % sq_size_get(sq)) * MLX5_SEND_WQE_BB; + uint8_t *wqe = (uint8_t *)sq->wqe + wqe_offset; + int ds = 4; /* ctrl + meta + input + output */ + + memcpy(job->input, + rte_pktmbuf_mtod(op->mbuf, void *), + rte_pktmbuf_data_len(op->mbuf)); + set_wqe_ctrl_seg((struct mlx5_wqe_ctrl_seg *)wqe, sq->pi, + MLX5_OPCODE_MMO, MLX5_OPC_MOD_MMO_REGEX, sq->obj->id, + 0, ds, 0, 0); + set_regex_ctrl_seg(wqe + 12, 0, op->group_id0, op->group_id1, + op->group_id2, + op->group_id3, 0); + struct mlx5_wqe_data_seg *input_seg = + (struct mlx5_wqe_data_seg *)(wqe + + MLX5_REGEX_WQE_GATHER_OFFSET); + input_seg->byte_count = + rte_cpu_to_be_32(rte_pktmbuf_data_len(op->mbuf)); + job->user_id = op->user_id; + sq->db_pi = sq->pi; + sq->pi = (sq->pi + 1) % MLX5_REGEX_MAX_WQE_INDEX; +} + +static inline void +send_doorbell(struct mlx5dv_devx_uar *uar, struct mlx5_regex_sq *sq) +{ + size_t wqe_offset = (sq->db_pi % sq_size_get(sq)) * MLX5_SEND_WQE_BB; + uint8_t *wqe = (uint8_t *)sq->wqe + wqe_offset; + ((struct mlx5_wqe_ctrl_seg *)wqe)->fm_ce_se = MLX5_WQE_CTRL_CQ_UPDATE; + uint64_t *doorbell_addr = + (uint64_t *)((uint8_t *)uar->base_addr + 0x800); + rte_cio_wmb(); + sq->dbr[MLX5_SND_DBR] = rte_cpu_to_be_32((sq->db_pi + 1) % + MLX5_REGEX_MAX_WQE_INDEX); + rte_wmb(); + *doorbell_addr = *(volatile uint64_t *)wqe; + rte_wmb(); +} + +static inline int +can_send(struct mlx5_regex_sq *sq) { + return unlikely(sq->ci > sq->pi) ? + MLX5_REGEX_MAX_WQE_INDEX + sq->pi - sq->ci < + sq_size_get(sq) : + sq->pi - sq->ci < sq_size_get(sq); +} + +static inline uint32_t +job_id_get(uint32_t qid, size_t sq_size, size_t index) { + return qid*sq_size + index % sq_size; +} + +/** + * DPDK callback for enqueue. + * + * @param dev + * Pointer to the regex dev structure. + * @param qp_id + * The queue to enqueue the traffic to. + * @param ops + * List of regex ops to enqueue. + * @param nb_ops + * Number of ops in ops parameter. + * + * @return + * Number of packets successfully enqueued (<= pkts_n). + */ +uint16_t +mlx5_regexdev_enqueue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_regex_qp *queue = &priv->qps[qp_id]; + struct mlx5_regex_sq *sq; + size_t sqid, job_id, i = 0; + + while ((sqid = ffs(queue->free_sqs))) { + sqid--; /* ffs returns 1 for bit 0 */ + sq = &queue->sqs[sqid]; + while (can_send(sq)) { + job_id = job_id_get(sqid, sq_size_get(sq), sq->pi); + prep_one(sq, ops[i], &queue->jobs[job_id]); + i++; + if (unlikely(i == nb_ops)) { + send_doorbell(priv->uar, sq); + goto out; + } + } + queue->free_sqs &= ~(1 << sqid); + send_doorbell(priv->uar, sq); + } + +out: + queue->pi += i; + return i; +} + static void setup_sqs(struct mlx5_regex_qp *queue) { @@ -147,7 +279,7 @@ struct mlx5_regex_job { goto err_output; } qp->outputs = mlx5_glue->reg_mr(pd, ptr, - MLX5_REGEX_MAX_OUTPUT*qp->nb_desc, + MLX5_REGEX_MAX_OUTPUT * qp->nb_desc, IBV_ACCESS_LOCAL_WRITE); if (!qp->outputs) { rte_free(ptr); @@ -159,13 +291,13 @@ struct mlx5_regex_job { for (i = 0; i < qp->nb_desc; i++) { qp->jobs[i].input = (uint8_t *)qp->inputs->addr + - (i%qp->nb_desc)*MLX5_REGEX_MAX_INPUT; + (i % qp->nb_desc) * MLX5_REGEX_MAX_INPUT; qp->jobs[i].output = (uint8_t *)qp->outputs->addr + - (i%qp->nb_desc)*MLX5_REGEX_MAX_OUTPUT; + (i % qp->nb_desc) * MLX5_REGEX_MAX_OUTPUT; qp->jobs[i].metadata = (uint8_t *)qp->metadata->addr + - (i%qp->nb_desc)*MLX5_REGEX_METADATA_SIZE; + (i % qp->nb_desc) * MLX5_REGEX_METADATA_SIZE; } return 0; From patchwork Sun Jul 12 20:59:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73857 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1760BA052A; Sun, 12 Jul 2020 23:02:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B1B211D5B9; Sun, 12 Jul 2020 23:01:26 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 1DE181D53D for ; Sun, 12 Jul 2020 23:01:20 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 13 Jul 2020 00:00:37 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFGE025997; Mon, 13 Jul 2020 00:00:37 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 12 Jul 2020 20:59:00 +0000 Message-Id: <1594587541-110442-20-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 19/20] regex/mlx5: implement dequeue function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Implement dequeue function for the regex API. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 1 + drivers/regex/mlx5/mlx5_regex.h | 4 ++ drivers/regex/mlx5/mlx5_regex_control.c | 1 + drivers/regex/mlx5/mlx5_regex_fastpath.c | 104 +++++++++++++++++++++++++++++++ 4 files changed, 110 insertions(+) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index d823e17..765b67b 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -181,6 +181,7 @@ } priv->regexdev->dev_ops = &mlx5_regexdev_ops; priv->regexdev->enqueue = mlx5_regexdev_enqueue; + priv->regexdev->dequeue = mlx5_regexdev_dequeue; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; priv->regexdev->state = RTE_REGEXDEV_READY; diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 468772c..aefd7da 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -38,6 +38,7 @@ struct mlx5_regex_cq { uint32_t dbr_umem; /* Door bell record umem id. */ volatile struct mlx5_cqe *cqe; /* The CQ ring buffer. */ struct mlx5dv_devx_umem *cqe_umem; /* CQ buffer umem. */ + size_t ci; uint32_t *dbr; }; @@ -97,4 +98,7 @@ int mlx5_regex_rules_db_import(struct rte_regexdev *dev, int mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id); uint16_t mlx5_regexdev_enqueue(struct rte_regexdev *dev, uint16_t qp_id, struct rte_regex_ops **ops, uint16_t nb_ops); +uint16_t mlx5_regexdev_dequeue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops); + #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index c2d080f..65c623a 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -120,6 +120,7 @@ cq->cqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, sizeof(struct mlx5_cqe) * cq_size, 7); + cq->ci = 0; if (!cq->cqe_umem) { DRV_LOG(ERR, "Can't register cqe mem."); rte_errno = ENOMEM; diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c index 1823353..0d4d069 100644 --- a/drivers/regex/mlx5/mlx5_regex_fastpath.c +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -41,6 +41,7 @@ #define MLX5_REGEX_WQE_METADATA_OFFSET 16 #define MLX5_REGEX_WQE_GATHER_OFFSET 32 #define MLX5_REGEX_WQE_SCATTER_OFFSET 48 +#define MLX5_REGEX_METADATA_OFF 32 static inline uint32_t @@ -209,6 +210,109 @@ struct mlx5_regex_job { return i; } +#define MLX5_REGEX_RESP_SZ 8 + +static inline void +extract_result(struct rte_regex_ops *op, struct mlx5_regex_job *job) +{ + size_t j, offset; + op->user_id = job->user_id; + op->nb_matches = DEVX_GET(regexp_metadata, job->metadata + + MLX5_REGEX_METADATA_OFF, match_count); + op->nb_actual_matches = DEVX_GET(regexp_metadata, job->metadata + + MLX5_REGEX_METADATA_OFF, + detected_match_count); + for (j = 0; j < op->nb_matches; j++) { + offset = MLX5_REGEX_RESP_SZ * j; + op->matches[j].rule_id = + DEVX_GET(regexp_match_tuple, (job->output + offset), + rule_id); + op->matches[j].start_offset = + DEVX_GET(regexp_match_tuple, (job->output + offset), + start_ptr); + op->matches[j].len = + DEVX_GET(regexp_match_tuple, (job->output + offset), + length); + } +} + +static inline volatile struct mlx5_cqe * +poll_one(struct mlx5_regex_cq *cq) +{ + volatile struct mlx5_cqe *cqe; + size_t next_cqe_offset; + + next_cqe_offset = (cq->ci % cq_size_get(cq)) * sizeof(*cqe); + cqe = (volatile struct mlx5_cqe *)(cq->cqe + next_cqe_offset); + rte_cio_wmb(); + + int ret = check_cqe(cqe, cq_size_get(cq), cq->ci); + + if (unlikely(ret == MLX5_CQE_STATUS_ERR)) { + DRV_LOG(ERR, "Completion with error on qp 0x%x", 0); + return NULL; + } + + if (unlikely(ret != MLX5_CQE_STATUS_SW_OWN)) + return NULL; + + return cqe; +} + + +/** + * DPDK callback for dequeue. + * + * @param dev + * Pointer to the regex dev structure. + * @param qp_id + * The queue to enqueue the traffic to. + * @param ops + * List of regex ops to dequeue. + * @param nb_ops + * Number of ops in ops parameter. + * + * @return + * Number of packets successfully dequeued (<= pkts_n). + */ +uint16_t +mlx5_regexdev_dequeue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_regex_qp *queue = &priv->qps[qp_id]; + struct mlx5_regex_cq *cq = &queue->cq; + volatile struct mlx5_cqe *cqe; + size_t i = 0; + + while ((cqe = poll_one(cq))) { + uint16_t wq_counter + = (rte_be_to_cpu_16(cqe->wqe_counter) + 1) % + MLX5_REGEX_MAX_WQE_INDEX; + size_t sqid = cqe->rsvd3[2]; + struct mlx5_regex_sq *sq = &queue->sqs[sqid]; + while (sq->ci != wq_counter) { + if (unlikely(i == nb_ops)) { + /* Return without updating cq->ci */ + goto out; + } + uint32_t job_id = job_id_get(sqid, sq_size_get(sq), + sq->ci); + extract_result(ops[i], &queue->jobs[job_id]); + sq->ci = (sq->ci + 1) % MLX5_REGEX_MAX_WQE_INDEX; + i++; + } + cq->ci = (cq->ci + 1) & 0xffffff; + rte_wmb(); + cq->dbr[0] = rte_cpu_to_be_32(cq->ci); + queue->free_sqs |= (1 << sqid); + } + +out: + queue->ci += i; + return i; +} + static void setup_sqs(struct mlx5_regex_qp *queue) { From patchwork Sun Jul 12 20:59:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73854 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D3867A052A; Sun, 12 Jul 2020 23:02:06 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F19CE1D535; Sun, 12 Jul 2020 23:01:20 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id DF7341D51F for ; Sun, 12 Jul 2020 23:01:19 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 13 Jul 2020 00:00:40 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06CKxFGF025997; Mon, 13 Jul 2020 00:00:40 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 12 Jul 2020 20:59:01 +0000 Message-Id: <1594587541-110442-21-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594587541-110442-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> <1594587541-110442-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH v2 20/20] regex/mlx5: add start stop functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the start, stop and close functions. In current implementation they are empty functions and are only exists in order that when called from rte level, the function will return with success code. Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 20 ++++++++++++++++++++ drivers/regex/mlx5/mlx5_regex.h | 6 ++++++ 2 files changed, 26 insertions(+) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 765b67b..8c1ec23 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -26,8 +26,28 @@ .dev_configure = mlx5_regex_configure, .dev_db_import = mlx5_regex_rules_db_import, .dev_qp_setup = mlx5_regex_qp_setup, + .dev_start = mlx5_regex_start, + .dev_stop = mlx5_regex_stop, + .dev_close = mlx5_regex_close, }; +int +mlx5_regex_start(struct rte_regexdev *dev __rte_unused) +{ + return 0; +} + +int +mlx5_regex_stop(struct rte_regexdev *dev __rte_unused) +{ + return 0; +} + +int +mlx5_regex_close(struct rte_regexdev *dev __rte_unused) +{ + return 0; +} static struct ibv_device * mlx5_regex_get_ib_device_match(struct rte_pci_addr *addr) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index aefd7da..a472414 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -84,6 +84,12 @@ struct mlx5_regex_priv { struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ }; +/* mlx5_regex.c */ + +int mlx5_regex_start(struct rte_regexdev *dev); +int mlx5_regex_stop(struct rte_regexdev *dev); +int mlx5_regex_close(struct rte_regexdev *dev); + /* mlx5_rxp.c */ int mlx5_regex_info_get(struct rte_regexdev *dev, struct rte_regexdev_info *info);