From patchwork Sun Jul 5 09:23:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73112 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B7124A00C5; Sun, 5 Jul 2020 11:24:30 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5C99B1DAAE; Sun, 5 Jul 2020 11:24:24 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id A68F91DAB4 for ; Sun, 5 Jul 2020 11:24:22 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:20 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3l028191; Sun, 5 Jul 2020 12:24:19 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Thomas Monjalon , Shahaf Shuler , Ray Kinsella , Neil Horman Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 5 Jul 2020 09:23:28 +0000 Message-Id: <1593941027-86651-2-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 01/20] regex/mlx5: add RegEx PMD layer and mlx5 driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery This commit introduce the RegEx pull mode drivers class, and adds Mellanox RegEx PMD. Signed-off-by: Yuval Avnery Signed-off-by: Ori Kam --- config/common_base | 5 +++ drivers/Makefile | 2 + drivers/common/Makefile | 2 +- drivers/common/mlx5/Makefile | 4 +- drivers/meson.build | 3 +- drivers/regex/Makefile | 8 ++++ drivers/regex/meson.build | 9 ++++ drivers/regex/mlx5/Makefile | 55 +++++++++++++++++++++++ drivers/regex/mlx5/meson.build | 32 +++++++++++++ drivers/regex/mlx5/mlx5_regex.c | 5 +++ drivers/regex/mlx5/mlx5_regex.h | 8 ++++ drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map | 3 ++ mk/rte.app.mk | 5 ++- 13 files changed, 135 insertions(+), 6 deletions(-) create mode 100644 drivers/regex/Makefile create mode 100644 drivers/regex/meson.build create mode 100644 drivers/regex/mlx5/Makefile create mode 100644 drivers/regex/mlx5/meson.build create mode 100644 drivers/regex/mlx5/mlx5_regex.c create mode 100644 drivers/regex/mlx5/mlx5_regex.h create mode 100644 drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map diff --git a/config/common_base b/config/common_base index e15517b..c97f8c5 100644 --- a/config/common_base +++ b/config/common_base @@ -374,6 +374,11 @@ CONFIG_RTE_LIBRTE_MLX5_PMD=n CONFIG_RTE_LIBRTE_MLX5_DEBUG=n # +# Compile regex-oriented Mellanox PMD +# +CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD=n + +# # Compile vdpa-oriented Mellanox ConnectX-6 & BlueField (MLX5) PMD # CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD=n diff --git a/drivers/Makefile b/drivers/Makefile index c70bdf9..73a9a72 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -24,5 +24,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += event DEPDIRS-event := common bus mempool net crypto DIRS-$(CONFIG_RTE_LIBRTE_RAWDEV) += raw DEPDIRS-raw := common bus mempool net event +DIRS-$(CONFIG_RTE_LIBRTE_REGEXDEV) += regex +DEPDIRS-regex := common include $(RTE_SDK)/mk/rte.subdir.mk diff --git a/drivers/common/Makefile b/drivers/common/Makefile index df2e840..cbc7107 100644 --- a/drivers/common/Makefile +++ b/drivers/common/Makefile @@ -36,7 +36,7 @@ ifneq (,$(findstring y,$(IAVF-y))) DIRS-y += iavf endif -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) DIRS-y += mlx5 endif diff --git a/drivers/common/mlx5/Makefile b/drivers/common/mlx5/Makefile index f6c762b..5f12be3 100644 --- a/drivers/common/mlx5/Makefile +++ b/drivers/common/mlx5/Makefile @@ -10,7 +10,7 @@ LIB_GLUE_BASE = librte_pmd_mlx5_glue.so LIB_GLUE_VERSION = 20.02.0 # Sources. -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) ifneq ($(CONFIG_RTE_IBVERBS_LINK_DLOPEN),y) SRCS-y += linux/mlx5_glue.c endif @@ -344,7 +344,7 @@ mlx5_autoconf.h: mlx5_autoconf.h.new cmp '$<' '$@' $(AUTOCONF_OUTPUT) || \ mv '$<' '$@' -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) $(SRCS-y:.c=.o): mlx5_autoconf.h endif diff --git a/drivers/meson.build b/drivers/meson.build index 01c860c..3319c6d 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -11,7 +11,8 @@ dpdk_driver_classes = ['common', 'compress', # depends on common, bus, mempool. 'vdpa', # depends on common, bus and mempool. 'event', # depends on common, bus, mempool and net. - 'baseband'] # depends on common and bus. + 'baseband', # depends on common and bus. + 'regex'] # depends on common, bus, regexdev. disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), ).stdout().split() diff --git a/drivers/regex/Makefile b/drivers/regex/Makefile new file mode 100644 index 0000000..906b205 --- /dev/null +++ b/drivers/regex/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +include $(RTE_SDK)/mk/rte.vars.mk + +DIRS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5 + +include $(RTE_SDK)/mk/rte.subdir.mk diff --git a/drivers/regex/meson.build b/drivers/regex/meson.build new file mode 100644 index 0000000..75522e3 --- /dev/null +++ b/drivers/regex/meson.build @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +drivers = ['mlx5'] +std_deps = ['ethdev', 'kvargs'] # 'ethdev' also pulls in mbuf, net, eal etc +std_deps += ['bus_pci'] # very many PMDs depend on PCI, so make std +std_deps += ['bus_vdev'] # same with vdev bus +config_flag_fmt = 'RTE_LIBRTE_@0@_PMD' +driver_name_fmt = 'rte_pmd_@0@' diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile new file mode 100644 index 0000000..2ddad2d --- /dev/null +++ b/drivers/regex/mlx5/Makefile @@ -0,0 +1,55 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +include $(RTE_SDK)/mk/rte.vars.mk + +# Library name. +LIB = librte_pmd_mlx5_regex.a + +# Sources. +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c + +# Basic CFLAGS. +CFLAGS += -O3 +CFLAGS += -std=c11 -Wall -Wextra +CFLAGS += -g +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 +CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -Wno-strict-prototypes +LDLIBS += -lrte_common_mlx5 +LDLIBS += -lm +LDLIBS += -lrte_eal -lrte_mbuf +LDLIBS += -lrte_kvargs +LDLIBS += -lrte_bus_pci + +# A few warnings cannot be avoided in external headers. +CFLAGS += -Wno-error=cast-qual + +EXPORT_MAP := rte_pmd_mlx5_regex_version.map + +# memseg walk is not part of stable API +CFLAGS += -DALLOW_EXPERIMENTAL_API + +# DEBUG which is usually provided on the command-line may enable +# CONFIG_RTE_LIBRTE_MLX5_DEBUG. +ifeq ($(DEBUG),1) +CONFIG_RTE_LIBRTE_MLX5_DEBUG := y +endif + +# User-defined CFLAGS. +ifeq ($(CONFIG_RTE_LIBRTE_MLX5_DEBUG),y) +CFLAGS += -pedantic -UNDEBUG +ifneq ($(CONFIG_RTE_TOOLCHAIN_ICC),y) +CFLAGS += -DPEDANTIC +endif +AUTO_CONFIG_CFLAGS += -Wno-pedantic +else +CFLAGS += -DNDEBUG -UPEDANTIC +endif + +include $(RTE_SDK)/mk/rte.lib.mk + diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build new file mode 100644 index 0000000..768a9d1 --- /dev/null +++ b/drivers/regex/mlx5/meson.build @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2019 Mellanox Technologies, Ltd + +if not is_linux + build = false + reason = 'only supported on Linux' + subdir_done() +endif + +fmt_name = 'mlx5_regex' +deps += ['common_mlx5', 'pci', 'bus_pci', 'eal', 'sched'] +sources = files( + 'mlx5_regex.c', +) +cflags_options = [ + '-std=c11', + '-Wno-strict-prototypes', + '-D_BSD_SOURCE', + '-D_DEFAULT_SOURCE', + '-D_XOPEN_SOURCE=600' +] +foreach option:cflags_options + if cc.has_argument(option) + cflags += option + endif +endforeach + +if get_option('buildtype').contains('debug') + cflags += [ '-pedantic', '-DPEDANTIC' ] +else + cflags += [ '-UPEDANTIC' ] +endif diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c new file mode 100644 index 0000000..b942a75 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -0,0 +1,5 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include "mlx5_regex.h" diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h new file mode 100644 index 0000000..0e0495c --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#ifndef MLX5_REGEX_H +#define MLX5_REGEX_H + +#endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map b/drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map new file mode 100644 index 0000000..4a76d1d --- /dev/null +++ b/drivers/regex/mlx5/rte_pmd_mlx5_regex_version.map @@ -0,0 +1,3 @@ +DPDK_21 { + local: *; +}; diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 0ce8cf5..1b9551e 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -201,11 +201,12 @@ endif _LDLIBS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += -lrte_pmd_lio _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_MEMIF) += -lrte_pmd_memif _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += -lrte_pmd_mlx4 -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) _LDLIBS-y += -lrte_common_mlx5 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += -lrte_pmd_mlx5 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD) += -lrte_pmd_mlx5_vdpa +_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += -lrte_pmd_mlx5_regex ifeq ($(CONFIG_RTE_IBVERBS_LINK_DLOPEN),y) _LDLIBS-y += -ldl else ifeq ($(CONFIG_RTE_IBVERBS_LINK_STATIC),y) @@ -214,7 +215,7 @@ _LDLIBS-y += --no-whole-archive _LDLIBS-y += $(LIBS_IBVERBS_STATIC) _LDLIBS-y += --whole-archive else -ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD)),y) _LDLIBS-y += -libverbs -lmlx5 endif _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += -libverbs -lmlx4 From patchwork Sun Jul 5 09:23:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73113 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3946FA00C5; Sun, 5 Jul 2020 11:24:39 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 85C6C1DABF; Sun, 5 Jul 2020 11:24:29 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id B06C41DA97 for ; Sun, 5 Jul 2020 11:24:27 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:24 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3m028191; Sun, 5 Jul 2020 12:24:24 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 5 Jul 2020 09:23:29 +0000 Message-Id: <1593941027-86651-3-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 02/20] regex/mlx5: add log utils X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Add the DRV_LOG macro which should be used for error prints. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/mlx5_regex.c | 4 ++++ drivers/regex/mlx5/mlx5_regex_utils.h | 19 +++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 drivers/regex/mlx5/mlx5_regex_utils.h diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 2ddad2d..40c03ab 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -15,6 +15,7 @@ CFLAGS += -std=c11 -Wall -Wextra CFLAGS += -g CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux CFLAGS += -D_BSD_SOURCE CFLAGS += -D_DEFAULT_SOURCE CFLAGS += -D_XOPEN_SOURCE=600 diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index b942a75..06826a6 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -3,3 +3,7 @@ */ #include "mlx5_regex.h" +#include "mlx5_regex_utils.h" + +int mlx5_regex_logtype; + diff --git a/drivers/regex/mlx5/mlx5_regex_utils.h b/drivers/regex/mlx5/mlx5_regex_utils.h new file mode 100644 index 0000000..06efe3e --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex_utils.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2019 Mellanox Technologies, Ltd + */ + +#ifndef RTE_PMD_MLX5_REGEX_UTILS_H_ +#define RTE_PMD_MLX5_REGEX_UTILS_H_ + +#include + +extern int mlx5_regex_logtype; + +#define MLX5_REGEX_LOG_PREFIX "regex_mlx5" +/* Generic printf()-like logging macro with automatic line feed. */ +#define DRV_LOG(level, ...) \ + PMD_DRV_LOG_(level, mlx5_regex_logtype, MLX5_REGEX_LOG_PREFIX, \ + __VA_ARGS__ PMD_DRV_LOG_STRIP PMD_DRV_LOG_OPAREN, \ + PMD_DRV_LOG_CPAREN) + +#endif /* RTE_PMD_MLX5_REGEX_UTILS_H_ */ From patchwork Sun Jul 5 09:23:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73114 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D0DCFA00C5; Sun, 5 Jul 2020 11:24:47 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B43021DAB2; Sun, 5 Jul 2020 11:24:34 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id AB3791DAAC for ; Sun, 5 Jul 2020 11:24:32 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:28 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3n028191; Sun, 5 Jul 2020 12:24:27 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 5 Jul 2020 09:23:30 +0000 Message-Id: <1593941027-86651-4-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 03/20] common/mlx5: add MMO and regexp structs/opcodes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Added General purpose PRM MMO structs, and regex specific structs. Signed-off-by: Yuval Avnery Signed-off-by: Ori Kam Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_prm.h | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index c63795f..c2b9a20 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -373,6 +373,42 @@ struct mlx5_cqe { uint8_t op_own; }; +/* MMO metadata segment */ + +#define MLX5_OPCODE_MMO 0x2f +#define MLX5_OPC_MOD_MMO_REGEX 0x4 + +struct mlx5_wqe_metadata_seg { + uint32_t mmo_control_31_0; /* mmo_control_63_32 is in ctrl_seg.imm */ + uint32_t lkey; + uint64_t addr; +}; + +struct mlx5_ifc_regexp_mmo_control_bits { + uint8_t reserved_at_31[0x2]; + uint8_t le[0x1]; + uint8_t reserved_at_28[0x1]; + uint8_t subset_id_0[0xc]; + uint8_t reserved_at_16[0x4]; + uint8_t subset_id_1[0xc]; + uint8_t ctrl[0x4]; + uint8_t subset_id_2[0xc]; + uint8_t reserved_at_16_1[0x4]; + uint8_t subset_id_3[0xc]; +}; + +struct mlx5_ifc_regexp_metadata_bits { + uint8_t rof_version[0x10]; + uint8_t latency_count[0x10]; + uint8_t instruction_count[0x10]; + uint8_t primary_thread_count[0x10]; + uint8_t match_count[0x8]; + uint8_t detected_match_count[0x8]; + uint8_t status[0x10]; + uint8_t job_id[0x20]; + uint8_t reserved[0x80]; +}; + /* Adding direct verbs to data-path. */ /* CQ sequence number mask. */ @@ -759,6 +795,10 @@ enum { MLX5_CMD_OP_CREATE_GENERAL_OBJECT = 0xa00, MLX5_CMD_OP_MODIFY_GENERAL_OBJECT = 0xa01, MLX5_CMD_OP_QUERY_GENERAL_OBJECT = 0xa02, + MLX5_CMD_SET_REGEX_PARAM = 0xb04, + MLX5_CMD_QUERY_REGEX_PARAMS = 0xb05, + MLX5_CMD_SET_REGEX_REGISTERS = 0xb06, + MLX5_CMD_QUERY_REGEX_REGISTERS = 0xb07, }; enum { From patchwork Sun Jul 5 09:23:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73115 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E48E9A00C5; Sun, 5 Jul 2020 11:24:57 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0B3071DAD7; Sun, 5 Jul 2020 11:24:39 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id BBBC91DAD3 for ; Sun, 5 Jul 2020 11:24:37 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:32 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3o028191; Sun, 5 Jul 2020 12:24:32 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 5 Jul 2020 09:23:31 +0000 Message-Id: <1593941027-86651-5-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 04/20] common/mlx5: add mlx5 regex command structs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Add regex commands structs to support regex. Signed-off-by: Yuval Avnery Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_prm.h | 89 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 88 insertions(+), 1 deletion(-) diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index c2b9a20..ede7810 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -795,7 +795,7 @@ enum { MLX5_CMD_OP_CREATE_GENERAL_OBJECT = 0xa00, MLX5_CMD_OP_MODIFY_GENERAL_OBJECT = 0xa01, MLX5_CMD_OP_QUERY_GENERAL_OBJECT = 0xa02, - MLX5_CMD_SET_REGEX_PARAM = 0xb04, + MLX5_CMD_SET_REGEX_PARAMS = 0xb04, MLX5_CMD_QUERY_REGEX_PARAMS = 0xb05, MLX5_CMD_SET_REGEX_REGISTERS = 0xb06, MLX5_CMD_QUERY_REGEX_REGISTERS = 0xb07, @@ -2526,6 +2526,93 @@ struct mlx5_ifc_query_qp_in_bits { u8 reserved_at_60[0x20]; }; +struct regexp_params_field_select_bits { + u8 reserved_at_0[0x1e]; + u8 stop_engine[0x1]; + u8 db_umem_id[0x1]; +}; + +struct mlx5_ifc_regexp_params_bits { + u8 reserved_at_0[0x1f]; + u8 stop_engine[0x1]; + u8 db_umem_id[0x20]; + u8 db_umem_offset[0x40]; + u8 reserved_at_80[0x100]; +}; + +struct mlx5_ifc_set_regexp_params_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + struct regexp_params_field_select_bits field_select; + struct mlx5_ifc_regexp_params_bits regexp_params; +}; + +struct mlx5_ifc_set_regexp_params_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved_at_18[0x40]; +}; + +struct mlx5_ifc_query_regexp_params_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + u8 reserved[0x20]; +}; + +struct mlx5_ifc_query_regexp_params_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved[0x40]; + struct mlx5_ifc_regexp_params_bits regexp_params; +}; + +struct mlx5_ifc_set_regexp_register_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + u8 register_address[0x20]; + u8 register_data[0x20]; + u8 reserved[0x40]; +}; + +struct mlx5_ifc_set_regexp_register_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved[0x40]; +}; + +struct mlx5_ifc_query_regexp_register_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x18]; + u8 engine_id[0x8]; + u8 register_address[0x20]; +}; + +struct mlx5_ifc_query_regexp_register_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved[0x20]; + u8 register_data[0x20]; +}; + /* CQE format mask. */ #define MLX5E_CQE_FORMAT_MASK 0xc From patchwork Sun Jul 5 09:23:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73116 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 857B0A00C5; Sun, 5 Jul 2020 11:25:07 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 521DA1DAE3; Sun, 5 Jul 2020 11:24:40 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id C42E81DAD4 for ; Sun, 5 Jul 2020 11:24:37 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:36 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3p028191; Sun, 5 Jul 2020 12:24:35 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 5 Jul 2020 09:23:32 +0000 Message-Id: <1593941027-86651-6-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 05/20] common/mlx5: add support for regex capability query X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Update hca cap struct and common query hca cap function. Signed-off-by: Yuval Avnery Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++ drivers/common/mlx5/mlx5_prm.h | 9 +++++++-- 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index ec92eb6..74df035 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -467,6 +467,9 @@ struct mlx5_devx_obj * attr->vdpa.queue_counters_valid = !!(MLX5_GET64(cmd_hca_cap, hcattr, general_obj_types) & MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS); + attr->regex = MLX5_GET(cmd_hca_cap, hcattr, regexp); + attr->regexp_num_of_engines = MLX5_GET(cmd_hca_cap, hcattr, + regexp_num_of_engines); if (attr->qos.sup) { MLX5_SET(query_hca_cap_in, in, op_mod, MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP | diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 25704ef..bb14ca5 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -90,6 +90,8 @@ struct mlx5_hca_attr { uint32_t vhca_id:16; uint32_t relaxed_ordering_write:1; uint32_t relaxed_ordering_read:1; + uint32_t regex:1; + uint32_t regexp_num_of_engines; struct mlx5_hca_qos_attr qos; struct mlx5_hca_vdpa_attr vdpa; }; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index ede7810..bfbc58b 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -1034,9 +1034,14 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 log_max_qp_sz[0x8]; u8 reserved_at_90[0xb]; u8 log_max_qp[0x5]; - u8 reserved_at_a0[0xb]; + u8 regexp[0x1]; + u8 reserved_at_a1[0x3]; + u8 regexp_num_of_engines[0x4]; + u8 reserved_at_a8[0x3]; u8 log_max_srq[0x5]; - u8 reserved_at_b0[0x10]; + u8 reserved_at_b0[0x3]; + u8 regexp_log_crspace_size[0x5]; + u8 reserved_at_b8[0x8]; u8 reserved_at_c0[0x8]; u8 log_max_cq_sz[0x8]; u8 reserved_at_d0[0xb]; From patchwork Sun Jul 5 09:23:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73117 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C610DA00C5; Sun, 5 Jul 2020 11:25:16 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7D84D1DAF3; Sun, 5 Jul 2020 11:24:44 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id C5A481DAAC for ; Sun, 5 Jul 2020 11:24:42 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:38 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3q028191; Sun, 5 Jul 2020 12:24:38 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Thomas Monjalon Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, orika@mellanox.com, rasland@mellanox.com, Parav Pandit Date: Sun, 5 Jul 2020 09:23:33 +0000 Message-Id: <1593941027-86651-7-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 06/20] regex/mlx5: add probe function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the probe function to the RegEx PMD. Signed-off-by: Parav Pandit Signed-off-by: Ori Kam --- drivers/regex/mlx5/Makefile | 3 + drivers/regex/mlx5/meson.build | 2 +- drivers/regex/mlx5/mlx5_regex.c | 215 ++++++++++++++++++++++++++++++++++++++++ drivers/regex/mlx5/mlx5_regex.h | 6 ++ mk/rte.app.mk | 1 + 5 files changed, 226 insertions(+), 1 deletion(-) diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 40c03ab..c66df7f 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -16,6 +16,8 @@ CFLAGS += -g CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux +CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5/linux CFLAGS += -D_BSD_SOURCE CFLAGS += -D_DEFAULT_SOURCE CFLAGS += -D_XOPEN_SOURCE=600 @@ -26,6 +28,7 @@ LDLIBS += -lm LDLIBS += -lrte_eal -lrte_mbuf LDLIBS += -lrte_kvargs LDLIBS += -lrte_bus_pci +LDLIBS += -lrte_regexdev # A few warnings cannot be avoided in external headers. CFLAGS += -Wno-error=cast-qual diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index 768a9d1..d56c4da 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -8,7 +8,7 @@ if not is_linux endif fmt_name = 'mlx5_regex' -deps += ['common_mlx5', 'pci', 'bus_pci', 'eal', 'sched'] +deps += ['common_mlx5', 'pci', 'bus_pci', 'eal', 'sched', 'regexdev'] sources = files( 'mlx5_regex.c', ) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 06826a6..d264ecd 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -2,8 +2,223 @@ * Copyright 2020 Mellanox Technologies, Ltd */ +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + #include "mlx5_regex.h" #include "mlx5_regex_utils.h" int mlx5_regex_logtype; +static const struct rte_regexdev_ops mlx5_regexdev_ops = {}; + +static struct ibv_device * +mlx5_regex_get_ib_device_match(struct rte_pci_addr *addr) +{ + int n; + struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n); + struct ibv_device *ibv_match = NULL; + + if (!ibv_list) { + rte_errno = ENOSYS; + return NULL; + } + while (n-- > 0) { + struct rte_pci_addr pci_addr; + + DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name); + if (mlx5_dev_to_pci_addr(ibv_list[n]->ibdev_path, &pci_addr)) + continue; + if (rte_pci_addr_cmp(addr, &pci_addr)) + continue; + ibv_match = ibv_list[n]; + break; + } + if (!ibv_match) + rte_errno = ENOENT; + mlx5_glue->free_device_list(ibv_list); + return ibv_match; +} + +static void +mlx5_regex_get_name(char *name, struct rte_pci_device *pci_dev __rte_unused) +{ + sprintf(name, "mlx5_regex_%02x:%02x.%02x", pci_dev->addr.bus, + pci_dev->addr.devid, pci_dev->addr.function); +} + +/** + * DPDK callback to register a PCI device. + * + * This function spawns RegEx device out of a given PCI device. + * + * @param[in] pci_drv + * PCI driver structure (mlx5_regex_driver). + * @param[in] pci_dev + * PCI device information. + * + * @return + * 0 on success, 1 to skip this driver, a negative errno value otherwise + * and rte_errno is set. + */ +static int +mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) +{ + struct ibv_device *ibv; + struct mlx5_regex_priv *priv = NULL; + struct ibv_context *ctx = NULL; + struct mlx5_hca_attr attr; + char name[RTE_REGEXDEV_NAME_MAX_LEN]; + int ret; + + ibv = mlx5_regex_get_ib_device_match(&pci_dev->addr); + if (!ibv) { + DRV_LOG(ERR, "No matching IB device for PCI slot " + PCI_PRI_FMT ".", pci_dev->addr.domain, + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + return -rte_errno; + } + DRV_LOG(INFO, "PCI information matches for device \"%s\".", + ibv->name); + ctx = mlx5_glue->dv_open_device(ibv); + if (!ctx) { + DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); + rte_errno = ENODEV; + return -rte_errno; + } + ret = mlx5_devx_cmd_query_hca_attr(ctx, &attr); + if (ret) { + DRV_LOG(ERR, "Unable to read HCA capabilities."); + rte_errno = ENOTSUP; + goto error; + } else if (!attr.regex || attr.regexp_num_of_engines == 0) { + DRV_LOG(ERR, "Not enough capabilities to support RegEx, maybe " + "old FW/OFED version?"); + rte_errno = ENOTSUP; + goto error; + } + priv = rte_zmalloc("mlx5 regex device private", sizeof(*priv), + RTE_CACHE_LINE_SIZE); + if (!priv) { + DRV_LOG(ERR, "Failed to allocate private memory."); + rte_errno = ENOMEM; + goto error; + } + priv->ctx = ctx; + mlx5_regex_get_name(name, pci_dev); + priv->regexdev = rte_regexdev_register(name); + if (priv->regexdev == NULL) { + DRV_LOG(ERR, "Failed to register RegEx device."); + rte_errno = rte_errno ? rte_errno : EINVAL; + goto error; + } + priv->regexdev->dev_ops = &mlx5_regexdev_ops; + priv->regexdev->device = (struct rte_device *)pci_dev; + priv->regexdev->data->dev_private = priv; + return 0; + +error: + if (ctx) + mlx5_glue->close_device(ctx); + if (priv) + rte_free(priv); + return -rte_errno; +} + +/** + * DPDK callback to remove a PCI device. + * + * This function removes all RegEx devices belong to a given PCI device. + * + * @param[in] pci_dev + * Pointer to the PCI device. + * + * @return + * 0 on success, the function cannot fail. + */ +static int +mlx5_regex_pci_remove(struct rte_pci_device *pci_dev) +{ + char name[RTE_REGEXDEV_NAME_MAX_LEN]; + struct rte_regexdev *dev; + struct mlx5_regex_priv *priv = NULL; + + mlx5_regex_get_name(name, pci_dev); + dev = rte_regexdev_get_device_by_name(name); + if (!dev) + return 0; + priv = dev->data->dev_private; + if (priv) { + if (priv->ctx) + mlx5_glue->close_device(priv->ctx); + if (priv->regexdev) + rte_regexdev_unregister(priv->regexdev); + rte_free(priv); + } + return 0; +} + +static const struct rte_pci_id mlx5_regex_pci_id_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6) + }, + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6VF) + }, + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6DX) + }, + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6DXVF) + }, + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6DXBF) + }, + { + .vendor_id = 0 + } +}; + +static struct rte_pci_driver mlx5_regex_driver = { + .driver = { + .name = "mlx5_regex", + }, + .id_table = mlx5_regex_pci_id_map, + .probe = mlx5_regex_pci_probe, + .remove = mlx5_regex_pci_remove, + .drv_flags = 0, +}; + +/** + * Driver initialization routine. + */ +RTE_INIT(rte_mlx5_regex_init) +{ + /* Initialize common log type. */ + mlx5_regex_logtype = rte_log_register("pmd.regex.mlx5"); + if (mlx5_regex_logtype >= 0) + rte_log_set_level(mlx5_regex_logtype, RTE_LOG_NOTICE); + if (mlx5_glue) + rte_pci_register(&mlx5_regex_driver); +} + +RTE_PMD_EXPORT_NAME(net_mlx5_regex, __COUNTER__); +RTE_PMD_REGISTER_PCI_TABLE(net_mlx5_regex, mlx5_regex_pci_id_map); +RTE_PMD_REGISTER_KMOD_DEP(net_mlx5_regex, "* ib_uverbs & mlx5_core & mlx5_ib"); diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 0e0495c..0ce1e4d 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -5,4 +5,10 @@ #ifndef MLX5_REGEX_H #define MLX5_REGEX_H +struct mlx5_regex_priv { + TAILQ_ENTRY(mlx5_regex_priv) next; + struct ibv_context *ctx; /* Device context. */ + struct rte_pci_device *pci_dev; + struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ +}; #endif /* MLX5_REGEX_H */ diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 1b9551e..6efd7ae 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -101,6 +101,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched _LDLIBS-$(CONFIG_RTE_LIBRTE_RCU) += -lrte_rcu _LDLIBS-$(CONFIG_RTE_LIBRTE_GRAPH) += -lrte_graph _LDLIBS-$(CONFIG_RTE_LIBRTE_NODE) += -lrte_node +_LDLIBS-$(CONFIG_RTE_LIBRTE_REGEXDEV) += -lrte_regexdev ifeq ($(CONFIG_RTE_EXEC_ENV_LINUX),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_KNI) += -lrte_kni From patchwork Sun Jul 5 09:23:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73118 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 61433A00C5; Sun, 5 Jul 2020 11:25:27 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E4F181DAFF; Sun, 5 Jul 2020 11:24:45 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id D43E61DAF3 for ; Sun, 5 Jul 2020 11:24:42 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:40 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3r028191; Sun, 5 Jul 2020 12:24:40 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler , Ray Kinsella , Neil Horman Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 5 Jul 2020 09:23:34 +0000 Message-Id: <1593941027-86651-8-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 07/20] common/mlx5: add rxp database set cmd X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the database set command for the RXP engine. Signed-off-by: Ori Kam Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_devx_cmds.c | 104 ++++++++++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 8 +- drivers/common/mlx5/rte_common_mlx5_version.map | 3 + 3 files changed, 114 insertions(+), 1 deletion(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 74df035..f8760e6 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1616,3 +1616,107 @@ struct mlx5_devx_obj * invalid_buffer); return ret; } + +/** + * Issue the RXP stop database command. + * + * @param[in] ctx + * Context returned from mlx5 open_device() glue function. + * @param[in] engine + * The engine to stop. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_regex_database_stop(void *ctx, uint8_t engine) +{ + uint32_t out[DEVX_ST_SZ_DW(set_regexp_params_out)] = {}; + uint32_t in[DEVX_ST_SZ_DW(set_regexp_params_in)] = {}; + int ret; + + DEVX_SET(set_regexp_params_in, in, opcode, MLX5_CMD_SET_REGEX_PARAMS); + DEVX_SET(set_regexp_params_in, in, engine_id, engine); + DEVX_SET(set_regexp_params_in, in, regexp_params.stop_engine, 1); + DEVX_SET(set_regexp_params_in, in, field_select.stop_engine, 1); + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Database stop failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + +/** + * Issue the RXP resume database command. + * + * @param[in] ctx + * Context returned from mlx5 open_device() glue function. + * @param[in] engine + * The engine to start. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_regex_database_resume(void *ctx, uint8_t engine) +{ + uint32_t out[DEVX_ST_SZ_DW(set_regexp_params_out)] = {}; + uint32_t in[DEVX_ST_SZ_DW(set_regexp_params_in)] = {}; + int ret; + + DEVX_SET(set_regexp_params_in, in, opcode, MLX5_CMD_SET_REGEX_PARAMS); + DEVX_SET(set_regexp_params_in, in, engine_id, engine); + DEVX_SET(set_regexp_params_in, in, regexp_params.stop_engine, 0); + DEVX_SET(set_regexp_params_in, in, field_select.stop_engine, 1); + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Database start failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + +/** + * Issue the RXP program database command. + * + * @param[in] ctx + * Context returned from mlx5 open_device() glue function. + * @param[in] engine + * The engine to program. + * @param[in] umem_id + * The umem id to use. + * @param[in] umem_offset + * The offset in the umem to start copying from. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_regex_database_program(void *ctx, uint8_t engine, uint32_t umem_id, + uint64_t umem_offset) +{ + uint32_t out[DEVX_ST_SZ_DW(set_regexp_params_out)] = {}; + uint32_t in[DEVX_ST_SZ_DW(set_regexp_params_in)] = {}; + int ret; + + DEVX_SET(set_regexp_params_in, in, opcode, MLX5_CMD_SET_REGEX_PARAMS); + DEVX_SET(set_regexp_params_in, in, engine_id, engine); + DEVX_SET(set_regexp_params_in, in, regexp_params.db_umem_id, umem_id); + DEVX_SET64(set_regexp_params_in, in, regexp_params.db_umem_offset, + umem_offset); + DEVX_SET(set_regexp_params_in, in, field_select.db_umem_id, 1); + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Database program failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index bb14ca5..655e31f 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -401,5 +401,11 @@ int mlx5_devx_cmd_modify_rqt(struct mlx5_devx_obj *rqt, __rte_internal int mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj, struct mlx5_devx_virtio_q_couners_attr *attr); - +__rte_internal +int mlx5_devx_regex_database_stop(void *ctx, uint8_t engine); +__rte_internal +int mlx5_devx_regex_database_resume(void *ctx, uint8_t engine); +__rte_internal +int mlx5_devx_regex_database_program(void *ctx, uint8_t engine, + uint32_t umem_id, uint64_t umem_offset); #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */ diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map index ae57ebd..6054d39 100644 --- a/drivers/common/mlx5/rte_common_mlx5_version.map +++ b/drivers/common/mlx5/rte_common_mlx5_version.map @@ -35,6 +35,9 @@ INTERNAL { mlx5_devx_cmd_query_virtio_q_counters; mlx5_devx_cmd_query_virtq; mlx5_devx_get_out_command_status; + mlx5_devx_regex_database_program; + mlx5_devx_regex_database_resume; + mlx5_devx_regex_database_stop; mlx5_get_ifname_sysfs; mlx5_get_dbr; From patchwork Sun Jul 5 09:23:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73120 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CF5F1A00C5; Sun, 5 Jul 2020 11:25:50 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 07B071DB49; Sun, 5 Jul 2020 11:24:51 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id D2AE31DAD4 for ; Sun, 5 Jul 2020 11:24:47 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:43 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3s028191; Sun, 5 Jul 2020 12:24:43 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Parav Pandit Date: Sun, 5 Jul 2020 09:23:35 +0000 Message-Id: <1593941027-86651-9-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 08/20] regex/mlx5: add RXP register definitions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Parav Pandit This commit indroduce the mlx5_rxp_csrs.h file. This file holds all the relevant defines for the RXP engine. Signed-off-by: Parav Pandit Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_rxp_csrs.h | 338 +++++++++++++++++++++++++++++++++++++ 1 file changed, 338 insertions(+) create mode 100644 drivers/regex/mlx5/mlx5_rxp_csrs.h diff --git a/drivers/regex/mlx5/mlx5_rxp_csrs.h b/drivers/regex/mlx5/mlx5_rxp_csrs.h new file mode 100644 index 0000000..bab2946 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_rxp_csrs.h @@ -0,0 +1,338 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ +#ifndef _MLX5_RXP_CSRS_H_ +#define _MLX5_RXP_CSRS_H_ + +/* + * Common to all RXP implementations + */ +#define MLX5_RXP_CSR_BASE_ADDRESS 0x0000ul +#define MLX5_RXP_RTRU_CSR_BASE_ADDRESS 0x0100ul +#define MLX5_RXP_STATS_CSR_BASE_ADDRESS 0x0200ul +#define MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS 0x0600ul + +#define MLX5_RXP_CSR_WIDTH 4 + +/* This is the identifier we expect to see in the first RXP CSR */ +#define MLX5_RXP_IDENTIFER 0x5254 + +/* Hyperion specific BAR0 offsets */ +#define MLX5_RXP_FPGA_BASE_ADDRESS 0x0000ul +#define MLX5_RXP_PCIE_BASE_ADDRESS 0x1000ul +#define MLX5_RXP_IDMA_BASE_ADDRESS 0x2000ul +#define MLX5_RXP_EDMA_BASE_ADDRESS 0x3000ul +#define MLX5_RXP_SYSMON_BASE_ADDRESS 0xf300ul +#define MLX5_RXP_ISP_CSR_BASE_ADDRESS 0xf400ul + +/* Offset to the RXP common 4K CSR space */ +#define MLX5_RXP_PCIE_CSR_BASE_ADDRESS 0xf000ul + +/* FPGA CSRs */ + +#define MLX5_RXP_FPGA_VERSION (MLX5_RXP_FPGA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) + +/* PCIe CSRs */ +#define MLX5_RXP_PCIE_INIT_ISR (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_PCIE_INIT_IMR (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_PCIE_INIT_CFG_STAT (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +#define MLX5_RXP_PCIE_INIT_FLR (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 3) +#define MLX5_RXP_PCIE_INIT_CTRL (MLX5_RXP_PCIE_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 4) + +/* IDMA CSRs */ +#define MLX5_RXP_IDMA_ISR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_IDMA_IMR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_IDMA_CSR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 4) +#define MLX5_RXP_IDMA_CSR_RST_MSK 0x0001 +#define MLX5_RXP_IDMA_CSR_PDONE_MSK 0x0002 +#define MLX5_RXP_IDMA_CSR_INIT_MSK 0x0004 +#define MLX5_RXP_IDMA_CSR_EN_MSK 0x0008 +#define MLX5_RXP_IDMA_QCR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 5) +#define MLX5_RXP_IDMA_QCR_QAVAIL_MSK 0x00FF +#define MLX5_RXP_IDMA_QCR_QEN_MSK 0xFF00 +#define MLX5_RXP_IDMA_DCR (MLX5_RXP_IDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 6) +#define MLX5_RXP_IDMA_DWCTR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 7) +#define MLX5_RXP_IDMA_DWTOR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 8) +#define MLX5_RXP_IDMA_PADCR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 9) +#define MLX5_RXP_IDMA_DFCR (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_IDMA_FOFLR0 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_IDMA_FOFLR1 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_IDMA_FOFLR2 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +#define MLX5_RXP_IDMA_FUFLR0 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_IDMA_FUFLR1 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_IDMA_FUFLR2 (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) + +#define MLX5_RXP_IDMA_QCSR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 128) +#define MLX5_RXP_IDMA_QCSR_RST_MSK 0x0001 +#define MLX5_RXP_IDMA_QCSR_PDONE_MSK 0x0002 +#define MLX5_RXP_IDMA_QCSR_INIT_MSK 0x0004 +#define MLX5_RXP_IDMA_QCSR_EN_MSK 0x0008 +#define MLX5_RXP_IDMA_QDPTR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 192) +#define MLX5_RXP_IDMA_QTPTR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 256) +#define MLX5_RXP_IDMA_QDRPTR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 320) +#define MLX5_RXP_IDMA_QDRALR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 384) +#define MLX5_RXP_IDMA_QDRAHR_BASE (MLX5_RXP_IDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 385) + +/* EDMA CSRs */ +#define MLX5_RXP_EDMA_ISR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_EDMA_IMR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_EDMA_CSR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 4) +#define MLX5_RXP_EDMA_CSR_RST_MSK 0x0001 +#define MLX5_RXP_EDMA_CSR_PDONE_MSK 0x0002 +#define MLX5_RXP_EDMA_CSR_INIT_MSK 0x0004 +#define MLX5_RXP_EDMA_CSR_EN_MSK 0x0008 +#define MLX5_RXP_EDMA_QCR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 5) +#define MLX5_RXP_EDMA_QCR_QAVAIL_MSK 0x00FF +#define MLX5_RXP_EDMA_QCR_QEN_MSK 0xFF00 +#define MLX5_RXP_EDMA_DCR (MLX5_RXP_EDMA_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 6) +#define MLX5_RXP_EDMA_DWCTR (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 7) +#define MLX5_RXP_EDMA_DWTOR (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 8) +#define MLX5_RXP_EDMA_DFCR (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_EDMA_FOFLR0 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_EDMA_FOFLR1 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_EDMA_FOFLR2 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +#define MLX5_RXP_EDMA_FUFLR0 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_EDMA_FUFLR1 (MLX5_RXP_EDMA_BASE_ADDRESS +\ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_EDMA_FUFLR2 (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) + +#define MLX5_RXP_EDMA_QCSR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 128) +#define MLX5_RXP_EDMA_QCSR_RST_MSK 0x0001 +#define MLX5_RXP_EDMA_QCSR_PDONE_MSK 0x0002 +#define MLX5_RXP_EDMA_QCSR_INIT_MSK 0x0004 +#define MLX5_RXP_EDMA_QCSR_EN_MSK 0x0008 +#define MLX5_RXP_EDMA_QTPTR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 256) +#define MLX5_RXP_EDMA_QDRPTR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 320) +#define MLX5_RXP_EDMA_QDRALR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 384) +#define MLX5_RXP_EDMA_QDRAHR_BASE (MLX5_RXP_EDMA_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 385) + +/* Main CSRs */ +#define MLX5_RXP_CSR_IDENTIFIER (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_CSR_REVISION (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_CSR_CAPABILITY_0 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +#define MLX5_RXP_CSR_CAPABILITY_1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 3) +#define MLX5_RXP_CSR_CAPABILITY_2 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 4) +#define MLX5_RXP_CSR_CAPABILITY_3 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 5) +#define MLX5_RXP_CSR_CAPABILITY_4 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 6) +#define MLX5_RXP_CSR_CAPABILITY_5 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 7) +#define MLX5_RXP_CSR_CAPABILITY_6 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 8) +#define MLX5_RXP_CSR_CAPABILITY_7 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 9) +#define MLX5_RXP_CSR_STATUS (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_CSR_STATUS_INIT_DONE 0x0001 +#define MLX5_RXP_CSR_STATUS_GOING 0x0008 +#define MLX5_RXP_CSR_STATUS_IDLE 0x0040 +#define MLX5_RXP_CSR_STATUS_TRACKER_OK 0x0080 +#define MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT 0x0100 +#define MLX5_RXP_CSR_FIFO_STATUS_0 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 11) +#define MLX5_RXP_CSR_FIFO_STATUS_1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 12) +#define MLX5_RXP_CSR_JOB_DDOS_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 13) +/* 14 + 15 reserved */ +#define MLX5_RXP_CSR_CORE_CLK_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_CSR_WRITE_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_CSR_JOB_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +#define MLX5_RXP_CSR_JOB_ERROR_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 19) +#define MLX5_RXP_CSR_JOB_BYTE_COUNT0 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 20) +#define MLX5_RXP_CSR_JOB_BYTE_COUNT1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 21) +#define MLX5_RXP_CSR_RESPONSE_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 22) +#define MLX5_RXP_CSR_MATCH_COUNT (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 23) +#define MLX5_RXP_CSR_CTRL (MLX5_RXP_CSR_BASE_ADDRESS + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_CSR_CTRL_INIT 0x0001 +#define MLX5_RXP_CSR_CTRL_GO 0x0008 +#define MLX5_RXP_CSR_MAX_MATCH (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_CSR_MAX_PREFIX (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) +#define MLX5_RXP_CSR_MAX_PRI_THREAD (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 27) +#define MLX5_RXP_CSR_MAX_LATENCY (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 28) +#define MLX5_RXP_CSR_SCRATCH_1 (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 29) +#define MLX5_RXP_CSR_CLUSTER_MASK (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 30) +#define MLX5_RXP_CSR_INTRA_CLUSTER_MASK (MLX5_RXP_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 31) + +/* Runtime Rule Update CSRs */ +/* 0 + 1 reserved */ +#define MLX5_RXP_RTRU_CSR_CAPABILITY (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +/* 3-9 reserved */ +#define MLX5_RXP_RTRU_CSR_STATUS (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 10) +#define MLX5_RXP_RTRU_CSR_STATUS_UPDATE_DONE 0x0002 +#define MLX5_RXP_RTRU_CSR_STATUS_IM_INIT_DONE 0x0010 +#define MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE 0x0020 +#define MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE 0x0040 +#define MLX5_RXP_RTRU_CSR_STATUS_EM_INIT_DONE 0x0080 +#define MLX5_RXP_RTRU_CSR_FIFO_STAT (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 11) +/* 12-15 reserved */ +#define MLX5_RXP_RTRU_CSR_CHECKSUM_0 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 16) +#define MLX5_RXP_RTRU_CSR_CHECKSUM_1 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 17) +#define MLX5_RXP_RTRU_CSR_CHECKSUM_2 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 18) +/* 19 + 20 reserved */ +#define MLX5_RXP_RTRU_CSR_RTRU_COUNT (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 21) +#define MLX5_RXP_RTRU_CSR_ROF_REV (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 22) +/* 23 reserved */ +#define MLX5_RXP_RTRU_CSR_CTRL (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT 0x0001 +#define MLX5_RXP_RTRU_CSR_CTRL_GO 0x0002 +#define MLX5_RXP_RTRU_CSR_CTRL_SIP 0x0004 +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_MASK (3 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2_EM (0 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2 (1 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2 (2 << 4) +#define MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_EM (3 << 4) +#define MLX5_RXP_RTRU_CSR_ADDR (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_RTRU_CSR_DATA_0 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 26) +#define MLX5_RXP_RTRU_CSR_DATA_1 (MLX5_RXP_RTRU_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 27) +/* 28-31 reserved */ + +/* Statistics CSRs */ +#define MLX5_RXP_STATS_CSR_CLUSTER (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_STATS_CSR_L2_CACHE (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 24) +#define MLX5_RXP_STATS_CSR_MPFE_FIFO (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 25) +#define MLX5_RXP_STATS_CSR_PE (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 28) +#define MLX5_RXP_STATS_CSR_CP (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 30) +#define MLX5_RXP_STATS_CSR_DP (MLX5_RXP_STATS_CSR_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 31) + +/* Sysmon Stats CSRs */ +#define MLX5_RXP_SYSMON_CSR_T_FPGA (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 0) +#define MLX5_RXP_SYSMON_CSR_V_VCCINT (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 1) +#define MLX5_RXP_SYSMON_CSR_V_VCCAUX (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 2) +#define MLX5_RXP_SYSMON_CSR_T_U1 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 20) +#define MLX5_RXP_SYSMON_CSR_I_EDG12V (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 21) +#define MLX5_RXP_SYSMON_CSR_I_VCC3V3 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 22) +#define MLX5_RXP_SYSMON_CSR_I_VCC2V5 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 23) +#define MLX5_RXP_SYSMON_CSR_T_U2 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 28) +#define MLX5_RXP_SYSMON_CSR_I_AUX12V (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 29) +#define MLX5_RXP_SYSMON_CSR_I_VCC1V8 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 30) +#define MLX5_RXP_SYSMON_CSR_I_VDDR3 (MLX5_RXP_SYSMON_BASE_ADDRESS + \ + MLX5_RXP_CSR_WIDTH * 31) + +/* In Service Programming CSRs */ + +/* RXP-F1 and RXP-ZYNQ specific CSRs */ +#define MLX5_RXP_MQ_CP_BASE (0x0500ul) +#define MLX5_RXP_MQ_CP_CAPABILITY_BASE (MLX5_RXP_MQ_CP_BASE + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_0 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 0 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_1 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 1 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_2 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_CAPABILITY_3 (MLX5_RXP_MQ_CP_CAPABILITY_BASE + \ + 3 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_BASE (MLX5_RXP_MQ_CP_BASE + \ + 11 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C0 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 0 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C1 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 1 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C2 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXP_MQ_CP_FIFO_STATUS_C3 (MLX5_RXP_MQ_CP_FIFO_STATUS_BASE + \ + 3 * MLX5_RXP_CSR_WIDTH) + +/* Royalty tracker / licensing related CSRs */ +#define MLX5_RXPL__CSR_IDENT (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 0 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__IDENTIFIER 0x4c505852 /* MLX5_RXPL_ */ +#define MLX5_RXPL__CSR_CAPABILITY (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 2 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__TYPE_MASK 0xFF +#define MLX5_RXPL__TYPE_NONE 0 +#define MLX5_RXPL__TYPE_MAXIM 1 +#define MLX5_RXPL__TYPE_XILINX_DNA 2 +#define MLX5_RXPL__CSR_STATUS (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 10 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__CSR_IDENT_0 (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 16 * MLX5_RXP_CSR_WIDTH) +#define MLX5_RXPL__CSR_KEY_0 (MLX5_RXP_ROYALTY_CSR_BASE_ADDRESS + \ + 24 * MLX5_RXP_CSR_WIDTH) + +#endif /* _MLX5_RXP_CSRS_H_ */ From patchwork Sun Jul 5 09:23:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73119 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E1149A00C5; Sun, 5 Jul 2020 11:25:39 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8D9C21DB26; Sun, 5 Jul 2020 11:24:49 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id D54EF1DB40 for ; Sun, 5 Jul 2020 11:24:47 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:46 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3t028191; Sun, 5 Jul 2020 12:24:45 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler , Ray Kinsella , Neil Horman Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 5 Jul 2020 09:23:36 +0000 Message-Id: <1593941027-86651-10-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 09/20] common/mlx5: add write and read RXP registers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commits add the write and read RXP registers functionality. Signed-off-by: Ori Kam Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_devx_cmds.c | 78 +++++++++++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 10 ++++ drivers/common/mlx5/rte_common_mlx5_version.map | 2 + 3 files changed, 90 insertions(+) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index f8760e6..4fad7cd 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1720,3 +1720,81 @@ struct mlx5_devx_obj * return 0; } +/** + * Write to RXP registers. + * + * @param ctx + * ibv device handle + * @param engine_id + * Chooses on which engine the register will be written.. + * @param addr + * Register address. + * @param data + * Data to be written to the register. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_regex_register_write(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t data) +{ + uint32_t out[DEVX_ST_SZ_DW(set_regexp_register_out)] = {}; + uint32_t in[DEVX_ST_SZ_DW(set_regexp_register_in)] = {}; + int ret; + + DEVX_SET(set_regexp_register_in, in, opcode, + MLX5_CMD_SET_REGEX_REGISTERS); + DEVX_SET(set_regexp_register_in, in, engine_id, engine_id); + DEVX_SET(set_regexp_register_in, in, register_address, addr); + DEVX_SET(set_regexp_register_in, in, register_data, data); + + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Set regexp register failed %d", ret); + rte_errno = errno; + return -errno; + } + return 0; +} + + +/** + * Read from RXP registers + * + * @param ctx + * ibv device handle + * @param engine_id + * Chooses from which engine to read. + * @param addr + * Register address. + * @param data + * Output containing the pointer to the data.. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_regex_register_read(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t *data) +{ + uint32_t out[DEVX_ST_SZ_DW(query_regexp_register_out)] = {}; + uint32_t in[DEVX_ST_SZ_DW(query_regexp_register_in)] = {}; + int ret; + + DEVX_SET(query_regexp_register_in, in, opcode, + MLX5_CMD_QUERY_REGEX_REGISTERS); + DEVX_SET(query_regexp_register_in, in, engine_id, engine_id); + DEVX_SET(query_regexp_register_in, in, register_address, addr); + + ret = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, + sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Query regexp register failed %d", ret); + rte_errno = errno; + return -errno; + } + *data = DEVX_GET(query_regexp_register_out, out, register_data); + return 0; +} diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 655e31f..a2a9045 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -374,6 +374,10 @@ int mlx5_devx_cmd_modify_qp_state(struct mlx5_devx_obj *qp, __rte_internal int mlx5_devx_cmd_modify_rqt(struct mlx5_devx_obj *rqt, struct mlx5_devx_rqt_attr *rqt_attr); +int mlx5_devx_regex_register_write(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t data); +int mlx5_devx_regex_register_read(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t *data); /** * Create virtio queue counters object DevX API. @@ -408,4 +412,10 @@ int mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj, __rte_internal int mlx5_devx_regex_database_program(void *ctx, uint8_t engine, uint32_t umem_id, uint64_t umem_offset); +__rte_internal +int mlx5_devx_regex_register_read(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t *data); +__rte_internal +int mlx5_devx_regex_register_write(struct ibv_context *ctx, int engine_id, + uint32_t addr, uint32_t data); #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */ diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map index 6054d39..138719d 100644 --- a/drivers/common/mlx5/rte_common_mlx5_version.map +++ b/drivers/common/mlx5/rte_common_mlx5_version.map @@ -38,6 +38,8 @@ INTERNAL { mlx5_devx_regex_database_program; mlx5_devx_regex_database_resume; mlx5_devx_regex_database_stop; + mlx5_devx_regex_register_read; + mlx5_devx_regex_register_write; mlx5_get_ifname_sysfs; mlx5_get_dbr; From patchwork Sun Jul 5 09:23:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73121 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9645BA00C5; Sun, 5 Jul 2020 11:26:02 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 910EC1DB4D; Sun, 5 Jul 2020 11:24:54 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id E22491DB4D for ; Sun, 5 Jul 2020 11:24:52 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:48 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3u028191; Sun, 5 Jul 2020 12:24:48 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 5 Jul 2020 09:23:37 +0000 Message-Id: <1593941027-86651-11-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 10/20] regex/mlx5: add engine status check X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit checks the engine status. Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index d264ecd..c469a10 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -17,6 +17,7 @@ #include "mlx5_regex.h" #include "mlx5_regex_utils.h" +#include "mlx5_rxp_csrs.h" int mlx5_regex_logtype; @@ -49,6 +50,28 @@ mlx5_glue->free_device_list(ibv_list); return ibv_match; } +static int +mlx5_regex_engines_status(struct ibv_context *ctx, int num_engines) +{ + uint32_t fpga_ident = 0; + int err; + int i; + + for (i = 0; i < num_engines; i++) { + err = mlx5_devx_regex_register_read(ctx, i, + MLX5_RXP_CSR_IDENTIFIER, + &fpga_ident); + fpga_ident = (fpga_ident & (0x0000FFFF)); + if (err || fpga_ident != MLX5_RXP_IDENTIFER) { + DRV_LOG(ERR, "Failed setup RXP %d err %d database " + "memory 0x%x", i, err, fpga_ident); + if (!err) + err = EINVAL; + return err; + } + } + return 0; +} static void mlx5_regex_get_name(char *name, struct rte_pci_device *pci_dev __rte_unused) @@ -109,6 +132,11 @@ rte_errno = ENOTSUP; goto error; } + if (mlx5_regex_engines_status(ctx, 2)) { + DRV_LOG(ERR, "RegEx engine error."); + rte_errno = ENOMEM; + goto error; + } priv = rte_zmalloc("mlx5 regex device private", sizeof(*priv), RTE_CACHE_LINE_SIZE); if (!priv) { From patchwork Sun Jul 5 09:23:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73122 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E275EA00C5; Sun, 5 Jul 2020 11:26:09 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E81B91DB5F; Sun, 5 Jul 2020 11:24:55 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id E75CF1DB5C for ; Sun, 5 Jul 2020 11:24:52 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:50 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3v028191; Sun, 5 Jul 2020 12:24:50 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 5 Jul 2020 09:23:38 +0000 Message-Id: <1593941027-86651-12-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 11/20] regex/mlx5: add get info function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the get info function. Signed-off-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/meson.build | 1 + drivers/regex/mlx5/mlx5_regex.c | 5 ++++- drivers/regex/mlx5/mlx5_regex.h | 5 +++++ drivers/regex/mlx5/mlx5_rxp.c | 41 +++++++++++++++++++++++++++++++++++++++++ 5 files changed, 52 insertions(+), 1 deletion(-) create mode 100644 drivers/regex/mlx5/mlx5_rxp.c diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index c66df7f..1ad5125 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -8,6 +8,7 @@ LIB = librte_pmd_mlx5_regex.a # Sources. SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_rxp.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index d56c4da..cb092ad 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -11,6 +11,7 @@ fmt_name = 'mlx5_regex' deps += ['common_mlx5', 'pci', 'bus_pci', 'eal', 'sched', 'regexdev'] sources = files( 'mlx5_regex.c', + 'mlx5_rxp.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index c469a10..2c4b7ce 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -21,7 +21,10 @@ int mlx5_regex_logtype; -static const struct rte_regexdev_ops mlx5_regexdev_ops = {}; +const struct rte_regexdev_ops mlx5_regexdev_ops = { + .dev_info_get = mlx5_regex_info_get, +}; + static struct ibv_device * mlx5_regex_get_ib_device_match(struct rte_pci_addr *addr) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 0ce1e4d..9d0fc16 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -11,4 +11,9 @@ struct mlx5_regex_priv { struct rte_pci_device *pci_dev; struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ }; + +/* mlx5_rxp.c */ +int mlx5_regex_info_get(struct rte_regexdev *dev, + struct rte_regexdev_info *info); + #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c new file mode 100644 index 0000000..12d55ed --- /dev/null +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include +#include +#include +#include +#include + +#include "mlx5_regex.h" + +#define MLX5_REGEX_MAX_MATCHES 255 +#define MLX5_REGEX_MAX_PAYLOAD_SIZE UINT16_MAX +#define MLX5_REGEX_MAX_RULES_PER_GROUP UINT16_MAX +#define MLX5_REGEX_MAX_GROUPS UINT16_MAX + +/** + * DPDK callback for reading device info. + * + * @param dev + * Pointer to RegEx device structure. + * @param[out] info + * Pointer to the regexdev info structure to be filled with the contextual + * information of the device. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_regex_info_get(struct rte_regexdev *dev __rte_unused, + struct rte_regexdev_info *info) +{ + info->max_matches = MLX5_REGEX_MAX_MATCHES; + info->max_payload_size = MLX5_REGEX_MAX_PAYLOAD_SIZE; + info->max_rules_per_group = MLX5_REGEX_MAX_RULES_PER_GROUP; + info->max_groups = MLX5_REGEX_MAX_GROUPS; + info->regexdev_capa = RTE_REGEXDEV_SUPP_PCRE_GREEDY_F; + info->rule_flags = 0; + return 0; +} From patchwork Sun Jul 5 09:23:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73123 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA929A00C5; Sun, 5 Jul 2020 11:26:17 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 20C4B1DB71; Sun, 5 Jul 2020 11:24:57 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id F34F01DB5D for ; Sun, 5 Jul 2020 11:24:52 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:52 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3w028191; Sun, 5 Jul 2020 12:24:51 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 5 Jul 2020 09:23:39 +0000 Message-Id: <1593941027-86651-13-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 12/20] regex/mlx5: add configure function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit implements the configure function. This function is responsible to configure the RegEx engine. Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 2 + drivers/regex/mlx5/mlx5_regex.h | 15 +++ drivers/regex/mlx5/mlx5_rxp.c | 279 +++++++++++++++++++++++++++++++++++++++- 3 files changed, 295 insertions(+), 1 deletion(-) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 2c4b7ce..94e4352 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -23,6 +23,7 @@ const struct rte_regexdev_ops mlx5_regexdev_ops = { .dev_info_get = mlx5_regex_info_get, + .dev_configure = mlx5_regex_configure, }; @@ -158,6 +159,7 @@ priv->regexdev->dev_ops = &mlx5_regexdev_ops; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; + priv->regexdev->state = RTE_REGEXDEV_READY; return 0; error: diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 9d0fc16..5238f24 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -5,15 +5,30 @@ #ifndef MLX5_REGEX_H #define MLX5_REGEX_H +struct mlx5_regex_sq { + uint32_t nb_desc; /* Number of desc for this object. */ +}; + +struct mlx5_regex_qp { + uint32_t flags; /* QP user flags. */ + uint32_t nb_desc; /* Total number of desc for thsi qp. */ + struct mlx5_regex_sq *sqs; /* Pointer to sq array. */ +}; + struct mlx5_regex_priv { TAILQ_ENTRY(mlx5_regex_priv) next; struct ibv_context *ctx; /* Device context. */ struct rte_pci_device *pci_dev; struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ + uint16_t nb_queues; /* Number of queues. */ + struct mlx5_regex_qp *qps; /* Pointer to the qp array. */ + uint16_t nb_max_matches; /* Max number of matches. */ }; /* mlx5_rxp.c */ int mlx5_regex_info_get(struct rte_regexdev *dev, struct rte_regexdev_info *info); +int mlx5_regex_configure(struct rte_regexdev *dev, + const struct rte_regexdev_config *cfg); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c index 12d55ed..60a4640 100644 --- a/drivers/regex/mlx5/mlx5_rxp.c +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -2,13 +2,22 @@ * Copyright 2020 Mellanox Technologies, Ltd */ +#include + #include #include +#include #include #include #include +#include +#include +#include + #include "mlx5_regex.h" +#include "mlx5_regex_utils.h" +#include "mlx5_rxp_csrs.h" #define MLX5_REGEX_MAX_MATCHES 255 #define MLX5_REGEX_MAX_PAYLOAD_SIZE UINT16_MAX @@ -29,7 +38,7 @@ */ int mlx5_regex_info_get(struct rte_regexdev *dev __rte_unused, - struct rte_regexdev_info *info) + struct rte_regexdev_info *info) { info->max_matches = MLX5_REGEX_MAX_MATCHES; info->max_payload_size = MLX5_REGEX_MAX_PAYLOAD_SIZE; @@ -39,3 +48,271 @@ info->rule_flags = 0; return 0; } + +static int +rxp_poll_csr_for_value(struct ibv_context *ctx, uint32_t *value, + uint32_t address, uint32_t expected_value, + uint32_t expected_mask, uint32_t timeout_ms, uint8_t id) +{ + unsigned int i; + int ret; + + ret = -EBUSY; + for (i = 0; i < timeout_ms; i++) { + if (mlx5_devx_regex_register_read(ctx, id, address, value)) + return -1; + + if ((*value & expected_mask) == expected_value) { + ret = 0; + break; + } + rte_delay_us(1000); + } + return ret; +} + +/** + * Start the selected engine. + * + * @param ctx + * The IBV context. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +rxp_start_engine(struct ibv_context *ctx, uint8_t id) +{ + uint32_t ctrl; + int ret; + + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + ctrl |= MLX5_RXP_CSR_CTRL_GO; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + return ret; +} + +/** + * Stop the selected engine. + * + * @param ctx + * The IBV context. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +rxp_stop_engine(struct ibv_context *ctx, uint8_t id) +{ + uint32_t ctrl; + int ret; + + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + ctrl &= ~MLX5_RXP_CSR_CTRL_GO; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + return ret; +} + +static int +rxp_init_rtru(struct ibv_context *ctx, uint8_t id, uint32_t init_bits) +{ + uint32_t ctrl_value; + uint32_t poll_value; + uint32_t expected_value; + uint32_t expected_mask; + int ret = 0; + + /* Read the rtru ctrl CSR */ + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + &ctrl_value); + if (ret) + return -1; + /* Clear any previous init modes */ + ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_MASK); + if (ctrl_value & MLX5_RXP_RTRU_CSR_CTRL_INIT) { + ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + } + /* Set the init_mode bits in the rtru ctrl CSR */ + ctrl_value |= init_bits; + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + /* Need to sleep for a short period after pulsing the rtru init bit. */ + rte_delay_us(20000); + /* Poll the rtru status CSR until all the init done bits are set. */ + DRV_LOG(DEBUG, "waiting for RXP rule memory to complete init"); + /* Set the init bit in the rtru ctrl CSR. */ + ctrl_value |= MLX5_RXP_RTRU_CSR_CTRL_INIT; + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + /* Clear the init bit in the rtru ctrl CSR */ + ctrl_value &= ~MLX5_RXP_RTRU_CSR_CTRL_INIT; + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + /* Check that the following bits are set in the RTRU_CSR. */ + if (init_bits == MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2) { + /* Must be incremental mode */ + expected_value = MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + } else { + expected_value = MLX5_RXP_RTRU_CSR_STATUS_IM_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + } + expected_mask = expected_value; + ret = rxp_poll_csr_for_value(ctx, &poll_value, + MLX5_RXP_RTRU_CSR_STATUS, + expected_value, expected_mask, + MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT, id); + if (ret) + return ret; + DRV_LOG(DEBUG, "rule Memory initialise: 0x%08X", poll_value); + /* Clear the init bit in the rtru ctrl CSR */ + ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); + mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + ctrl_value); + return 0; +} + +/** + * Init the engine. + * + * @param ctx + * The IBV context. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +rxp_init(struct mlx5_regex_priv *priv, uint8_t id) +{ + uint32_t ctrl; + uint32_t reg; + struct ibv_context *ctx = priv->ctx; + int ret; + + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + if (ctrl & MLX5_RXP_CSR_CTRL_INIT) { + ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, + ctrl); + if (ret) + return ret; + } + ctrl |= MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + if (ret) + return ret; + ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); + rte_delay_us(20000); + + ret = rxp_poll_csr_for_value(ctx, &ctrl, MLX5_RXP_CSR_STATUS, + MLX5_RXP_CSR_STATUS_INIT_DONE, + MLX5_RXP_CSR_STATUS_INIT_DONE, + MLX5_RXP_CSR_STATUS_TRIAL_TIMEOUT, id); + if (ret) + return ret; + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CTRL, &ctrl); + if (ret) + return ret; + ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, + ctrl); + if (ret) + return ret; + rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); + ret = rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); + if (ret) + return ret; + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_CSR_CAPABILITY_5, + ®); + if (ret) + return ret; + DRV_LOG(DEBUG, "max matches: %d, DDOS threshold: %d", reg >> 16, + reg & 0xffff); + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_MATCH, + priv->nb_max_matches); + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_LATENCY, 0); + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_PRI_THREAD, 0); + return ret; +} + +/** + * DPDK callback for reading device info. + * + * @param dev + * Pointer to RegEx device structure. + * @param[in] cfg + * Pointer to the regexdev device configuration structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_regex_configure(struct rte_regexdev *dev, + const struct rte_regexdev_config *cfg) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + int ret; + uint8_t id; + + priv->nb_queues = cfg->nb_queue_pairs; + priv->qps = rte_zmalloc(NULL, sizeof(struct mlx5_regex_qp) * + priv->nb_queues, 0); + if (!priv->nb_queues) { + DRV_LOG(ERR, "can't allocate qps memory"); + rte_errno = ENOMEM; + return -rte_errno; + } + priv->nb_max_matches = cfg->nb_max_matches; + for (id = 0; id < 2; id++) { + ret = rxp_stop_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "can't stop engine."); + rte_errno = ENODEV; + return -rte_errno; + } + ret = rxp_init(priv, id); + if (ret) { + DRV_LOG(ERR, "can't init engine."); + rte_errno = ENODEV; + return -rte_errno; + } + ret = mlx5_devx_regex_register_write(priv->ctx, id, + MLX5_RXP_CSR_MAX_MATCH, + priv->nb_max_matches); + if (ret) { + DRV_LOG(ERR, "can't update number of matches."); + rte_errno = ENODEV; + goto configure_error; + } + ret = rxp_start_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "can't start engine."); + rte_errno = ENODEV; + goto configure_error; + } + + } + return 0; +configure_error: + if (priv->qps) + rte_free(priv->qps); + return -rte_errno; +} From patchwork Sun Jul 5 09:23:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73125 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DD4D1A00C5; Sun, 5 Jul 2020 11:26:35 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6617D1DB8F; Sun, 5 Jul 2020 11:25:00 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id F228B1DB52 for ; Sun, 5 Jul 2020 11:24:57 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:54 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC3x028191; Sun, 5 Jul 2020 12:24:53 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Francis Kelly Date: Sun, 5 Jul 2020 09:23:40 +0000 Message-Id: <1593941027-86651-14-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 13/20] regex/mlx5: add program rules support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Francis Kelly This commit introduce the ability to program rules to the RegEx engine. Signed-off-by: Francis Kelly Acked-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 33 ++ drivers/regex/mlx5/mlx5_regex.h | 55 ++- drivers/regex/mlx5/mlx5_rxp.c | 1039 +++++++++++++++++++++++++++++++++++++-- drivers/regex/mlx5/mlx5_rxp.h | 138 ++++++ 4 files changed, 1216 insertions(+), 49 deletions(-) create mode 100644 drivers/regex/mlx5/mlx5_rxp.h diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 94e4352..8801cd7 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -24,6 +24,7 @@ const struct rte_regexdev_ops mlx5_regexdev_ops = { .dev_info_get = mlx5_regex_info_get, .dev_configure = mlx5_regex_configure, + .dev_db_import = mlx5_regex_rules_db_import, }; @@ -149,6 +150,8 @@ goto error; } priv->ctx = ctx; + /* Default RXP programming mode to Shared. */ + priv->prog_mode = MLX5_RXP_SHARED_PROG_MODE; mlx5_regex_get_name(name, pci_dev); priv->regexdev = rte_regexdev_register(name); if (priv->regexdev == NULL) { @@ -156,6 +159,24 @@ rte_errno = rte_errno ? rte_errno : EINVAL; goto error; } + ret = mlx5_glue->devx_query_eqn(ctx, 0, &priv->eqn); + if (ret) { + DRV_LOG(ERR, "can't query event queue number."); + rte_errno = ENOMEM; + goto error; + } + priv->uar = mlx5_glue->devx_alloc_uar(ctx, 0); + if (!priv->uar) { + DRV_LOG(ERR, "can't allocate uar."); + rte_errno = ENOMEM; + goto error; + } + priv->pd = mlx5_glue->alloc_pd(ctx); + if (!priv->pd) { + DRV_LOG(ERR, "can't allocate pd."); + rte_errno = ENOMEM; + goto error; + } priv->regexdev->dev_ops = &mlx5_regexdev_ops; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; @@ -163,6 +184,12 @@ return 0; error: + if (priv->pd) + mlx5_glue->dealloc_pd(priv->pd); + if (priv->uar) + mlx5_glue->devx_free_uar(priv->uar); + if (priv->regexdev) + rte_regexdev_unregister(priv->regexdev); if (ctx) mlx5_glue->close_device(ctx); if (priv) @@ -194,6 +221,12 @@ return 0; priv = dev->data->dev_private; if (priv) { + if (priv->pd) + mlx5_glue->dealloc_pd(priv->pd); + if (priv->uar) + mlx5_glue->devx_free_uar(priv->uar); + if (priv->regexdev) + rte_regexdev_unregister(priv->regexdev); if (priv->ctx) mlx5_glue->close_device(priv->ctx); if (priv->regexdev) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 5238f24..fca9ebb 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -5,14 +5,53 @@ #ifndef MLX5_REGEX_H #define MLX5_REGEX_H +#ifdef PEDANTIC +#pragma GCC diagnostic ignored "-Wpedantic" +#endif +#include +#include +#ifdef PEDANTIC +#pragma GCC diagnostic error "-Wpedantic" +#endif + +#include + +#include "mlx5_rxp.h" + struct mlx5_regex_sq { - uint32_t nb_desc; /* Number of desc for this object. */ + uint16_t log_nb_desc; /* Log 2 number of desc for this object. */ + struct mlx5_devx_obj *obj; /* The SQ DevX object. */ + int64_t dbr_offset; /* Door bell record offset. */ + uint32_t dbr_umem; /* Door bell record umem id. */ + volatile struct mlx5_cqe *wqe; /* The SQ ring buffer. */ + struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ +}; + +struct mlx5_regex_cq { + uint32_t log_nb_desc; /* Log 2 number of desc for this object. */ + struct mlx5_devx_obj *obj; /* The CQ DevX object. */ + int64_t dbr_offset; /* Door bell record offset. */ + uint32_t dbr_umem; /* Door bell record umem id. */ + volatile struct mlx5_cqe *cqe; /* The CQ ring buffer. */ + struct mlx5dv_devx_umem *cqe_umem; /* CQ buffer umem. */ }; struct mlx5_regex_qp { uint32_t flags; /* QP user flags. */ - uint32_t nb_desc; /* Total number of desc for thsi qp. */ + uint16_t nb_desc; /* Total number of desc for this qp. */ struct mlx5_regex_sq *sqs; /* Pointer to sq array. */ + uint16_t nb_obj; /* Number of sq objects. */ + struct mlx5_regex_cq cq; /* CQ struct. */ +}; + +struct mlx5_regex_db { + void *ptr; /* Pointer to the db memory. */ + uint32_t len; /* The memory len. */ + bool active; /* Active flag. */ + uint8_t db_assigned_to_eng_num; + /**< To which engine the db is connected. */ + struct mlx5_regex_umem umem; + /**< The umem struct. */ }; struct mlx5_regex_priv { @@ -23,6 +62,13 @@ struct mlx5_regex_priv { uint16_t nb_queues; /* Number of queues. */ struct mlx5_regex_qp *qps; /* Pointer to the qp array. */ uint16_t nb_max_matches; /* Max number of matches. */ + enum mlx5_rxp_program_mode prog_mode; + struct mlx5_regex_db db[MLX5_RXP_MAX_ENGINES + + MLX5_RXP_EM_COUNT]; + struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ + uint32_t eqn; /* EQ number. */ + struct mlx5dv_devx_uar *uar; /* UAR object. */ + struct ibv_pd *pd; }; /* mlx5_rxp.c */ @@ -30,5 +76,8 @@ int mlx5_regex_info_get(struct rte_regexdev *dev, struct rte_regexdev_info *info); int mlx5_regex_configure(struct rte_regexdev *dev, const struct rte_regexdev_config *cfg); - +int mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, + const struct rte_regexdev_qp_conf *cfg); +int mlx5_regex_rules_db_import(struct rte_regexdev *dev, + const char *rule_db, uint32_t rule_db_len); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c index 60a4640..2bb3229 100644 --- a/drivers/regex/mlx5/mlx5_rxp.c +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -3,6 +3,7 @@ */ #include +#include #include #include @@ -14,15 +15,113 @@ #include #include #include +#include #include "mlx5_regex.h" #include "mlx5_regex_utils.h" #include "mlx5_rxp_csrs.h" +#include "mlx5_rxp.h" -#define MLX5_REGEX_MAX_MATCHES 255 -#define MLX5_REGEX_MAX_PAYLOAD_SIZE UINT16_MAX -#define MLX5_REGEX_MAX_RULES_PER_GROUP UINT16_MAX -#define MLX5_REGEX_MAX_GROUPS UINT16_MAX +/* TODO: get actual VIPER version capability bit when available! */ +#define VIPER_BF2 1 + +#define MLX5_REGEX_MAX_MATCHES MLX5_RXP_MAX_MATCHES +#define MLX5_REGEX_MAX_PAYLOAD_SIZE MLX5_RXP_MAX_JOB_LENGTH +#define MLX5_REGEX_MAX_RULES_PER_GROUP UINT32_MAX +#define MLX5_REGEX_MAX_GROUPS MLX5_RXP_MAX_SUBSETS + +/* Private Declarations */ +static int +rxp_poll_csr_for_value(struct ibv_context *ctx, uint32_t *value, + uint32_t address, uint32_t expected_value, + uint32_t expected_mask, uint32_t timeout_ms, uint8_t id); +static int +mlnx_set_database(struct mlx5_regex_priv *priv, uint8_t id, uint8_t db_to_use); +static int +mlnx_resume_database(struct mlx5_regex_priv *priv, uint8_t id); +static int +mlnx_update_database(struct mlx5_regex_priv *priv, uint8_t id); +static int +program_rxp_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint8_t id); +static int +rxp_init_eng(struct mlx5_regex_priv *priv, uint8_t id); +static int +write_private_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, + uint8_t id); +static int +write_shared_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint32_t count, + uint8_t db_to_program); +static int +rxp_db_setup(struct mlx5_regex_priv *priv); +static void +rxp_dump_csrs(struct ibv_context *ctx, uint8_t id); +static int +rxp_write_rules_via_cp(struct ibv_context *ctx, + struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id); +static int +rxp_flush_rules(struct ibv_context *ctx, struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id); +static int +rxp_start_engine(struct ibv_context *ctx, uint8_t id); +static int +rxp_stop_engine(struct ibv_context *ctx, uint8_t id); + +/** + * RXP Register display function + * + * @param ctx + * The IBV context. + * @param id + * The selected engine to read back registers. + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static void __rte_unused +rxp_dump_csrs(struct ibv_context *ctx __rte_unused, uint8_t id __rte_unused) +{ + uint32_t reg, i; + + /* Main CSRs*/ + for (i = 0; i < MLX5_RXP_CSR_NUM_ENTRIES; i++) { + if (mlx5_devx_regex_register_read(ctx, id, + (MLX5_RXP_CSR_WIDTH * i) + + MLX5_RXP_CSR_BASE_ADDRESS, + ®)) { + DRV_LOG(ERR, "Failed to read Main CSRs Engine %d!", id); + return; + } + DRV_LOG(DEBUG, "RXP Main CSRs (Eng%d) register (%d): %08x", + id, i, reg); + } + /* RTRU CSRs*/ + for (i = 0; i < MLX5_RXP_CSR_NUM_ENTRIES; i++) { + if (mlx5_devx_regex_register_read(ctx, id, + (MLX5_RXP_CSR_WIDTH * i) + + MLX5_RXP_RTRU_CSR_BASE_ADDRESS, + ®)) { + DRV_LOG(ERR, "Failed to read RTRU CSRs Engine %d!", id); + return; + } + DRV_LOG(DEBUG, "RXP RTRU CSRs (Eng%d) register (%d): %08x", + id, i, reg); + } + /* STAT CSRs */ + for (i = 0; i < MLX5_RXP_CSR_NUM_ENTRIES; i++) { + if (mlx5_devx_regex_register_read(ctx, id, + (MLX5_RXP_CSR_WIDTH * i) + + MLX5_RXP_STATS_CSR_BASE_ADDRESS, + ®)) { + DRV_LOG(ERR, "Failed to read STAT CSRs Engine %d!", id); + return; + } + DRV_LOG(DEBUG, "RXP STAT CSRs (Eng%d) register (%d): %08x", + id, i, reg); + } +} /** * DPDK callback for reading device info. @@ -44,24 +143,168 @@ info->max_payload_size = MLX5_REGEX_MAX_PAYLOAD_SIZE; info->max_rules_per_group = MLX5_REGEX_MAX_RULES_PER_GROUP; info->max_groups = MLX5_REGEX_MAX_GROUPS; + info->max_queue_pairs = 1; info->regexdev_capa = RTE_REGEXDEV_SUPP_PCRE_GREEDY_F; info->rule_flags = 0; return 0; } +/** + * Actual writing of RXP instructions to RXP via CSRs. + * + * @param ctx + * The IBV context. + * @param rules + * RXP instructions to be written. + * @param count + * The number of instructions to be written. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +rxp_write_rules_via_cp(struct ibv_context *ctx, + struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id) +{ + int i, ret = 0; + uint32_t tmp; + + for (i = 0; i < count; i++) { + tmp = (uint32_t)rules[i].value; + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_RTRU_CSR_DATA_0, + tmp); + tmp = (uint32_t)(rules[i].value >> 32); + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_RTRU_CSR_DATA_0 + + MLX5_RXP_CSR_WIDTH, tmp); + tmp = rules[i].addr; + ret |= mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_RTRU_CSR_ADDR, + tmp); + if (ret) { + DRV_LOG(ERR, "Failed to copy instructions to RXP."); + return -1; + } + } + DRV_LOG(DEBUG, "Written %d instructions", count); + return 0; +} + +/** + * Flushing the programmed instructions to RXP. + * + * @param ctx + * The IBV context. + * @param rules + * RXP instructions to be written. + * @param count + * The number of instructions to be written. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +rxp_flush_rules(struct ibv_context *ctx, struct mlx5_rxp_rof_entry *rules, + int count, uint8_t id) +{ + uint32_t val, fifo_depth; + int ret; + + ret = rxp_write_rules_via_cp(ctx, rules, count, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to write rules via CSRs."); + return -1; + } + ret = mlx5_devx_regex_register_read(ctx, id, + MLX5_RXP_RTRU_CSR_CAPABILITY, + &fifo_depth); + if (ret) { + DRV_LOG(ERR, "CSR read failed!"); + return -1; + } + ret = rxp_poll_csr_for_value(ctx, &val, MLX5_RXP_RTRU_CSR_FIFO_STAT, + count, ~0, + MLX5_RXP_POLL_CSR_FOR_VALUE_TIMEOUT, id); + if (ret < 0) { + DRV_LOG(ERR, "Rules not rx by RXP: credit: %d, depth: %d", val, + fifo_depth); + return ret; + } + DRV_LOG(DEBUG, "RTRU FIFO depth: 0x%x", fifo_depth); + DRV_LOG(DEBUG, "Rules flush took %d cycles.", ret); + ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + &val); + if (ret) { + DRV_LOG(ERR, "CSR read failed!"); + return -1; + } + val |= MLX5_RXP_RTRU_CSR_CTRL_GO; + ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + val); + ret = rxp_poll_csr_for_value(ctx, &val, MLX5_RXP_RTRU_CSR_STATUS, + MLX5_RXP_RTRU_CSR_STATUS_UPDATE_DONE, + MLX5_RXP_RTRU_CSR_STATUS_UPDATE_DONE, + MLX5_RXP_POLL_CSR_FOR_VALUE_TIMEOUT, id); + if (ret < 0) { + DRV_LOG(ERR, "Rules update timeout: 0x%08X", val); + return ret; + } + DRV_LOG(DEBUG, "Rules update took %d cycles", ret); + if (mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + &val)) { + DRV_LOG(ERR, "CSR read failed!"); + return -1; + } + val &= ~(MLX5_RXP_RTRU_CSR_CTRL_GO); + if (mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, + val)) { + DRV_LOG(ERR, "CSR write write failed!"); + return -1; + } + + DRV_LOG(DEBUG, "RXP Flush rules finished."); + return 0; +} + +/** + * Poll RXP CSRs for expected value. + * + * @param ctx + * The IBV context. + * @param value + * The returned value from a CSR read. + * @param address + * The RXP CSR register to read from. + * @param expected_value + * The expected value to compare read value against. + * @param expected_mask + * A mask used to only compare particular bits of read value. + * @param timeout_ms + * The number of poll interations. + * @param id + * The selected engine to poll. + * + * @return + * 0 on success, a negative errno value otherwise. + */ static int rxp_poll_csr_for_value(struct ibv_context *ctx, uint32_t *value, uint32_t address, uint32_t expected_value, uint32_t expected_mask, uint32_t timeout_ms, uint8_t id) { unsigned int i; - int ret; + int ret = 0; ret = -EBUSY; for (i = 0; i < timeout_ms; i++) { if (mlx5_devx_regex_register_read(ctx, id, address, value)) return -1; - if ((*value & expected_mask) == expected_value) { ret = 0; break; @@ -92,6 +335,7 @@ if (ret) return ret; ctrl |= MLX5_RXP_CSR_CTRL_GO; + ctrl |= MLX5_RXP_CSR_CTRL_DISABLE_L2C; ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); return ret; } @@ -121,6 +365,18 @@ return ret; } +/** + * Initialise the selected RXP engine to specified init mode. + * + * @param ctx + * The IBV context. + * @param id + * The selected engine. + * @param init_bits + * The RXP initialisation modes. + * @return + * 0 on success, a negative errno value otherwise. + */ static int rxp_init_rtru(struct ibv_context *ctx, uint8_t id, uint32_t init_bits) { @@ -130,23 +386,23 @@ uint32_t expected_mask; int ret = 0; - /* Read the rtru ctrl CSR */ + /* Read the rtru ctrl CSR. */ ret = mlx5_devx_regex_register_read(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, &ctrl_value); if (ret) return -1; - /* Clear any previous init modes */ + /* Clear any previous init modes. */ ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_MASK); if (ctrl_value & MLX5_RXP_RTRU_CSR_CTRL_INIT) { ctrl_value &= ~(MLX5_RXP_RTRU_CSR_CTRL_INIT); mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, ctrl_value); } - /* Set the init_mode bits in the rtru ctrl CSR */ + /* Set the init_mode bits in the rtru ctrl CSR. */ ctrl_value |= init_bits; mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_RTRU_CSR_CTRL, ctrl_value); - /* Need to sleep for a short period after pulsing the rtru init bit. */ + /* Need to sleep for a short period after pulsing the rtru init bit. */ rte_delay_us(20000); /* Poll the rtru status CSR until all the init done bits are set. */ DRV_LOG(DEBUG, "waiting for RXP rule memory to complete init"); @@ -162,11 +418,11 @@ if (init_bits == MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2) { /* Must be incremental mode */ expected_value = MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | - MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; } else { expected_value = MLX5_RXP_RTRU_CSR_STATUS_IM_INIT_DONE | - MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | - MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; + MLX5_RXP_RTRU_CSR_STATUS_L1C_INIT_DONE | + MLX5_RXP_RTRU_CSR_STATUS_L2C_INIT_DONE; } expected_mask = expected_value; ret = rxp_poll_csr_for_value(ctx, &poll_value, @@ -184,6 +440,257 @@ } /** + * Extract instructions from buffer into rules format ready for RXP programming. + * + * @param buf + * The buffer holding RXP instructions. + * @param len + * The actual length of the buffer. + * @param init_bits + * The RXP initialisation modes. + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +rxp_parse_rof(const char *buf, uint32_t len, + struct mlx5_rxp_ctl_rules_pgm **rules) +{ + static const char del[] = "\n\r"; + char *line; + char *tmp; + char *cur_pos; + uint32_t lines = 0; + uint32_t entries; + struct mlx5_rxp_rof_entry *curentry; + + tmp = rte_malloc("", len, 0); + if (!tmp) + return -ENOMEM; + memcpy(tmp, buf, len); + line = strtok(tmp, del); + while (line) { + if (line[0] != '#' && line[0] != '\0') + lines++; + line = strtok(NULL, del); + } + *rules = rte_malloc("", lines * sizeof(*curentry) + sizeof(**rules), 0); + if (!(*rules)) { + rte_free(tmp); + return -ENOMEM; + } + memset(*rules, 0, lines * sizeof(curentry) + sizeof(**rules)); + curentry = (*rules)->rules; + (*rules)->hdr.cmd = MLX5_RXP_CTL_RULES_PGM; + entries = 0; + memcpy(tmp, buf, len); + line = strtok(tmp, del); + while (line) { + if (line[0] == '#' || line[0] == '\0') { + line = strtok(NULL, del); + continue; + } + curentry->type = strtoul(line, &cur_pos, 10); + if (cur_pos == line || cur_pos[0] != ',') + goto parse_error; + cur_pos++; + curentry->addr = strtoul(cur_pos, &cur_pos, 16); + if (cur_pos[0] != ',') + goto parse_error; + cur_pos++; + curentry->value = strtoull(cur_pos, &cur_pos, 16); + if (cur_pos[0] != '\0' && cur_pos[0] != '\n') + goto parse_error; + curentry++; + entries++; + if (entries > lines) + goto parse_error; + line = strtok(NULL, del); + } + (*rules)->count = entries; + (*rules)->hdr.len = entries * sizeof(*curentry) + sizeof(**rules); + rte_free(tmp); + return 0; +parse_error: + rte_free(tmp); + if (*rules) + rte_free(*rules); + return -EINVAL; +} + +/** + * Provide db pointer to EM and ensure idle. + * + * @param priv + * Pointer to the private device data structure. + * @param id + * The RXP engine to setup. + * @param db_to_use + * Index to which db pointer allocated for RXP engine. + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +mlnx_set_database(struct mlx5_regex_priv *priv, uint8_t id, uint8_t db_to_use) +{ + int ret; + uint32_t umem_id; + + ret = mlx5_devx_regex_database_stop(priv->ctx, id); + if (ret < 0) { + DRV_LOG(ERR, "stop engine failed!"); + return ret; + } + umem_id = mlx5_os_get_umem_id(priv->db[db_to_use].umem.umem); + ret = mlx5_devx_regex_database_program(priv->ctx, id, umem_id, 0); + if (ret < 0) { + DRV_LOG(ERR, "program db failed!"); + return ret; + } + return 0; +} + +/** + * Resume engine. + * + * @param priv + * Pointer to the private device data structure. + * @param id + * The RXP engine to resume. + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +mlnx_resume_database(struct mlx5_regex_priv *priv, uint8_t id) +{ + mlx5_devx_regex_database_resume(priv->ctx, id); + return 0; +} + +/** + * Assign db memory for RXP programming. + * + * @param priv + * Pointer to the private device data structure. + * @param id + * The RXP engine to assign db. + * @return + * Index to allocated db buffer on success, a negative errno value otherwise. + */ +static int +mlnx_update_database(struct mlx5_regex_priv *priv, uint8_t id) +{ + unsigned int i; + uint8_t db_free = MLX5_RXP_DB_NOT_ASSIGNED; + uint8_t eng_assigned = MLX5_RXP_DB_NOT_ASSIGNED; + + /* Check which database rxp_eng is currently located if any? */ + for (i = 0; i < (MLX5_RXP_MAX_ENGINES + MLX5_RXP_EM_COUNT); + i++) { + if (priv->db[i].db_assigned_to_eng_num == id) { + eng_assigned = i; + break; + } + } + /* + * If private mode then, we can keep the same db ptr as RXP will be + * programming EM itself if necessary, however need to see if + * programmed yet. + */ + if ((priv->prog_mode == MLX5_RXP_PRIVATE_PROG_MODE) && + (eng_assigned != MLX5_RXP_DB_NOT_ASSIGNED)) + return eng_assigned; + /* Check for inactive db memory to use. */ + for (i = 0; i < (MLX5_RXP_MAX_ENGINES + MLX5_RXP_EM_COUNT); + i++) { + if (priv->db[i].active == true) + continue; /* Already in use, so skip db. */ + /* Set this db to active now as free to use. */ + priv->db[i].active = true; + /* Now unassign last db index in use by RXP Eng. */ + if (eng_assigned != MLX5_RXP_DB_NOT_ASSIGNED) { + priv->db[eng_assigned].active = false; + priv->db[eng_assigned].db_assigned_to_eng_num = + MLX5_RXP_DB_NOT_ASSIGNED; + + /* Set all DB memory to 0's before setting up DB. */ + memset(priv->db[i].ptr, 0x00, MLX5_MAX_DB_SIZE); + } + /* Now reassign new db index with RXP Engine. */ + priv->db[i].db_assigned_to_eng_num = id; + db_free = i; + break; + } + if (db_free == MLX5_RXP_DB_NOT_ASSIGNED) + return -1; + return db_free; +} + +/** + * Program RXP instruction db to RXP engine/s. + * + * @param priv + * Pointer to the private device data structure. + * @param rule_buf + * Pointer to buffer that will be holding RXP instructions. + * @param len + * The length of the rule buffer. + * @param id + * The selected engine. + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +program_rxp_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint8_t id) +{ + int ret, db_free; + uint32_t rule_cnt; + + rule_cnt = rules->count; + db_free = mlnx_update_database(priv, id); + if (db_free < 0) { + DRV_LOG(ERR, "Failed to setup db memory!"); + return db_free; + } + if (priv->prog_mode == MLX5_RXP_PRIVATE_PROG_MODE) { + /* Register early to ensure RXP writes to EM use valid addr. */ + ret = mlnx_set_database(priv, id, db_free); + if (ret < 0) { + DRV_LOG(ERR, "Failed to register db memory!"); + return ret; + } + } + ret = write_private_rules(priv, rules, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to write rules!"); + return ret; + } + if (priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE) { + /* Write external rules directly to EM. */ + rules->count = rule_cnt; + /* Now write external instructions to EM. */ + ret = write_shared_rules(priv, rules, rules->hdr.len, db_free); + if (ret < 0) { + DRV_LOG(ERR, "Failed to write EM rules!"); + return ret; + } + ret = mlnx_set_database(priv, id, db_free); + if (ret < 0) { + DRV_LOG(ERR, "Failed to register db memory!"); + return ret; + } + } + ret = mlnx_resume_database(priv, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to resume engine!"); + return ret; + } + DRV_LOG(DEBUG, "Programmed RXP Engine %d\n", id); + rules->count = rule_cnt; + return 0; +} + +/** * Init the engine. * * @param ctx @@ -192,10 +699,10 @@ * The selected engine. * * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. + * 0 on success, a negative errno value otherwise. */ static int -rxp_init(struct mlx5_regex_priv *priv, uint8_t id) +rxp_init_eng(struct mlx5_regex_priv *priv, uint8_t id) { uint32_t ctrl; uint32_t reg; @@ -219,7 +726,6 @@ ctrl &= ~MLX5_RXP_CSR_CTRL_INIT; ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_CTRL, ctrl); rte_delay_us(20000); - ret = rxp_poll_csr_for_value(ctx, &ctrl, MLX5_RXP_CSR_STATUS, MLX5_RXP_CSR_STATUS_INIT_DONE, MLX5_RXP_CSR_STATUS_INIT_DONE, @@ -234,7 +740,6 @@ ctrl); if (ret) return ret; - rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); ret = rxp_init_rtru(ctx, id, MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_IM_L1_L2); if (ret) return ret; @@ -244,8 +749,16 @@ return ret; DRV_LOG(DEBUG, "max matches: %d, DDOS threshold: %d", reg >> 16, reg & 0xffff); - ret = mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_MATCH, - priv->nb_max_matches); + if ((reg >> 16) >= priv->nb_max_matches) + ret = mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_MATCH, + priv->nb_max_matches); + else + ret = mlx5_devx_regex_register_write(ctx, id, + MLX5_RXP_CSR_MAX_MATCH, + (reg >> 16)); + ret |= mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_PREFIX, + (reg & 0xFFFF)); ret |= mlx5_devx_regex_register_write(ctx, id, MLX5_RXP_CSR_MAX_LATENCY, 0); ret |= mlx5_devx_regex_register_write(ctx, id, @@ -254,6 +767,451 @@ } /** + * Private programming of RXP, here all private/internal instructions will + * be written to RXP private memories, and all external instructions will be + * written to EM via RXP and not host. + * + * @param priv + * Pointer to private regex structure. + * @param rules + * Pointer to actual rules buffer containing instructions. + * @param count + * The number of instructions to program. + * @param id + * The selected engine. + * + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +write_private_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, + uint8_t id) +{ + unsigned int pending; + uint32_t block, reg, val, rule_cnt, rule_offset, rtru_max_num_entries; + int ret = 1; + + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) + return -EINVAL; + if (rules->hdr.len == 0 || rules->hdr.cmd < MLX5_RXP_CTL_RULES_PGM || + rules->hdr.cmd > MLX5_RXP_CTL_RULES_PGM_INCR) + return -EINVAL; + /* For a non-incremental rules program, re-init the RXP. */ + if (rules->hdr.cmd == MLX5_RXP_CTL_RULES_PGM) { + ret = rxp_init_eng(priv, id); + if (ret < 0) + return ret; + } else if (rules->hdr.cmd == MLX5_RXP_CTL_RULES_PGM_INCR) { + /* Flush RXP L1 and L2 cache by using MODE_L1_L2. */ + ret = rxp_init_rtru(priv->ctx, id, + MLX5_RXP_RTRU_CSR_CTRL_INIT_MODE_L1_L2); + if (ret < 0) + return ret; + } + if (rules->count == 0) + return -EINVAL; + /* Confirm the RXP is initialised. */ + if (mlx5_devx_regex_register_read(priv->ctx, id, + MLX5_RXP_CSR_STATUS, &val)) { + DRV_LOG(ERR, "Failed to read from RXP!"); + return -ENODEV; + } + if (!(val & MLX5_RXP_CSR_STATUS_INIT_DONE)) { + DRV_LOG(ERR, "RXP not initialised..."); + return -EBUSY; + } + /* Get the RTRU maximum number of entries allowed. */ + if (mlx5_devx_regex_register_read(priv->ctx, id, + MLX5_RXP_RTRU_CSR_CAPABILITY, &rtru_max_num_entries)) { + DRV_LOG(ERR, "Failed to read RTRU capability!"); + return -ENODEV; + } + rtru_max_num_entries = (rtru_max_num_entries & 0x00FF); + rule_cnt = 0; + pending = 0; + while (rules->count > 0) { + if ((rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_INST) || + (rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_IM) || + (rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_EM)) { + if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_EM) && + (priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE)) { + /* Skip EM rules programming. */ + if (pending > 0) { + /* Flush any rules that are pending. */ + rule_offset = (rule_cnt - pending); + ret = rxp_flush_rules(priv->ctx, + &rules->rules[rule_offset], + pending, id); + if (ret < 0) { + DRV_LOG(ERR, "Flushing rules."); + return -ENODEV; + } + pending = 0; + } + rule_cnt++; + } else { + pending++; + rule_cnt++; + /* + * If parsing the last rule, or if reached the + * maximum number of rules for this batch, then + * flush the rules batch to the RXP. + */ + if ((rules->count == 1) || + (pending == rtru_max_num_entries)) { + rule_offset = (rule_cnt - pending); + ret = rxp_flush_rules(priv->ctx, + &rules->rules[rule_offset], + pending, id); + if (ret < 0) { + DRV_LOG(ERR, "Flushing rules."); + return -ENODEV; + } + pending = 0; + } + } + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_EQ) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_GTE) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_LTE) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM) || + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM_EX_EM)) { + if (pending) { + /* Flush rules before checking reg values. */ + rule_offset = (rule_cnt - pending); + ret = rxp_flush_rules(priv->ctx, + &rules->rules[rule_offset], + pending, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to flush rules."); + return -ENODEV; + } + } + block = (rules->rules[rule_cnt].addr >> 16) & 0xFFFF; + if (block == 0) + reg = MLX5_RXP_CSR_BASE_ADDRESS; + else if (block == 1) + reg = MLX5_RXP_RTRU_CSR_BASE_ADDRESS; + else { + DRV_LOG(ERR, "Invalid ROF register 0x%08X!", + rules->rules[rule_cnt].addr); + return -EINVAL; + } + reg += (rules->rules[rule_cnt].addr & 0xFFFF) * + MLX5_RXP_CSR_WIDTH; + ret = mlx5_devx_regex_register_read(priv->ctx, id, + reg, &val); + if (ret) { + DRV_LOG(ERR, "RXP CSR read failed!"); + return ret; + } + if ((priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE) && + ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM_EX_EM) && + (val != rules->rules[rule_cnt].value))) { + DRV_LOG(ERR, "Unexpected value for register:"); + DRV_LOG(ERR, "reg %x" PRIu32 " got %x" PRIu32, + rules->rules[rule_cnt].addr, val); + DRV_LOG(ERR, "expected %" PRIx64 ".", + rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((priv->prog_mode == + MLX5_RXP_PRIVATE_PROG_MODE) && + (rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_CHECKSUM) && + (val != rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value for register:"); + DRV_LOG(ERR, "reg %x" PRIu32 " got %x" PRIu32, + rules->rules[rule_cnt].addr, val); + DRV_LOG(ERR, "expected %" PRIx64 ".", + rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_EQ) && + (val != rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value for register:"); + DRV_LOG(ERR, "reg %x" PRIu32 " got %x" PRIu32, + rules->rules[rule_cnt].addr, val); + DRV_LOG(ERR, "expected %" PRIx64 ".", + rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_GTE) && + (val < rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value reg 0x%08X,", + rules->rules[rule_cnt].addr); + DRV_LOG(ERR, "got %X, expected >= %" PRIx64 ".", + val, rules->rules[rule_cnt].value); + return -EINVAL; + } else if ((rules->rules[rule_cnt].type == + MLX5_RXP_ROF_ENTRY_LTE) && + (val > rules->rules[rule_cnt].value)) { + DRV_LOG(ERR, "Unexpected value reg 0x%08X,", + rules->rules[rule_cnt].addr); + DRV_LOG(ERR, "got %08X expected <= %" PRIx64, + val, rules->rules[rule_cnt].value); + return -EINVAL; + } + rule_cnt++; + pending = 0; + } else { + DRV_LOG(ERR, "Error: Invalid rule type %d!", + rules->rules[rule_cnt].type); + return -EINVAL; + } + rules->count--; + } + return ret; +} + +/** + * Shared memory programming mode, here all external db instructions are written + * to EM via the host. + * + * @param priv + * Pointer to private regex structure. + * @param rules + * Pointer to actual rules buffer containing instructions. + * @param count + * The number of instructions to program. + * @param db_to_program + * The selected db memory to write too. + * + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +write_shared_rules(struct mlx5_regex_priv *priv, + struct mlx5_rxp_ctl_rules_pgm *rules, uint32_t count, + uint8_t db_to_program) +{ + uint32_t rule_cnt, rof_rule_addr; + uint64_t tmp_write_swap[4]; +#ifndef VIPER_BF2 + uint64_t rof_value = 0; +#endif + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) + return -EINVAL; + if ((rules->count == 0) || (count == 0)) + return -EINVAL; + rule_cnt = 0; + while (rule_cnt < rules->count) { + if ((rules->rules[rule_cnt].type == MLX5_RXP_ROF_ENTRY_EM) && + (priv->prog_mode == MLX5_RXP_SHARED_PROG_MODE)) { +#ifdef VIPER_BF2 + /* + * Note there are always blocks of 8 instructions for + * 7's written sequentially. However there is no + * guarantee that all blocks are sequential! + */ + if (count >= (rule_cnt + MLX5_RXP_INST_BLOCK_SIZE)) { + /* + * Ensure memory write not exceeding boundary + * Check essential to ensure 0x10000 offset + * accounted for! + */ + if ((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + ((rules->rules[rule_cnt + 7].addr << + MLX5_RXP_INST_OFFSET))) >= + ((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + MLX5_MAX_DB_SIZE))) { + DRV_LOG(ERR, "DB exceeded memory!"); + return -ENODEV; + } + /* + * Rule address Offset to align with RXP + * external instruction offset. + */ + rof_rule_addr = (rules->rules[rule_cnt].addr << + MLX5_RXP_INST_OFFSET); + /* 32 byte instruction swap (sw work around)! */ + tmp_write_swap[0] = le64toh( + rules->rules[(rule_cnt + 4)].value); + tmp_write_swap[1] = le64toh( + rules->rules[(rule_cnt + 5)].value); + tmp_write_swap[2] = le64toh( + rules->rules[(rule_cnt + 6)].value); + tmp_write_swap[3] = le64toh( + rules->rules[(rule_cnt + 7)].value); + /* Write only 4 of the 8 instructions. */ + memcpy((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + rof_rule_addr), &tmp_write_swap, + (sizeof(uint64_t) * 4)); + /* Write 1st 4 rules of block after last 4. */ + rof_rule_addr = (rules->rules[ + (rule_cnt + 4)].addr << + MLX5_RXP_INST_OFFSET); + tmp_write_swap[0] = le64toh( + rules->rules[(rule_cnt + 0)].value); + tmp_write_swap[1] = le64toh( + rules->rules[(rule_cnt + 1)].value); + tmp_write_swap[2] = le64toh( + rules->rules[(rule_cnt + 2)].value); + tmp_write_swap[3] = le64toh( + rules->rules[(rule_cnt + 3)].value); + memcpy((uint8_t *)((uint8_t *) + priv->db[db_to_program].ptr + + rof_rule_addr), &tmp_write_swap, + (sizeof(uint64_t) * 4)); + } else + return -1; + /* Fast forward as already handled block of 8. */ + rule_cnt += MLX5_RXP_INST_BLOCK_SIZE; +#else + /* + * Ensure memory write not exceeding boundary check + * essential to ensure 0x10000 offset accounted for! + */ + if ((uint8_t *)((uint8_t *)priv->db[db_to_program].ptr + + (rules->rules[rule_cnt].addr << + MLX5_RXP_INST_OFFSET)) >= + (uint8_t *)((uint8_t *)priv->db[db_to_program].ptr + + MLX5_MAX_DB_SIZE)) { + /* Exceeded database memory boundary. */ + DRV_LOG(ERR, "DB exceeded memory boundary!"); + return -ENODEV; + } + /* + * Rule address Offset to align with RXP external + * instruction offset. + */ + rof_rule_addr = (rules->rules[rule_cnt].addr << + MLX5_RXP_INST_OFFSET); + rof_value = le64toh(rules->rules[rule_cnt].value); + memcpy((uint8_t *)( + (uint8_t *)priv->db[db_to_program].ptr + + rof_rule_addr), &rof_value, sizeof(uint64_t)); + rule_cnt++; +#endif /* VIPER_BF2 */ + } else + rule_cnt++; /* Must be something other than EM rule. */ + } + return 0; +} + +/** + * RXP initial db setup function for EM instructions + * + * @param priv + * Pointer to private RegEx structure. + * + * @return + * 0 on success, a negative errno value otherwise. + */ +static int +rxp_db_setup(struct mlx5_regex_priv *priv) +{ + int ret; + uint8_t i; + + /* Setup database memories for both RXP engines + reprogram memory. */ + for (i = 0; i < (MLX5_RXP_MAX_ENGINES + MLX5_RXP_EM_COUNT); i++) { + priv->db[i].ptr = rte_malloc("", MLX5_MAX_DB_SIZE, 0); + if (!priv->db[i].ptr) { + DRV_LOG(ERR, "Failed to alloc db memory!"); + ret = ENODEV; + goto tidyup_error; + } + /* Register the memory. */ + priv->db[i].umem.umem = mlx5_glue->devx_umem_reg(priv->ctx, + priv->db[i].ptr, + MLX5_MAX_DB_SIZE, 7); + if (!priv->db[i].umem.umem) { + DRV_LOG(ERR, "Failed to register memory!"); + ret = ENODEV; + goto tidyup_error; + } + /* Ensure set all DB memory to 0's before setting up DB. */ + memset(priv->db[i].ptr, 0x00, MLX5_MAX_DB_SIZE); + /* No data currently in database. */ + priv->db[i].len = 0; + priv->db[i].active = false; + priv->db[i].db_assigned_to_eng_num = MLX5_RXP_DB_NOT_ASSIGNED; + } + return 0; +tidyup_error: + for (i = 0; i < (MLX5_RXP_MAX_ENGINES + MLX5_RXP_EM_COUNT); i++) { + if (priv->db[i].ptr) + munmap(priv->db[i].ptr, MLX5_MAX_DB_SIZE); + if (priv->db[i].umem.umem) + mlx5_glue->devx_umem_dereg(priv->db[i].umem.umem); + } + return -ret; +} + +/** + * DPDK callback for loading RXP rules. + * + * @param dev + * Pointer to RegEx device structure. + * @param[in] cfg + * Pointer to the regexdev device configuration structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_regex_rules_db_import(struct rte_regexdev *dev, + const char *rule_db, uint32_t rule_db_len) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_rxp_ctl_rules_pgm *rules = NULL; + uint8_t id; + int ret; + + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) { + DRV_LOG(ERR, "RXP programming mode not set!"); + return -1; + } + if (rule_db == NULL) { + DRV_LOG(ERR, "Database empty!"); + return -ENODEV; + } + if (rule_db_len == 0) + return -EINVAL; + ret = rxp_parse_rof(rule_db, rule_db_len, &rules); + if (ret) { + DRV_LOG(ERR, "Can't parse ROF file."); + return ret; + } + /* Need to ensure RXP not busy before stop! */ + for (id = 0; id < MLX5_RXP_MAX_ENGINES; id++) { + ret = rxp_stop_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "Can't stop engine."); + ret = -ENODEV; + goto tidyup_error; + } + ret = program_rxp_rules(priv, rules, id); + if (ret < 0) { + DRV_LOG(ERR, "Failed to program rxp rules."); + ret = -ENODEV; + goto tidyup_error; + } + ret = rxp_start_engine(priv->ctx, id); + if (ret) { + DRV_LOG(ERR, "Can't start engine."); + ret = -ENODEV; + goto tidyup_error; + } + } + rte_free(rules); + return 0; +tidyup_error: + rte_free(rules); + return ret; +} + +/** * DPDK callback for reading device info. * * @param dev @@ -270,9 +1228,11 @@ { struct mlx5_regex_priv *priv = dev->data->dev_private; int ret; - uint8_t id; + if (priv->prog_mode == MLX5_RXP_MODE_NOT_DEFINED) + return -1; priv->nb_queues = cfg->nb_queue_pairs; + dev->data->dev_conf.nb_queue_pairs = priv->nb_queues; priv->qps = rte_zmalloc(NULL, sizeof(struct mlx5_regex_qp) * priv->nb_queues, 0); if (!priv->nb_queues) { @@ -281,35 +1241,22 @@ return -rte_errno; } priv->nb_max_matches = cfg->nb_max_matches; - for (id = 0; id < 2; id++) { - ret = rxp_stop_engine(priv->ctx, id); - if (ret) { - DRV_LOG(ERR, "can't stop engine."); - rte_errno = ENODEV; - return -rte_errno; - } - ret = rxp_init(priv, id); - if (ret) { - DRV_LOG(ERR, "can't init engine."); - rte_errno = ENODEV; - return -rte_errno; - } - ret = mlx5_devx_regex_register_write(priv->ctx, id, - MLX5_RXP_CSR_MAX_MATCH, - priv->nb_max_matches); - if (ret) { - DRV_LOG(ERR, "can't update number of matches."); - rte_errno = ENODEV; - goto configure_error; - } - ret = rxp_start_engine(priv->ctx, id); - if (ret) { - DRV_LOG(ERR, "can't start engine."); + /* Setup rxp db memories. */ + if (rxp_db_setup(priv)) { + DRV_LOG(ERR, "Failed to setup RXP db memory"); + rte_errno = ENOMEM; + return -rte_errno; + } + if (cfg->rule_db != NULL) { + ret = mlx5_regex_rules_db_import(dev, cfg->rule_db, + cfg->rule_db_len); + if (ret < 0) { + DRV_LOG(ERR, "Failed to program rxp rules."); rte_errno = ENODEV; goto configure_error; } - - } + } else + DRV_LOG(DEBUG, "Regex config without rules programming!"); return 0; configure_error: if (priv->qps) diff --git a/drivers/regex/mlx5/mlx5_rxp.h b/drivers/regex/mlx5/mlx5_rxp.h new file mode 100644 index 0000000..fd19d40 --- /dev/null +++ b/drivers/regex/mlx5/mlx5_rxp.h @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2019 Mellanox Technologies, Ltd + */ + +#ifndef RTE_PMD_MLX5_REGEX_RXP_H_ +#define RTE_PMD_MLX5_REGEX_RXP_H_ + +#define MLX5_RXP_MAX_JOB_LENGTH 16384 +#define MLX5_RXP_MAX_SUBSETS 4095 +#define MLX5_RXP_CSR_NUM_ENTRIES 31 + +#define MLX5_RXP_CTRL_TYPE_MASK 7 +#define MLX5_RXP_CTRL_TYPE_JOB_DESCRIPTOR 0 +#define MLX5_RXP_CTRL_TYPE_RESPONSE_DESCRIPTOR 1 +#define MLX5_RXP_CTRL_TYPE_MEMORY_WRITE 4 +#define MLX5_RXP_CSR_CTRL_DISABLE_L2C (1 << 7) + +#define MLX5_RXP_CTRL_JOB_DESC_SOF 0x0010 +#define MLX5_RXP_CTRL_JOB_DESC_EOF 0x0020 +#define MLX5_RXP_CTRL_JOB_DESC_HPM_ENABLE 0x0100 +#define MLX5_RXP_CTRL_JOB_DESC_ANYMATCH_ENABLE 0x0200 +#define MLX5_RXP_CTRL_JOB_DESC_FLAGS (MLX5_RXP_CTRL_JOB_DESC_SOF | \ + MLX5_RXP_CTRL_JOB_DESC_EOF | \ + MLX5_RXP_CTRL_JOB_DESC_HPM_ENABLE | \ + MLX5_RXP_CTRL_JOB_DESC_ANYMATCH_ENABLE) + +#define MLX5_RXP_CTRL_VALID 0x8000 + +#define MLX5_RXP_RESP_STATUS_MAX_PRI_THREADS (1 << 3) +#define MLX5_RXP_RESP_STATUS_MAX_SEC_THREADS (1 << 4) +#define MLX5_RXP_RESP_STATUS_MAX_LATENCY (1 << 5) +#define MLX5_RXP_RESP_STATUS_MAX_MATCH (1 << 6) +#define MLX5_RXP_RESP_STATUS_MAX_PREFIX (1 << 7) +#define MLX5_RXP_RESP_STATUS_HPM (1 << 8) +#define MLX5_RXP_RESP_STATUS_ANYMATCH (1 << 9) +#define MLX5_RXP_RESP_STATUS_PMI_SOJ (1 << 13) +#define MLX5_RXP_RESP_STATUS_PMI_EOJ (1 << 14) + +/* This describes the header the RXP expects for any search data. */ +struct mlx5_rxp_job_desc { + uint32_t job_id; + uint16_t ctrl; + uint16_t len; + uint16_t subset[4]; +} __rte_packed; + +struct mlx5_rxp_response_desc { + uint32_t job_id; + uint16_t status; + uint8_t detected_match_count; + uint8_t match_count; + uint16_t primary_thread_count; + uint16_t instruction_count; + uint16_t latency_count; + uint16_t pmi_min_byte_ptr; +} __rte_packed; + +struct mlx5_rxp_match_tuple { + uint32_t rule_id; + uint16_t start_ptr; + uint16_t length; +} __rte_packed; + +struct mlx5_rxp_response { + struct mlx5_rxp_response_desc header; + struct mlx5_rxp_match_tuple matches[0]; +}; + +#define MLX5_RXP_MAX_MATCHES 254 + +#define MLX5_RXP_CTL_RULES_PGM 1 +#define MLX5_RXP_CTL_RULES_PGM_INCR 2 + +#define MLX5_RXP_ROF_ENTRY_INST 0 +#define MLX5_RXP_ROF_ENTRY_EQ 1 +#define MLX5_RXP_ROF_ENTRY_GTE 2 +#define MLX5_RXP_ROF_ENTRY_LTE 3 +#define MLX5_RXP_ROF_ENTRY_CHECKSUM 4 +#define MLX5_RXP_ROF_ENTRY_CHECKSUM_EX_EM 5 +#define MLX5_RXP_ROF_ENTRY_IM 6 +#define MLX5_RXP_ROF_ENTRY_EM 7 +#define MLX5_RXP_ROF_ENTRY_TYPE_MAX 7 + +#define MLX5_RXP_INST_OFFSET 3 +#define MLX5_RXP_INST_BLOCK_SIZE 8 +#define MLX5_MAX_SIZE_RES_DES (sizeof(struct mlx5_rxp_response_desc)) +#define MLX5_MAX_DB_SIZE (1u << 27u) +#define MLX5_MAX_SIZE_MATCH_RESP (254 * sizeof(struct mlx5_rxp_match_tuple)) +#define MLX5_RXP_SQ_NOT_BUSY false +#define MLX5_RXP_SQ_BUSY true + + +struct mlx5_rxp_ctl_hdr { + uint16_t cmd; + uint32_t len; +}; + +struct mlx5_rxp_rof_entry { + uint8_t type; + uint32_t addr; + uint64_t value; +}; + +struct mlx5_rxp_rof { + uint32_t rof_version; + char *timestamp; + char *rxp_compiler_version; + uint32_t rof_revision; + uint32_t number_of_entries; + struct mlx5_rxp_rof_entry *rof_entries; +}; + +struct mlx5_rxp_ctl_rules_pgm { + struct mlx5_rxp_ctl_hdr hdr; + uint32_t count; + struct mlx5_rxp_rof_entry rules[0]; +} __rte_packed; + +/* RXP programming mode setting. */ +enum mlx5_rxp_program_mode { + MLX5_RXP_MODE_NOT_DEFINED = 0, + MLX5_RXP_SHARED_PROG_MODE, + MLX5_RXP_PRIVATE_PROG_MODE, +}; + +#define MLX5_RXP_POLL_CSR_FOR_VALUE_TIMEOUT 3000 /* Poll timeout in ms. */ +#define MLX5_RXP_INITIALIZATION_TIMEOUT 60000 /* Initialize timeout in ms. */ +#define MLX5_RXP_MAX_ENGINES 2u /* Number of RXP engines. */ +#define MLX5_RXP_EM_COUNT 1u /* Extra External Memories to use. */ +#define MLX5_RXP_DB_NOT_ASSIGNED 0xFF + +struct mlx5_regex_umem { + struct mlx5dv_devx_umem *umem; + uint32_t id; + uint64_t offset; +}; + +#endif /* RTE_PMD_MLX5_REGEX_RXP_H_ */ From patchwork Sun Jul 5 09:23:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73124 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C01B1A00C5; Sun, 5 Jul 2020 11:26:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3D5401DB79; Sun, 5 Jul 2020 11:24:59 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 031C61DB79 for ; Sun, 5 Jul 2020 11:24:57 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:55 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC40028191; Sun, 5 Jul 2020 12:24:55 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 5 Jul 2020 09:23:41 +0000 Message-Id: <1593941027-86651-15-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 14/20] regex/mlx5: add completion queue creation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds the creation of CQ Signed-off-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/meson.build | 1 + drivers/regex/mlx5/mlx5_regex.c | 1 + drivers/regex/mlx5/mlx5_regex.h | 4 +- drivers/regex/mlx5/mlx5_regex_control.c | 199 ++++++++++++++++++++++++++++++++ drivers/regex/mlx5/mlx5_rxp.c | 1 + 6 files changed, 205 insertions(+), 2 deletions(-) create mode 100644 drivers/regex/mlx5/mlx5_regex_control.c diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index 1ad5125..bc7e13e 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -9,6 +9,7 @@ LIB = librte_pmd_mlx5_regex.a # Sources. SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_rxp.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_control.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index cb092ad..165d660 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -12,6 +12,7 @@ deps += ['common_mlx5', 'pci', 'bus_pci', 'eal', 'sched', 'regexdev'] sources = files( 'mlx5_regex.c', 'mlx5_rxp.c', + 'mlx5_regex_control.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 8801cd7..0f20cde 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -25,6 +25,7 @@ .dev_info_get = mlx5_regex_info_get, .dev_configure = mlx5_regex_configure, .dev_db_import = mlx5_regex_rules_db_import, + .dev_qp_setup = mlx5_regex_qp_setup, }; diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index fca9ebb..88903e4 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -23,7 +23,7 @@ struct mlx5_regex_sq { struct mlx5_devx_obj *obj; /* The SQ DevX object. */ int64_t dbr_offset; /* Door bell record offset. */ uint32_t dbr_umem; /* Door bell record umem id. */ - volatile struct mlx5_cqe *wqe; /* The SQ ring buffer. */ + uint8_t *wqe; /* The SQ ring buffer. */ struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ }; @@ -65,10 +65,10 @@ struct mlx5_regex_priv { enum mlx5_rxp_program_mode prog_mode; struct mlx5_regex_db db[MLX5_RXP_MAX_ENGINES + MLX5_RXP_EM_COUNT]; - struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ uint32_t eqn; /* EQ number. */ struct mlx5dv_devx_uar *uar; /* UAR object. */ struct ibv_pd *pd; + struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ }; /* mlx5_rxp.c */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c new file mode 100644 index 0000000..c5ae48c --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -0,0 +1,199 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "mlx5_regex.h" +#include "mlx5_regex_utils.h" +#include "mlx5_rxp_csrs.h" +#include "mlx5_rxp.h" + +#define MLX5_REGEX_PAGE_SIZE 4096 + +/** + * Returns the number of qp obj to be created. + * + * @param nb_desc + * The number of descriptors for the queue. + * + * @return + * The number of obj to be created. + */ +static uint16_t +regex_ctrl_get_nb_obj(uint16_t nb_desc) +{ + int pages; + + pages = nb_desc / MLX5_REGEX_PAGE_SIZE; + if (pages > 1) + return pages + 1; + return 2; +} + +/** + * destroy CQ. + * + * @param priv + * Pointer to the priv object. + * @param cp + * Pointer to the CQ to be destroyed. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_destroy_cq(struct mlx5_regex_priv *priv, struct mlx5_regex_cq *cq) +{ + if (cq->cqe_umem) { + mlx5_glue->devx_umem_dereg(cq->cqe_umem); + cq->cqe_umem = NULL; + } + if (cq->cqe) { + rte_free((void *)(uintptr_t)cq->cqe); + cq->cqe = NULL; + } + if (cq->dbr_offset) { + mlx5_release_dbr(&priv->dbrpgs, cq->dbr_umem, cq->dbr_offset); + cq->dbr_offset = -1; + } + if (cq->obj) { + mlx5_devx_cmd_destroy(cq->obj); + cq->obj = NULL; + } + return 0; +} + +/** + * create the CQ object. + * + * @param priv + * Pointer to the priv object. + * @param cp + * Pointer to the CQ to be created. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_create_cq(struct mlx5_regex_priv *priv, struct mlx5_regex_cq *cq) +{ + struct mlx5_devx_cq_attr attr = { + .q_umem_valid = 1, + .db_umem_valid = 1, + .eqn = priv->eqn, + }; + struct mlx5_devx_dbr_page *dbr_page = NULL; + void *buf = NULL; + size_t pgsize = sysconf(_SC_PAGESIZE); + uint32_t cq_size = 1 << cq->log_nb_desc; + uint32_t i; + + cq->dbr_offset = mlx5_get_dbr(priv->ctx, &priv->dbrpgs, &dbr_page); + if (cq->dbr_offset < 0) { + DRV_LOG(ERR, "Can't allocate cq door bell record."); + rte_errno = ENOMEM; + goto error; + } + cq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); + buf = rte_calloc(NULL, 1, sizeof(struct mlx5_cqe) * cq_size, 4096); + if (!buf) { + DRV_LOG(ERR, "Can't allocate cqe buffer."); + rte_errno = ENOMEM; + goto error; + } + cq->cqe = buf; + for (i = 0; i < cq_size; i++) + cq->cqe[i].op_own = 0xff; + cq->cqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, + sizeof(struct mlx5_cqe) * + cq_size, 7); + if (!cq->cqe_umem) { + DRV_LOG(ERR, "Can't register cqe mem."); + rte_errno = ENOMEM; + goto error; + } + attr.db_umem_offset = cq->dbr_offset; + attr.db_umem_id = cq->dbr_umem; + attr.q_umem_id = mlx5_os_get_umem_id(cq->cqe_umem); + attr.log_cq_size = cq->log_nb_desc; + attr.uar_page_id = priv->uar->page_id; + attr.log_page_size = rte_log2_u32(pgsize); + cq->obj = mlx5_devx_cmd_create_cq(priv->ctx, &attr); + if (!cq->obj) { + DRV_LOG(ERR, "Can't create cq object."); + rte_errno = ENOMEM; + goto error; + } + return 0; +error: + if (cq->cqe_umem) + mlx5_glue->devx_umem_dereg(cq->cqe_umem); + if (buf) + rte_free(buf); + if (cq->dbr_offset) + mlx5_release_dbr(&priv->dbrpgs, cq->dbr_umem, cq->dbr_offset); + return -rte_errno; +} + +/** + * Setup the qp. + * + * @param dev + * Pointer to RegEx dev structure. + * @param qp_ind + * The queue index to setup. + * @param cfg + * The queue requested configuration. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, + const struct rte_regexdev_qp_conf *cfg) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_regex_qp *qp; + int ret; + + qp = &priv->qps[qp_ind]; + qp->flags = cfg->qp_conf_flags; + qp->nb_desc = cfg->nb_desc; + if (qp->flags & RTE_REGEX_QUEUE_PAIR_CFG_OOS_F) + qp->nb_obj = regex_ctrl_get_nb_obj(qp->nb_desc); + else + qp->nb_obj = 1; + qp->sqs = rte_malloc(NULL, + qp->nb_obj * sizeof(struct mlx5_regex_sq), 64); + if (!qp->sqs) { + DRV_LOG(ERR, "Can't allocate sq array memory."); + rte_errno = ENOMEM; + return -rte_errno; + } + qp->cq.log_nb_desc = rte_log2_u32(cfg->nb_desc); + ret = regex_ctrl_create_cq(priv, &qp->cq); + if (ret) { + DRV_LOG(ERR, "Can't create cq."); + goto error; + } + return 0; + +error: + regex_ctrl_destroy_cq(priv, &qp->cq); + return -rte_errno; + +} diff --git a/drivers/regex/mlx5/mlx5_rxp.c b/drivers/regex/mlx5/mlx5_rxp.c index 2bb3229..12e16b1 100644 --- a/drivers/regex/mlx5/mlx5_rxp.c +++ b/drivers/regex/mlx5/mlx5_rxp.c @@ -146,6 +146,7 @@ info->max_queue_pairs = 1; info->regexdev_capa = RTE_REGEXDEV_SUPP_PCRE_GREEDY_F; info->rule_flags = 0; + info->max_queue_pairs = 10; return 0; } From patchwork Sun Jul 5 09:23:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73127 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1E390A00C5; Sun, 5 Jul 2020 11:27:02 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C59821DBB4; Sun, 5 Jul 2020 11:25:05 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 0D1261DBC4 for ; Sun, 5 Jul 2020 11:25:02 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:24:58 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC41028191; Sun, 5 Jul 2020 12:24:58 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com Date: Sun, 5 Jul 2020 09:23:42 +0000 Message-Id: <1593941027-86651-16-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 15/20] regex/mlx5: add send queue support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit introduce the SQ creation. The SQ is used for enqueuing a job. In order to support out of order matches, we create number os SQ per one applicaiton QP. Signed-off-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.h | 2 + drivers/regex/mlx5/mlx5_regex_control.c | 169 ++++++++++++++++++++++++++++++++ 2 files changed, 171 insertions(+) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 88903e4..d3267fa 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -25,6 +25,7 @@ struct mlx5_regex_sq { uint32_t dbr_umem; /* Door bell record umem id. */ uint8_t *wqe; /* The SQ ring buffer. */ struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ + uint32_t *dbr; }; struct mlx5_regex_cq { @@ -34,6 +35,7 @@ struct mlx5_regex_cq { uint32_t dbr_umem; /* Door bell record umem id. */ volatile struct mlx5_cqe *cqe; /* The CQ ring buffer. */ struct mlx5dv_devx_umem *cqe_umem; /* CQ buffer umem. */ + uint32_t *dbr; }; struct mlx5_regex_qp { diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index c5ae48c..08f60cc 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -109,6 +109,9 @@ goto error; } cq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); + cq->dbr = (uint32_t *)((uintptr_t)dbr_page->dbrs + + (uintptr_t)cq->dbr_offset); + buf = rte_calloc(NULL, 1, sizeof(struct mlx5_cqe) * cq_size, 4096); if (!buf) { DRV_LOG(ERR, "Can't allocate cqe buffer."); @@ -149,6 +152,159 @@ return -rte_errno; } +static int +regex_get_pdn(void *pd, uint32_t *pdn) +{ + struct mlx5dv_obj obj; + struct mlx5dv_pd pd_info; + int ret = 0; + + obj.pd.in = pd; + obj.pd.out = &pd_info; + ret = mlx5_glue->dv_init_obj(&obj, MLX5DV_OBJ_PD); + if (ret) { + DRV_LOG(DEBUG, "Fail to get PD object info"); + return ret; + } + *pdn = pd_info.pdn; + return 0; +} + +/** + * create the SQ object. + * + * @param priv + * Pointer to the priv object. + * @param qp + * Pointer to the QP element + * @param q_ind + * The index of the queue. + * @param log_nb_desc + * Log 2 of the number of descriptors to be used. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_create_sq(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, + uint16_t q_ind, uint16_t log_nb_desc) +{ + struct mlx5_devx_create_sq_attr attr = { 0 }; + struct mlx5_devx_modify_sq_attr modify_attr = { 0 }; + struct mlx5_devx_wq_attr *wq_attr = &attr.wq_attr; + struct mlx5_devx_dbr_page *dbr_page = NULL; + struct mlx5_regex_sq *sq = &qp->sqs[q_ind]; + void *buf = NULL; + uint32_t sq_size; + uint32_t pd_num = 0; + int ret; + + sq->log_nb_desc = log_nb_desc; + sq_size = 1 << sq->log_nb_desc; + sq->dbr_offset = mlx5_get_dbr(priv->ctx, &priv->dbrpgs, &dbr_page); + if (sq->dbr_offset < 0) { + DRV_LOG(ERR, "Can't allocate sq door bell record."); + rte_errno = ENOMEM; + goto error; + } + sq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); + sq->dbr = (uint32_t *)((uintptr_t)dbr_page->dbrs + + (uintptr_t)sq->dbr_offset); + + buf = rte_calloc(NULL, 1, 64 * sq_size, 4096); + if (!buf) { + DRV_LOG(ERR, "Can't allocate wqe buffer."); + rte_errno = ENOMEM; + goto error; + } + sq->wqe = buf; + sq->wqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, 64 * sq_size, + 7); + if (!sq->wqe_umem) { + DRV_LOG(ERR, "Can't register wqe mem."); + rte_errno = ENOMEM; + goto error; + } + attr.state = MLX5_SQC_STATE_RST; + attr.tis_lst_sz = 0; + attr.tis_num = 0; + attr.user_index = q_ind; + attr.cqn = qp->cq.obj->id; + wq_attr->uar_page = priv->uar->page_id; + regex_get_pdn(priv->pd, &pd_num); + wq_attr->pd = pd_num; + wq_attr->wq_type = MLX5_WQ_TYPE_CYCLIC; + wq_attr->dbr_umem_id = sq->dbr_umem; + wq_attr->dbr_addr = sq->dbr_offset; + wq_attr->dbr_umem_valid = 1; + wq_attr->wq_umem_id = mlx5_os_get_umem_id(sq->wqe_umem); + wq_attr->wq_umem_offset = 0; + wq_attr->wq_umem_valid = 1; + wq_attr->log_wq_stride = 6; + wq_attr->log_wq_sz = sq->log_nb_desc; + sq->obj = mlx5_devx_cmd_create_sq(priv->ctx, &attr); + if (!sq->obj) { + DRV_LOG(ERR, "Can't create sq object."); + rte_errno = ENOMEM; + goto error; + } + modify_attr.state = MLX5_SQC_STATE_RDY; + ret = mlx5_devx_cmd_modify_sq(sq->obj, &modify_attr); + if (ret) { + DRV_LOG(ERR, "Can't change sq state to ready."); + rte_errno = ENOMEM; + goto error; + } + + return 0; +error: + if (sq->wqe_umem) + mlx5_glue->devx_umem_dereg(sq->wqe_umem); + if (buf) + rte_free(buf); + if (sq->dbr_offset) + mlx5_release_dbr(&priv->dbrpgs, sq->dbr_umem, sq->dbr_offset); + return -rte_errno; +} + +/** + * Destroy the SQ object. + * + * @param priv + * Pointer to the priv object. + * @param qp + * Pointer to the QP element + * @param q_ind + * The index of the queue. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_destroy_sq(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, + uint16_t q_ind) +{ + struct mlx5_regex_sq *sq = &qp->sqs[q_ind]; + + if (sq->wqe_umem) { + mlx5_glue->devx_umem_dereg(sq->wqe_umem); + sq->wqe_umem = NULL; + } + if (sq->wqe) { + rte_free((void *)(uintptr_t)sq->wqe); + sq->wqe = NULL; + } + if (sq->dbr_offset) { + mlx5_release_dbr(&priv->dbrpgs, sq->dbr_umem, sq->dbr_offset); + sq->dbr_offset = -1; + } + if (sq->obj) { + mlx5_devx_cmd_destroy(sq->obj); + sq->obj = NULL; + } + return 0; +} + /** * Setup the qp. * @@ -168,7 +324,9 @@ { struct mlx5_regex_priv *priv = dev->data->dev_private; struct mlx5_regex_qp *qp; + int i; int ret; + uint16_t log_desc; qp = &priv->qps[qp_ind]; qp->flags = cfg->qp_conf_flags; @@ -185,15 +343,26 @@ return -rte_errno; } qp->cq.log_nb_desc = rte_log2_u32(cfg->nb_desc); + log_desc = rte_log2_u32(qp->nb_desc / qp->nb_obj); + qp->cq.log_nb_desc = log_desc; ret = regex_ctrl_create_cq(priv, &qp->cq); if (ret) { DRV_LOG(ERR, "Can't create cq."); goto error; } + for (i = 0; i < qp->nb_obj; i++) { + ret = regex_ctrl_create_sq(priv, qp, i, log_desc); + if (ret) { + DRV_LOG(ERR, "Can't create sq."); + goto error; + } + } return 0; error: regex_ctrl_destroy_cq(priv, &qp->cq); + for (i = 0; i < qp->nb_obj; i++) + ret = regex_ctrl_destroy_sq(priv, qp, i); return -rte_errno; } From patchwork Sun Jul 5 09:23:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73128 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4F073A00C5; Sun, 5 Jul 2020 11:27:11 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 10CB71DBD4; Sun, 5 Jul 2020 11:25:07 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 163D41DBC5 for ; Sun, 5 Jul 2020 11:25:03 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:25:00 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC42028191; Sun, 5 Jul 2020 12:24:59 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Shahaf Shuler Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 5 Jul 2020 09:23:43 +0000 Message-Id: <1593941027-86651-17-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 16/20] common/mlx5: add match tuple hw layout X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Add the found match tuple. Signed-off-by: Yuval Avnery Acked-by: Ori Kam Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_prm.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index bfbc58b..874dde6 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -409,6 +409,12 @@ struct mlx5_ifc_regexp_metadata_bits { uint8_t reserved[0x80]; }; +struct mlx5_ifc_regexp_match_tuple_bits { + uint8_t length[0x10]; + uint8_t start_ptr[0x10]; + uint8_t rule_id[0x20]; +}; + /* Adding direct verbs to data-path. */ /* CQ sequence number mask. */ From patchwork Sun Jul 5 09:23:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73126 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EFA96A00C5; Sun, 5 Jul 2020 11:26:51 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 643A71DBA0; Sun, 5 Jul 2020 11:25:04 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 1CD431DBC7 for ; Sun, 5 Jul 2020 11:25:03 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:25:01 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC43028191; Sun, 5 Jul 2020 12:25:01 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 5 Jul 2020 09:23:44 +0000 Message-Id: <1593941027-86651-18-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 17/20] regex/mlx5: fastpath setup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Allocated and register input/output buffers and metadata. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/regex/mlx5/Makefile | 1 + drivers/regex/mlx5/meson.build | 1 + drivers/regex/mlx5/mlx5_regex.h | 8 ++ drivers/regex/mlx5/mlx5_regex_control.c | 2 + drivers/regex/mlx5/mlx5_regex_fastpath.c | 192 +++++++++++++++++++++++++++++++ 5 files changed, 204 insertions(+) create mode 100644 drivers/regex/mlx5/mlx5_regex_fastpath.c diff --git a/drivers/regex/mlx5/Makefile b/drivers/regex/mlx5/Makefile index bc7e13e..dc2613e 100644 --- a/drivers/regex/mlx5/Makefile +++ b/drivers/regex/mlx5/Makefile @@ -10,6 +10,7 @@ LIB = librte_pmd_mlx5_regex.a SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_rxp.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_control.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_REGEX_PMD) += mlx5_regex_fastpath.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/regex/mlx5/meson.build b/drivers/regex/mlx5/meson.build index 165d660..3b1c3e4 100644 --- a/drivers/regex/mlx5/meson.build +++ b/drivers/regex/mlx5/meson.build @@ -13,6 +13,7 @@ sources = files( 'mlx5_regex.c', 'mlx5_rxp.c', 'mlx5_regex_control.c', + 'mlx5_regex_fastpath.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index d3267fa..3a63e33 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -44,6 +44,11 @@ struct mlx5_regex_qp { struct mlx5_regex_sq *sqs; /* Pointer to sq array. */ uint16_t nb_obj; /* Number of sq objects. */ struct mlx5_regex_cq cq; /* CQ struct. */ + uint32_t free_sqs; + struct mlx5_regex_job *jobs; + struct ibv_mr *metadata; + struct ibv_mr *inputs; + struct ibv_mr *outputs; }; struct mlx5_regex_db { @@ -82,4 +87,7 @@ int mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, const struct rte_regexdev_qp_conf *cfg); int mlx5_regex_rules_db_import(struct rte_regexdev *dev, const char *rule_db, uint32_t rule_db_len); + +/* mlx5_regex_fastpath.c */ +int mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index 08f60cc..f054184 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -357,6 +357,8 @@ goto error; } } + + mlx5_regexdev_setup_fastpath(priv, qp_ind); return 0; error: diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c new file mode 100644 index 0000000..7f82e1e --- /dev/null +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -0,0 +1,192 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2019 Mellanox Technologies, Ltd + */ +#include +#include + +#include +#include +#include +#include +#include +#include +#include + + +#include +#include +#include +#include +#include + +#include "mlx5_regex_utils.h" +#include "mlx5_rxp.h" +#include "mlx5_regex.h" + +/* Verbs header. */ +/* ISO C doesn't support unnamed structs/unions, disabling -pedantic. */ +#ifdef PEDANTIC +#pragma GCC diagnostic ignored "-Wpedantic" +#endif +#include +#ifdef PEDANTIC +#pragma GCC diagnostic error "-Wpedantic" +#endif + +#define MAX_WQE_INDEX 0xffff +#define MLX5_REGEX_METADATA_SIZE 64 +#define MLX5_REGEX_MAX_INPUT (1<<14) +#define MLX5_REGEX_MAX_OUTPUT (1<<11) + + +static inline uint32_t +sq_size_get(struct mlx5_regex_sq *sq) +{ + return (1U << sq->log_nb_desc); +} +static inline uint32_t +cq_size_get(struct mlx5_regex_cq *cq) +{ + return (1U << cq->log_nb_desc); +} + +struct mlx5_regex_job { + uint64_t user_id; + uint8_t *input; + volatile uint8_t *output; + volatile uint8_t *metadata; +} __rte_cached_aligned; + + +static MLX5DV_ALWAYS_INLINE +void mlx5dv_set_metadata_seg(struct mlx5_wqe_metadata_seg *seg, + uint32_t mmo_control_31_0, uint32_t lkey, + uintptr_t address) +{ + seg->mmo_control_31_0 = htobe32(mmo_control_31_0); + seg->lkey = htobe32(lkey); + seg->addr = htobe64(address); +} + +static void +setup_sqs(struct mlx5_regex_qp *queue) +{ + size_t sqid, entry; + uint32_t job_id; + for (sqid = 0; sqid < queue->nb_obj; sqid++) { + struct mlx5_regex_sq *sq = &queue->sqs[sqid]; + uint8_t *wqe = (uint8_t *)sq->wqe; + for (entry = 0 ; entry < sq_size_get(sq); entry++) { + job_id = sqid * sq_size_get(sq) + entry; + struct mlx5_regex_job *job = &queue->jobs[job_id]; + + mlx5dv_set_metadata_seg((struct mlx5_wqe_metadata_seg *) + (wqe + 16), + 0, queue->metadata->lkey, + (uintptr_t)job->metadata); + mlx5dv_set_data_seg((struct mlx5_wqe_data_seg *) + (wqe + 32), + 0, queue->inputs->lkey, + (uintptr_t)job->input); + mlx5dv_set_data_seg((struct mlx5_wqe_data_seg *) + (wqe + 48), + 1 << 11, queue->outputs->lkey, + (uintptr_t)job->output); + wqe += 64; + } + queue->free_sqs |= 1 << sqid; + } +} + +static int +setup_buffers(struct mlx5_regex_qp *qp, struct ibv_pd *pd) +{ + int i, err; + + void *ptr = rte_calloc(__func__, qp->nb_desc, + MLX5_REGEX_METADATA_SIZE, + MLX5_REGEX_METADATA_SIZE); + if (!ptr) + return -ENOMEM; + + qp->metadata = mlx5_glue->reg_mr(pd, ptr, + MLX5_REGEX_METADATA_SIZE*qp->nb_desc, + IBV_ACCESS_LOCAL_WRITE); + if (!qp->metadata) { + rte_free(ptr); + return -EINVAL; + } + ptr = rte_calloc(__func__, qp->nb_desc, + MLX5_REGEX_MAX_INPUT, + MLX5_REGEX_MAX_INPUT); + + if (!ptr) { + err = -ENOMEM; + goto err_input; + } + qp->inputs = mlx5_glue->reg_mr(pd, ptr, + MLX5_REGEX_MAX_INPUT*qp->nb_desc, + IBV_ACCESS_LOCAL_WRITE); + if (!qp->inputs) { + rte_free(ptr); + err = -EINVAL; + goto err_input; + } + + ptr = rte_calloc(__func__, qp->nb_desc, + MLX5_REGEX_MAX_OUTPUT, + MLX5_REGEX_MAX_OUTPUT); + if (!ptr) { + err = -ENOMEM; + goto err_output; + } + qp->outputs = mlx5_glue->reg_mr(pd, ptr, + MLX5_REGEX_MAX_OUTPUT*qp->nb_desc, + IBV_ACCESS_LOCAL_WRITE); + if (!qp->outputs) { + rte_free(ptr); + err = -EINVAL; + goto err_output; + } + + /* distribute buffers to jobs */ + for (i = 0; i < qp->nb_desc; i++) { + qp->jobs[i].input = + (uint8_t *)qp->inputs->addr + + (i%qp->nb_desc)*MLX5_REGEX_MAX_INPUT; + qp->jobs[i].output = + (uint8_t *)qp->outputs->addr + + (i%qp->nb_desc)*MLX5_REGEX_MAX_OUTPUT; + qp->jobs[i].metadata = + (uint8_t *)qp->metadata->addr + + (i%qp->nb_desc)*MLX5_REGEX_METADATA_SIZE; + } + return 0; + +err_output: + ptr = qp->inputs->addr; + rte_free(ptr); + mlx5_glue->dereg_mr(qp->inputs); +err_input: + ptr = qp->metadata->addr; + rte_free(ptr); + mlx5_glue->dereg_mr(qp->metadata); + return err; +} + +int +mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id) +{ + struct mlx5_regex_qp *qp = &priv->qps[qp_id]; + int err; + + qp->jobs = rte_calloc(__func__, qp->nb_desc, sizeof(*qp->jobs), + sizeof(*qp->jobs)); + if (!qp->jobs) + return -ENOMEM; + err = setup_buffers(qp, priv->pd); + if (err) + return err; + setup_sqs(qp); + return 0; +} From patchwork Sun Jul 5 09:23:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73129 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BF84AA00C5; Sun, 5 Jul 2020 11:27:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2C8051DB9C; Sun, 5 Jul 2020 11:25:09 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 250DC1DBDE for ; Sun, 5 Jul 2020 11:25:08 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:25:03 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC44028191; Sun, 5 Jul 2020 12:25:03 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 5 Jul 2020 09:23:45 +0000 Message-Id: <1593941027-86651-19-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 18/20] regex/mlx5: add enqueue implementation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Will look for a free SQ to send the job on. doorbell will be given when sq is full, or no more jobs on the burst. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 1 + drivers/regex/mlx5/mlx5_regex.h | 6 ++ drivers/regex/mlx5/mlx5_regex_control.c | 2 + drivers/regex/mlx5/mlx5_regex_fastpath.c | 125 +++++++++++++++++++++++++++++++ 4 files changed, 134 insertions(+) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 0f20cde..503bdf8 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -179,6 +179,7 @@ goto error; } priv->regexdev->dev_ops = &mlx5_regexdev_ops; + priv->regexdev->enqueue = mlx5_regexdev_enqueue; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; priv->regexdev->state = RTE_REGEXDEV_READY; diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 3a63e33..e28c7d3 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -25,6 +25,9 @@ struct mlx5_regex_sq { uint32_t dbr_umem; /* Door bell record umem id. */ uint8_t *wqe; /* The SQ ring buffer. */ struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ + size_t pi, db_pi; + size_t ci; + uint32_t sqn; uint32_t *dbr; }; @@ -49,6 +52,7 @@ struct mlx5_regex_qp { struct ibv_mr *metadata; struct ibv_mr *inputs; struct ibv_mr *outputs; + size_t ci, pi; }; struct mlx5_regex_db { @@ -90,4 +94,6 @@ int mlx5_regex_rules_db_import(struct rte_regexdev *dev, /* mlx5_regex_fastpath.c */ int mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id); +uint16_t mlx5_regexdev_enqueue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops); #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index f054184..c0ed0b5 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -220,6 +220,8 @@ sq->wqe = buf; sq->wqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, 64 * sq_size, 7); + sq->ci = 0; + sq->pi = 0; if (!sq->wqe_umem) { DRV_LOG(ERR, "Can't register wqe mem."); rte_errno = ENOMEM; diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c index 7f82e1e..072bc57 100644 --- a/drivers/regex/mlx5/mlx5_regex_fastpath.c +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -59,6 +59,131 @@ struct mlx5_regex_job { static MLX5DV_ALWAYS_INLINE +void mlx5_regex_set_ctrl_seg(void *seg, + uint8_t le, uint16_t subset_id0, + uint16_t subset_id1, uint16_t subset_id2, + uint16_t subset_id3, uint8_t ctrl) +{ + DEVX_SET(regexp_mmo_control, seg, le, le); + DEVX_SET(regexp_mmo_control, seg, ctrl, ctrl); + DEVX_SET(regexp_mmo_control, seg, subset_id_0, subset_id0); + DEVX_SET(regexp_mmo_control, seg, subset_id_1, subset_id1); + DEVX_SET(regexp_mmo_control, seg, subset_id_2, subset_id2); + DEVX_SET(regexp_mmo_control, seg, subset_id_3, subset_id3); +} + +static inline void +prep_one(struct mlx5_regex_sq *sq, struct rte_regex_ops *op, + struct mlx5_regex_job *job) +{ + memcpy(job->input, + rte_pktmbuf_mtod(op->mbuf, void *), + rte_pktmbuf_data_len(op->mbuf)); + + size_t wqe_offset = (sq->pi % sq_size_get(sq)) * MLX5_SEND_WQE_BB; + uint8_t *wqe = (uint8_t *)sq->wqe + wqe_offset; + int ds = 4; /* ctrl + meta + input + output */ + + mlx5dv_set_ctrl_seg((struct mlx5_wqe_ctrl_seg *)wqe, sq->pi, + MLX5_OPCODE_MMO, + MLX5_OPC_MOD_MMO_REGEX, sq->obj->id, + 0, ds, 0, 0); + + mlx5_regex_set_ctrl_seg(wqe+12, 0, op->group_id0, op->group_id1, + op->group_id2, + op->group_id3, 0); + + struct mlx5_wqe_data_seg *input_seg = + (struct mlx5_wqe_data_seg *)(wqe+32); + input_seg->byte_count = htobe32(rte_pktmbuf_data_len(op->mbuf)); + + job->user_id = op->user_id; + sq->db_pi = sq->pi; + sq->pi = (sq->pi+1)%MAX_WQE_INDEX; +} + +static inline void +send_doorbell(struct mlx5dv_devx_uar *uar, struct mlx5_regex_sq *sq) +{ + size_t wqe_offset = (sq->db_pi % sq_size_get(sq)) * MLX5_SEND_WQE_BB; + uint8_t *wqe = (uint8_t *)sq->wqe + wqe_offset; + ((struct mlx5_wqe_ctrl_seg *)wqe)->fm_ce_se = MLX5_WQE_CTRL_CQ_UPDATE; + uint64_t *doorbell_addr = + (uint64_t *)((uint8_t *)uar->base_addr + 0x800); + rte_cio_wmb(); + sq->dbr[MLX5_SND_DBR] = htobe32(sq->db_pi); + rte_wmb(); + *doorbell_addr = *(volatile uint64_t *)wqe; + rte_wmb(); +} + +static inline int +can_send(struct mlx5_regex_sq *sq) { + return unlikely(sq->ci > sq->pi) ? + MAX_WQE_INDEX + sq->pi - sq->ci < sq_size_get(sq) : + sq->pi - sq->ci < sq_size_get(sq); +} + +static inline uint32_t +job_id_get(uint32_t qid, size_t sq_size, size_t index) { + return qid*sq_size + index%sq_size; +} + +/** + * DPDK callback for enqueue. + * + * @param dev + * Pointer to the regex dev structure. + * @param qp_id + * The queue to enqueue the traffic to. + * @param ops + * List of regex ops to enqueue. + * @param nb_ops + * Number of ops in ops parameter. + * + * @return + * Number of packets successfully enqueued (<= pkts_n). + */ +uint16_t +mlx5_regexdev_enqueue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_regex_qp *queue = &priv->qps[qp_id]; + struct mlx5_regex_sq *sq; + size_t sqid, job_id, i = 0; + + while ((sqid = ffs(queue->free_sqs))) { + sqid--; /* ffs returns 1 for bit 0 */ + sq = &queue->sqs[sqid]; + while (can_send(sq)) { + job_id = job_id_get(sqid, sq_size_get(sq), sq->pi); + if (unlikely(rte_pktmbuf_data_len(ops[i]->mbuf) > + MLX5_REGEX_MAX_INPUT)) { + DRV_LOG(ERR, "Warning, Job input " + "size %d exceed %d bytes", + rte_pktmbuf_data_len(ops[i]->mbuf), + MLX5_REGEX_MAX_INPUT); + return 0; + } + prep_one(sq, ops[i], &queue->jobs[job_id]); + i++; + if (unlikely(i == nb_ops)) { + send_doorbell(priv->uar, sq); + goto out; + } + } + queue->free_sqs &= ~(1 << sqid); + send_doorbell(priv->uar, sq); + } + +out: + queue->pi += i; + return i; +} + + +static MLX5DV_ALWAYS_INLINE void mlx5dv_set_metadata_seg(struct mlx5_wqe_metadata_seg *seg, uint32_t mmo_control_31_0, uint32_t lkey, uintptr_t address) From patchwork Sun Jul 5 09:23:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73131 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 86C39A00C5; Sun, 5 Jul 2020 11:27:36 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0E0831DBE5; Sun, 5 Jul 2020 11:25:17 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 3BC251DC0B for ; Sun, 5 Jul 2020 11:25:13 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:25:11 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC45028191; Sun, 5 Jul 2020 12:25:05 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, thomas@monjalon.net, orika@mellanox.com, rasland@mellanox.com, Yuval Avnery Date: Sun, 5 Jul 2020 09:23:46 +0000 Message-Id: <1593941027-86651-20-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 19/20] regex/mlx5: implement dequeue function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yuval Avnery Implement dequeue function for the regex API. Signed-off-by: Yuval Avnery Acked-by: Ori Kam --- drivers/regex/mlx5/mlx5_regex.c | 1 + drivers/regex/mlx5/mlx5_regex.h | 4 ++ drivers/regex/mlx5/mlx5_regex_control.c | 1 + drivers/regex/mlx5/mlx5_regex_fastpath.c | 100 +++++++++++++++++++++++++++++++ 4 files changed, 106 insertions(+) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index 503bdf8..433ed17 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -180,6 +180,7 @@ } priv->regexdev->dev_ops = &mlx5_regexdev_ops; priv->regexdev->enqueue = mlx5_regexdev_enqueue; + priv->regexdev->dequeue = mlx5_regexdev_dequeue; priv->regexdev->device = (struct rte_device *)pci_dev; priv->regexdev->data->dev_private = priv; priv->regexdev->state = RTE_REGEXDEV_READY; diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index e28c7d3..390d9d4 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -38,6 +38,7 @@ struct mlx5_regex_cq { uint32_t dbr_umem; /* Door bell record umem id. */ volatile struct mlx5_cqe *cqe; /* The CQ ring buffer. */ struct mlx5dv_devx_umem *cqe_umem; /* CQ buffer umem. */ + size_t ci; uint32_t *dbr; }; @@ -96,4 +97,7 @@ int mlx5_regex_rules_db_import(struct rte_regexdev *dev, int mlx5_regexdev_setup_fastpath(struct mlx5_regex_priv *priv, uint32_t qp_id); uint16_t mlx5_regexdev_enqueue(struct rte_regexdev *dev, uint16_t qp_id, struct rte_regex_ops **ops, uint16_t nb_ops); +uint16_t mlx5_regexdev_dequeue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops); + #endif /* MLX5_REGEX_H */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index c0ed0b5..d17a6b9 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -124,6 +124,7 @@ cq->cqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, sizeof(struct mlx5_cqe) * cq_size, 7); + cq->ci = 0; if (!cq->cqe_umem) { DRV_LOG(ERR, "Can't register cqe mem."); rte_errno = ENOMEM; diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c index 072bc57..d754e91 100644 --- a/drivers/regex/mlx5/mlx5_regex_fastpath.c +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -182,6 +182,106 @@ void mlx5_regex_set_ctrl_seg(void *seg, return i; } +#define MLX5_REGEX_RESP_SZ 8 + +static inline void +extract_result(struct rte_regex_ops *op, struct mlx5_regex_job *job) +{ + size_t j, offset; + op->user_id = job->user_id; + op->nb_matches = DEVX_GET(regexp_metadata, job->metadata + 32, + match_count); + op->nb_actual_matches = DEVX_GET(regexp_metadata, job->metadata + 32, + detected_match_count); + for (j = 0; j < op->nb_matches; j++) { + offset = MLX5_REGEX_RESP_SZ * j; + op->matches[j].rule_id = + DEVX_GET(regexp_match_tuple, (job->output + offset), + rule_id); + op->matches[j].start_offset = + DEVX_GET(regexp_match_tuple, (job->output + offset), + start_ptr); + op->matches[j].len = + DEVX_GET(regexp_match_tuple, (job->output + offset), + length); + } +} + +static inline volatile struct mlx5_cqe * +poll_one(struct mlx5_regex_cq *cq) +{ + volatile struct mlx5_cqe *cqe; + size_t next_cqe_offset; + + next_cqe_offset = (cq->ci % cq_size_get(cq)) * sizeof(*cqe); + cqe = (volatile struct mlx5_cqe *)(cq->cqe + next_cqe_offset); + rte_cio_wmb(); + + int ret = check_cqe(cqe, cq_size_get(cq), cq->ci); + + if (unlikely(ret == MLX5_CQE_STATUS_ERR)) { + DRV_LOG(ERR, "Completion with error on qp 0x%x", 0); + exit(-1); + } + + if (unlikely(ret != MLX5_CQE_STATUS_SW_OWN)) + return NULL; + + return cqe; +} + + +/** + * DPDK callback for dequeue. + * + * @param dev + * Pointer to the regex dev structure. + * @param qp_id + * The queue to enqueue the traffic to. + * @param ops + * List of regex ops to dequeue. + * @param nb_ops + * Number of ops in ops parameter. + * + * @return + * Number of packets successfully dequeued (<= pkts_n). + */ +uint16_t +mlx5_regexdev_dequeue(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops) +{ + struct mlx5_regex_priv *priv = dev->data->dev_private; + struct mlx5_regex_qp *queue = &priv->qps[qp_id]; + struct mlx5_regex_cq *cq = &queue->cq; + volatile struct mlx5_cqe *cqe; + size_t i = 0; + + while ((cqe = poll_one(cq))) { + uint16_t wq_counter + = (be16toh(cqe->wqe_counter) + 1)%MAX_WQE_INDEX; + size_t sqid = cqe->rsvd3[2]; + struct mlx5_regex_sq *sq = &queue->sqs[sqid]; + while (sq->ci != wq_counter) { + if (unlikely(i == nb_ops)) { + /* Return without updating cq->ci */ + goto out; + } + uint32_t job_id = job_id_get(sqid, + sq_size_get(sq), sq->ci); + extract_result(ops[i], &queue->jobs[job_id]); + sq->ci = (sq->ci+1)%MAX_WQE_INDEX; + i++; + } + cq->ci = (cq->ci + 1) & 0xffffff; + asm volatile("" ::: "memory"); + cq->dbr[0] = htobe32(cq->ci); + queue->free_sqs |= (1 << sqid); + } + +out: + queue->ci += i; + return i; +} static MLX5DV_ALWAYS_INLINE void mlx5dv_set_metadata_seg(struct mlx5_wqe_metadata_seg *seg, From patchwork Sun Jul 5 09:23:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 73130 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E974A00C5; Sun, 5 Jul 2020 11:27:29 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 011381DBED; Sun, 5 Jul 2020 11:25:15 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 357411DB48 for ; Sun, 5 Jul 2020 11:25:13 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from orika@mellanox.com) with SMTP; 5 Jul 2020 12:25:07 +0300 Received: from pegasus04.mtr.labs.mlnx. (pegasus04.mtr.labs.mlnx [10.210.16.126]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0659OC46028191; Sun, 5 Jul 2020 12:25:07 +0300 From: Ori Kam To: jerinj@marvell.com, xiang.w.wang@intel.com, matan@mellanox.com, viacheslavo@mellanox.com, Thomas Monjalon Cc: guyk@marvell.com, dev@dpdk.org, pbhagavatula@marvell.com, shahafs@mellanox.com, hemant.agrawal@nxp.com, opher@mellanox.com, alexr@mellanox.com, dovrat@marvell.com, pkapoor@marvell.com, nipun.gupta@nxp.com, bruce.richardson@intel.com, yang.a.hong@intel.com, harry.chang@intel.com, gu.jian1@zte.com.cn, shanjiangh@chinatelecom.cn, zhangy.yun@chinatelecom.cn, lixingfu@huachentel.com, wushuai@inspur.com, yuyingxia@yxlink.com, fanchenggang@sunyainfo.com, davidfgao@tencent.com, liuzhong1@chinaunicom.cn, zhaoyong11@huawei.com, oc@yunify.com, jim@netgate.com, hongjun.ni@intel.com, deri@ntop.org, fc@napatech.com, arthur.su@lionic.com, orika@mellanox.com, rasland@mellanox.com Date: Sun, 5 Jul 2020 09:23:47 +0000 Message-Id: <1593941027-86651-21-git-send-email-orika@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593941027-86651-1-git-send-email-orika@mellanox.com> References: <1593941027-86651-1-git-send-email-orika@mellanox.com> Subject: [dpdk-dev] [PATCH 20/20] maintainers: add maintainers to regexdev lib X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds a maintainers to RegEx lib and to the new Mellanox RegEx PMD. Signed-off-by: Ori Kam --- MAINTAINERS | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 53a5e9a..d11306a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -447,6 +447,11 @@ F: app/test/test_compressdev* F: doc/guides/prog_guide/compressdev.rst F: doc/guides/compressdevs/features/default.ini +RegEx API +M: Ori Kam +F: lib/librte_regexdev/ +F: doc/guides/prog_guid/regexdev_lib.rst + Eventdev API M: Jerin Jacob T: git://dpdk.org/next/dpdk-next-eventdev @@ -1189,6 +1194,16 @@ F: drivers/event/opdl/ F: doc/guides/eventdevs/opdl.rst +RegEx Drivers +------------------ +M: Ori Kam +F: drivers/regex/ + +Mellanox BF2 +M: Ori Kam +F: drivers/regex/mlx5/ + + Rawdev Drivers --------------