From patchwork Wed Aug 26 15:14:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 75996 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F0E6AA04B4; Wed, 26 Aug 2020 17:15:12 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C18A41BE80; Wed, 26 Aug 2020 17:15:05 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id F3B36B62 for ; Wed, 26 Aug 2020 17:15:02 +0200 (CEST) IronPort-SDR: OxDKUKs2mIAcEgQSTFdx5dGSd+ewy/aI2bhBDTE/Hs/V0BU8GOrk/GYd4t3hW8nuGHRKkSSjPQ K+Gy+MJPRhpA== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879477" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879477" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:48 -0700 IronPort-SDR: z1xrZjfxgr8yGw7NsY820B2H7MNYnLVbibmVSX+6a0y3fT8Mlgj/NtTPYI6pYOIr8xTc+2x9v2 Xl//ZjjY7ktQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081252" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:47 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:06 +0100 Message-Id: <20200826151445.51500-2-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 01/40] pipeline: add pipeline X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add improved pipeline type that supports dynamically-defined packet headers, meta-data, actions and pipelines. Actions and pipelines are defined through instructions. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/Makefile | 2 + lib/librte_pipeline/meson.build | 10 ++- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 70 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 79 ++++++++++++++++++++ 5 files changed, 162 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index cfbbd1828..32582db9e 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -21,8 +21,10 @@ EXPORT_MAP := rte_pipeline_version.map SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := rte_pipeline.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_port_in_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_table_action.c +SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_pipeline.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d70b1a023..880c2b274 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -1,6 +1,12 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c') -headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h') +sources = files('rte_pipeline.c', + 'rte_port_in_action.c', + 'rte_table_action.c', + 'rte_swx_pipeline.c',) +headers = files('rte_pipeline.h', + 'rte_port_in_action.h', + 'rte_table_action.h', + 'rte_swx_pipeline.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 9ed80eb04..39593f1ee 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -55,4 +55,7 @@ EXPERIMENTAL { rte_table_action_time_read; rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; + rte_swx_pipeline_config; + rte_swx_pipeline_build; + rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c new file mode 100644 index 000000000..2319d4570 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include +#include +#include +#include + +#include + +#include "rte_swx_pipeline.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define CHECK_NAME(name, err_code) \ + CHECK((name) && (name)[0], err_code) + +/* + * Pipeline. + */ +struct rte_swx_pipeline { + int build_done; + int numa_node; +}; + + +/* + * Pipeline. + */ +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) +{ + struct rte_swx_pipeline *pipeline; + + /* Check input parameters. */ + CHECK(p, EINVAL); + + /* Memory allocation. */ + pipeline = calloc(1, sizeof(struct rte_swx_pipeline)); + CHECK(pipeline, ENOMEM); + + /* Initialization. */ + pipeline->numa_node = numa_node; + + *p = pipeline; + return 0; +} + +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p) +{ + if (!p) + return; + + free(p); +} + +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p) +{ + CHECK(p, EINVAL); + CHECK(p->build_done == 0, EEXIST); + + p->build_done = 1; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h new file mode 100644 index 000000000..ded26a4e4 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PIPELINE_H__ +#define __INCLUDE_RTE_SWX_PIPELINE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline + */ + +#include +#include + +#include + +/* + * Pipeline setup and operation + */ + +/** Pipeline opaque data structure. */ +struct rte_swx_pipeline; + +/** + * Pipeline configure + * + * @param[out] p + * Pipeline handle. Must point to valid memory. Contains valid pipeline handle + * when the function returns successfully. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, + int numa_node); + +/** + * Pipeline build + * + * Once called, the pipeline build operation marks the end of pipeline + * configuration. At this point, all the internal data structures needed to run + * the pipeline are built. + * + * @param[in] p + * Pipeline handle. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Pipeline was already built successfully. + */ +__rte_experimental +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p); + +/** + * Pipeline free + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p); + +#ifdef __cplusplus +} +#endif + +#endif From patchwork Wed Aug 26 15:14:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 75997 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 69CCBA04B4; Wed, 26 Aug 2020 17:15:24 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 355B71BECC; Wed, 26 Aug 2020 17:15:08 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id ED528A3 for ; Wed, 26 Aug 2020 17:15:02 +0200 (CEST) IronPort-SDR: o3cn974pRHGdu1PdcGMEqIEujRweD/KBXE4hFABRvqlX3tBgS4HBGNzOLTcEYSyzRitq7s3m3T 8NHR3tLQvNBA== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879480" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879480" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:49 -0700 IronPort-SDR: c+xUf4Y8yPlvIc25ohZcCktWkaHCdB0rWY0rzKh379BflD9sj7tkvG4Gbk8BNh5RhzX5ChZ8tn zsPhidH31RtQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081258" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:48 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:07 +0100 Message-Id: <20200826151445.51500-3-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 02/40] pipeline: add input port X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add input ports to the pipeline. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The RX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 209 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 54 +++++ lib/librte_port/Makefile | 1 + lib/librte_port/meson.build | 3 +- lib/librte_port/rte_swx_port.h | 118 +++++++++++ 6 files changed, 386 insertions(+), 1 deletion(-) create mode 100644 lib/librte_port/rte_swx_port.h diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 39593f1ee..a9ebd3b1f 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -56,6 +56,8 @@ EXPERIMENTAL { rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; rte_swx_pipeline_config; + rte_swx_pipeline_port_in_type_register; + rte_swx_pipeline_port_in_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2319d4570..5b1559209 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5,6 +5,7 @@ #include #include #include +#include #include @@ -19,14 +20,206 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Input port. + */ +struct port_in_type { + TAILQ_ENTRY(port_in_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_in_ops ops; +}; + +TAILQ_HEAD(port_in_type_tailq, port_in_type); + +struct port_in { + TAILQ_ENTRY(port_in) node; + struct port_in_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_in_tailq, port_in); + +struct port_in_runtime { + rte_swx_port_in_pkt_rx_t pkt_rx; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { + struct port_in_type_tailq port_in_types; + struct port_in_tailq ports_in; + + struct port_in_runtime *in; + + uint32_t n_ports_in; int build_done; int numa_node; }; +/* + * Input port. + */ +static struct port_in_type * +port_in_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_in_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_in_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops) +{ + struct port_in_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_rx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_in_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_in_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_in_types, elem, node); + + return 0; +} + +static struct port_in * +port_in_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_in *port; + + TAILQ_FOREACH(port, &p->ports_in, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_in_type *type = NULL; + struct port_in *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_in_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_in_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_in)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_in, port, node); + if (p->n_ports_in < port_id + 1) + p->n_ports_in = port_id + 1; + + return 0; +} + +static int +port_in_build(struct rte_swx_pipeline *p) +{ + struct port_in *port; + uint32_t i; + + CHECK(p->n_ports_in, EINVAL); + CHECK(rte_is_power_of_2(p->n_ports_in), EINVAL); + + for (i = 0; i < p->n_ports_in; i++) + CHECK(port_in_find(p, i), EINVAL); + + p->in = calloc(p->n_ports_in, sizeof(struct port_in_runtime)); + CHECK(p->in, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_in, node) { + struct port_in_runtime *in = &p->in[port->id]; + + in->pkt_rx = port->type->ops.pkt_rx; + in->obj = port->obj; + } + + return 0; +} + +static void +port_in_build_free(struct rte_swx_pipeline *p) +{ + free(p->in); + p->in = NULL; +} + +static void +port_in_free(struct rte_swx_pipeline *p) +{ + port_in_build_free(p); + + /* Input ports. */ + for ( ; ; ) { + struct port_in *port; + + port = TAILQ_FIRST(&p->ports_in); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_in, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Input port types. */ + for ( ; ; ) { + struct port_in_type *elem; + + elem = TAILQ_FIRST(&p->port_in_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_in_types, elem, node); + free(elem); + } +} /* * Pipeline. @@ -44,6 +237,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->port_in_types); + TAILQ_INIT(&pipeline->ports_in); + pipeline->numa_node = numa_node; *p = pipeline; @@ -56,15 +252,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_in_free(p); + free(p); } int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { + int status; + CHECK(p, EINVAL); CHECK(p->build_done == 0, EEXIST); + status = port_in_build(p); + if (status) + goto error; + p->build_done = 1; return 0; + +error: + port_in_build_free(p); + + return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index ded26a4e4..3dbe7ce0b 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -18,6 +18,12 @@ extern "C" { #include +#include "rte_swx_port.h" + +/** Name size. */ +#ifndef RTE_SWX_NAME_SIZE +#define RTE_SWX_NAME_SIZE 64 +#endif /* * Pipeline setup and operation */ @@ -43,6 +49,54 @@ int rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node); +/* + * Pipeline input ports + */ + +/** + * Pipeline input port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Input port type name. + * @param[in] ops + * Input port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Input port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops); + +/** + * Pipeline input port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Input port ID. + * @param[in] port_type_name + * Existing input port type name. + * @param[in] args + * Input port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Input port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); /** * Pipeline build * diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile index 57d2aedbc..4221618b3 100644 --- a/lib/librte_port/Makefile +++ b/lib/librte_port/Makefile @@ -55,5 +55,6 @@ endif SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_source_sink.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_sym_crypto.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_eventdev.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 0d5ede44a..5b5fbf6c4 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -21,7 +21,8 @@ headers = files( 'rte_port_sched.h', 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', - 'rte_port_eventdev.h') + 'rte_port_eventdev.h', + 'rte_swx_port.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h new file mode 100644 index 000000000..a6f80de9a --- /dev/null +++ b/lib/librte_port/rte_swx_port.h @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_H__ +#define __INCLUDE_RTE_SWX_PORT_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Port + * + * Packet I/O port interface. + */ + +#include + +/** Packet. */ +struct rte_swx_pkt { + /** Opaque packet handle. */ + void *handle; + + /** Buffer where the packet is stored. */ + uint8_t *pkt; + + /** Packet buffer offset of the first packet byte. */ + uint32_t offset; + + /** Packet length in bytes. */ + uint32_t length; +}; + +/* + * Input port + */ + +/** + * Input port create + * + * @param[in] args + * Arguments for input port creation. Format specific to each port type. + * @return + * Handle to input port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_in_create_t)(void *args); + +/** + * Input port free + * + * @param[in] args + * Input port handle. + */ +typedef void +(*rte_swx_port_in_free_t)(void *port); + +/** + * Input port packet receive + * + * @param[in] port + * Input port handle. + * @param[out] pkt + * Received packet. Only valid when the function returns 1. Must point to + * valid memory. + * @return + * 0 when no packet was received, 1 when a packet was received. No other + * return values are allowed. + */ +typedef int +(*rte_swx_port_in_pkt_rx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** Input port statistics counters. */ +struct rte_swx_port_in_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; + + /** Number of empty polls. */ + uint64_t n_empty; +}; + +/** + * Input port statistics counters read + * + * @param[in] port + * Input port handle. + * @param[out] stats + * Input port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_in_stats_read_t)(void *port, + struct rte_swx_port_in_stats *stats); + +/** Input port operations. */ +struct rte_swx_port_in_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_in_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_in_free_t free; + + /** Packet reception. Must be non-NULL. */ + rte_swx_port_in_pkt_rx_t pkt_rx; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_in_stats_read_t stats_read; +}; + +#ifdef __cplusplus +} +#endif + +#endif From patchwork Wed Aug 26 15:14:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 75998 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D4A8FA04B4; Wed, 26 Aug 2020 17:15:37 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3E0371BFE1; Wed, 26 Aug 2020 17:15:10 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id CB9EBAAB7 for ; Wed, 26 Aug 2020 17:15:03 +0200 (CEST) IronPort-SDR: xVnuWvCj+wU6TcAiC6JJ1HhwQfBzoa7FFj7BmgY22zUuwZCaf6Cf8iA/SOWX9QdIfMqjDEdqik ztD9k0Lxuqrw== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879483" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879483" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:50 -0700 IronPort-SDR: YtRLPWdyVCGaxVCuSD/NXawxx34fyPtxdmYTV7Ur+hxvruqA3rcQgupyV8TT1yNEV36BJc6bVK UBzmCM8JZycA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081263" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:49 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:08 +0100 Message-Id: <20200826151445.51500-4-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 03/40] pipeline: add output port X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add output ports to the pipeline. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The TX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 200 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 50 +++++ lib/librte_port/rte_swx_port.h | 84 ++++++++ 4 files changed, 336 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index a9ebd3b1f..88fd38ca8 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -58,6 +58,8 @@ EXPERIMENTAL { rte_swx_pipeline_config; rte_swx_pipeline_port_in_type_register; rte_swx_pipeline_port_in_config; + rte_swx_pipeline_port_out_type_register; + rte_swx_pipeline_port_out_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 5b1559209..7aeac8cc8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -45,16 +45,46 @@ struct port_in_runtime { void *obj; }; +/* + * Output port. + */ +struct port_out_type { + TAILQ_ENTRY(port_out_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_out_ops ops; +}; + +TAILQ_HEAD(port_out_type_tailq, port_out_type); + +struct port_out { + TAILQ_ENTRY(port_out) node; + struct port_out_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_out_tailq, port_out); + +struct port_out_runtime { + rte_swx_port_out_pkt_tx_t pkt_tx; + rte_swx_port_out_flush_t flush; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; + struct port_out_type_tailq port_out_types; + struct port_out_tailq ports_out; struct port_in_runtime *in; + struct port_out_runtime *out; uint32_t n_ports_in; + uint32_t n_ports_out; int build_done; int numa_node; }; @@ -221,6 +251,168 @@ port_in_free(struct rte_swx_pipeline *p) } } +/* + * Output port. + */ +static struct port_out_type * +port_out_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_out_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_out_types, node) + if (!strcmp(elem->name, name)) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops) +{ + struct port_out_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_tx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_out_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_out_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_out_types, elem, node); + + return 0; +} + +static struct port_out * +port_out_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_out *port; + + TAILQ_FOREACH(port, &p->ports_out, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_out_type *type = NULL; + struct port_out *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_out_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_out_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_out)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_out, port, node); + if (p->n_ports_out < port_id + 1) + p->n_ports_out = port_id + 1; + + return 0; +} + +static int +port_out_build(struct rte_swx_pipeline *p) +{ + struct port_out *port; + uint32_t i; + + CHECK(p->n_ports_out, EINVAL); + + for (i = 0; i < p->n_ports_out; i++) + CHECK(port_out_find(p, i), EINVAL); + + p->out = calloc(p->n_ports_out, sizeof(struct port_out_runtime)); + CHECK(p->out, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_out, node) { + struct port_out_runtime *out = &p->out[port->id]; + + out->pkt_tx = port->type->ops.pkt_tx; + out->flush = port->type->ops.flush; + out->obj = port->obj; + } + + return 0; +} + +static void +port_out_build_free(struct rte_swx_pipeline *p) +{ + free(p->out); + p->out = NULL; +} + +static void +port_out_free(struct rte_swx_pipeline *p) +{ + port_out_build_free(p); + + /* Output ports. */ + for ( ; ; ) { + struct port_out *port; + + port = TAILQ_FIRST(&p->ports_out); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_out, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Output port types. */ + for ( ; ; ) { + struct port_out_type *elem; + + elem = TAILQ_FIRST(&p->port_out_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_out_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -239,6 +431,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) /* Initialization. */ TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); + TAILQ_INIT(&pipeline->port_out_types); + TAILQ_INIT(&pipeline->ports_out); pipeline->numa_node = numa_node; @@ -252,6 +446,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_out_free(p); port_in_free(p); free(p); @@ -269,10 +464,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = port_out_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + port_out_build_free(p); port_in_build_free(p); return status; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 3dbe7ce0b..2be83bd35 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -97,6 +97,56 @@ rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, uint32_t port_id, const char *port_type_name, void *args); + +/* + * Pipeline output ports + */ + +/** + * Pipeline output port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Output port type name. + * @param[in] ops + * Output port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Output port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops); + +/** + * Pipeline output port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Output port ID. + * @param[in] port_type_name + * Existing output port type name. + * @param[in] args + * Output port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Output port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); + /** * Pipeline build * diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h index a6f80de9a..4beb59991 100644 --- a/lib/librte_port/rte_swx_port.h +++ b/lib/librte_port/rte_swx_port.h @@ -111,6 +111,90 @@ struct rte_swx_port_in_ops { rte_swx_port_in_stats_read_t stats_read; }; +/* + * Output port + */ + +/** + * Output port create + * + * @param[in] args + * Arguments for output port creation. Format specific to each port type. + * @return + * Handle to output port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_out_create_t)(void *args); + +/** + * Output port free + * + * @param[in] args + * Output port handle. + */ +typedef void +(*rte_swx_port_out_free_t)(void *port); + +/** + * Output port packet transmit + * + * @param[in] port + * Output port handle. + * @param[in] pkt + * Packet to be transmitted. + */ +typedef void +(*rte_swx_port_out_pkt_tx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** + * Output port flush + * + * @param[in] port + * Output port handle. + */ +typedef void +(*rte_swx_port_out_flush_t)(void *port); + +/** Output port statistics counters. */ +struct rte_swx_port_out_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; +}; + +/** + * Output port statistics counters read + * + * @param[in] port + * Output port handle. + * @param[out] stats + * Output port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_out_stats_read_t)(void *port, + struct rte_swx_port_out_stats *stats); + +/** Output port operations. */ +struct rte_swx_port_out_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_out_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_out_free_t free; + + /** Packet transmission. Must be non-NULL. */ + rte_swx_port_out_pkt_tx_t pkt_tx; + + /** Flush. May be NULL. */ + rte_swx_port_out_flush_t flush; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_out_stats_read_t stats_read; +}; + #ifdef __cplusplus } #endif From patchwork Wed Aug 26 15:14:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 75999 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A49FCA04B4; Wed, 26 Aug 2020 17:15:51 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 71C6C1C08E; Wed, 26 Aug 2020 17:15:11 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id ED5FC14581 for ; Wed, 26 Aug 2020 17:15:03 +0200 (CEST) IronPort-SDR: 6prBRT95rUYX0VcsPFsYE3kINLrBt/CR9SJ6qZgWp2TsVhCNn/gdT2JrOBiHSSggrfg6Bxd0hd kkk1lNJ/2Fcg== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879491" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879491" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:51 -0700 IronPort-SDR: ZJbS+ufdKZ4/prIxHW5/bgNsj+frLsyCIJrjaA85kWPZaxUUYH8u8l5TZSl2sdTAFYGtPBX5YW gEKgpaEivkNw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081269" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:50 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:09 +0100 Message-Id: <20200826151445.51500-5-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 04/40] pipeline: add headers and meta-data X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support for dynamically-defined packet headers and meta-data. The header and meta-data format are defined by the struct type they instantiate. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 413 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 85 ++++ 3 files changed, 501 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 88fd38ca8..6a48c3666 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,9 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_struct_type_register; + rte_swx_pipeline_packet_header_register; + rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 7aeac8cc8..cb2e32b83 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -20,6 +20,25 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Struct. + */ +struct field { + char name[RTE_SWX_NAME_SIZE]; + uint32_t n_bits; + uint32_t offset; +}; + +struct struct_type { + TAILQ_ENTRY(struct_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct field *fields; + uint32_t n_fields; + uint32_t n_bits; +}; + +TAILQ_HEAD(struct_type_tailq, struct_type); + /* * Input port. */ @@ -71,24 +90,198 @@ struct port_out_runtime { void *obj; }; +/* + * Header. + */ +struct header { + TAILQ_ENTRY(header) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(header_tailq, header); + +struct header_runtime { + uint8_t *ptr0; +}; + +struct header_out_runtime { + uint8_t *ptr0; + uint8_t *ptr; + uint32_t n_bytes; +}; + /* * Pipeline. */ +struct thread { + /* Structures. */ + uint8_t **structs; + + /* Packet headers. */ + struct header_runtime *headers; /* Extracted or generated headers. */ + struct header_out_runtime *headers_out; /* Emitted headers. */ + uint8_t *header_storage; + uint8_t *header_out_storage; + uint64_t valid_headers; + uint32_t n_headers_out; + + /* Packet meta-data. */ + uint8_t *metadata; +}; + +#ifndef RTE_SWX_PIPELINE_THREADS_MAX +#define RTE_SWX_PIPELINE_THREADS_MAX 16 +#endif + struct rte_swx_pipeline { + struct struct_type_tailq struct_types; struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct header_tailq headers; + struct struct_type *metadata_st; + uint32_t metadata_struct_id; struct port_in_runtime *in; struct port_out_runtime *out; + struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; + uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_headers; int build_done; int numa_node; }; +/* + * Struct. + */ +static struct struct_type * +struct_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct struct_type *elem; + + TAILQ_FOREACH(elem, &p->struct_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields) +{ + struct struct_type *st; + uint32_t i; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(fields, EINVAL); + CHECK(n_fields, EINVAL); + + for (i = 0; i < n_fields; i++) { + struct rte_swx_field_params *f = &fields[i]; + uint32_t j; + + CHECK_NAME(f->name, EINVAL); + CHECK(f->n_bits, EINVAL); + CHECK(f->n_bits <= 64, EINVAL); + CHECK((f->n_bits & 7) == 0, EINVAL); + + for (j = 0; j < i; j++) { + struct rte_swx_field_params *f_prev = &fields[j]; + + CHECK(strcmp(f->name, f_prev->name), EINVAL); + } + } + + CHECK(!struct_type_find(p, name), EEXIST); + + /* Node allocation. */ + st = calloc(1, sizeof(struct struct_type)); + CHECK(st, ENOMEM); + + st->fields = calloc(n_fields, sizeof(struct field)); + if (!st->fields) { + free(st); + CHECK(0, ENOMEM); + } + + /* Node initialization. */ + strcpy(st->name, name); + for (i = 0; i < n_fields; i++) { + struct field *dst = &st->fields[i]; + struct rte_swx_field_params *src = &fields[i]; + + strcpy(dst->name, src->name); + dst->n_bits = src->n_bits; + dst->offset = st->n_bits; + + st->n_bits += src->n_bits; + } + st->n_fields = n_fields; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->struct_types, st, node); + + return 0; +} + +static int +struct_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + t->structs = calloc(p->n_structs, sizeof(uint8_t *)); + CHECK(t->structs, ENOMEM); + } + + return 0; +} + +static void +struct_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->structs); + t->structs = NULL; + } +} + +static void +struct_free(struct rte_swx_pipeline *p) +{ + struct_build_free(p); + + /* Struct types. */ + for ( ; ; ) { + struct struct_type *elem; + + elem = TAILQ_FIRST(&p->struct_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->struct_types, elem, node); + free(elem->fields); + free(elem); + } +} + /* * Input port. */ @@ -413,6 +606,205 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Header. + */ +static struct header * +header_find(struct rte_swx_pipeline *p, const char *name) +{ + struct header *elem; + + TAILQ_FOREACH(elem, &p->headers, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name) +{ + struct struct_type *st; + struct header *h; + size_t n_headers_max; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK_NAME(struct_type_name, EINVAL); + + CHECK(!header_find(p, name), EEXIST); + + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + + n_headers_max = RTE_SIZEOF_FIELD(struct thread, valid_headers) * 8; + CHECK(p->n_headers < n_headers_max, ENOSPC); + + /* Node allocation. */ + h = calloc(1, sizeof(struct header)); + CHECK(h, ENOMEM); + + /* Node initialization. */ + strcpy(h->name, name); + h->st = st; + h->struct_id = p->n_structs; + h->id = p->n_headers; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->headers, h, node); + p->n_headers++; + p->n_structs++; + + return 0; +} + +static int +header_build(struct rte_swx_pipeline *p) +{ + struct header *h; + uint32_t n_bytes = 0, i; + + TAILQ_FOREACH(h, &p->headers, node) { + n_bytes += h->st->n_bits / 8; + } + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t offset = 0; + + t->headers = calloc(p->n_headers, + sizeof(struct header_runtime)); + CHECK(t->headers, ENOMEM); + + t->headers_out = calloc(p->n_headers, + sizeof(struct header_out_runtime)); + CHECK(t->headers_out, ENOMEM); + + t->header_storage = calloc(1, n_bytes); + CHECK(t->header_storage, ENOMEM); + + t->header_out_storage = calloc(1, n_bytes); + CHECK(t->header_out_storage, ENOMEM); + + TAILQ_FOREACH(h, &p->headers, node) { + uint8_t *header_storage; + + header_storage = &t->header_storage[offset]; + offset += h->st->n_bits / 8; + + t->headers[h->id].ptr0 = header_storage; + t->structs[h->struct_id] = header_storage; + } + } + + return 0; +} + +static void +header_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->headers_out); + t->headers_out = NULL; + + free(t->headers); + t->headers = NULL; + + free(t->header_out_storage); + t->header_out_storage = NULL; + + free(t->header_storage); + t->header_storage = NULL; + } +} + +static void +header_free(struct rte_swx_pipeline *p) +{ + header_build_free(p); + + for ( ; ; ) { + struct header *elem; + + elem = TAILQ_FIRST(&p->headers); + if (!elem) + break; + + TAILQ_REMOVE(&p->headers, elem, node); + free(elem); + } +} + +/* + * Meta-data. + */ +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name) +{ + struct struct_type *st = NULL; + + CHECK(p, EINVAL); + + CHECK_NAME(struct_type_name, EINVAL); + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + CHECK(!p->metadata_st, EINVAL); + + p->metadata_st = st; + p->metadata_struct_id = p->n_structs; + + p->n_structs++; + + return 0; +} + +static int +metadata_build(struct rte_swx_pipeline *p) +{ + uint32_t n_bytes = p->metadata_st->n_bits / 8; + uint32_t i; + + /* Thread-level initialization. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint8_t *metadata; + + metadata = calloc(1, n_bytes); + CHECK(metadata, ENOMEM); + + t->metadata = metadata; + t->structs[p->metadata_struct_id] = metadata; + } + + return 0; +} + +static void +metadata_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->metadata); + t->metadata = NULL; + } +} + +static void +metadata_free(struct rte_swx_pipeline *p) +{ + metadata_build_free(p); +} + /* * Pipeline. */ @@ -429,11 +821,14 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->struct_types); TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->headers); + pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; *p = pipeline; @@ -446,8 +841,11 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + metadata_free(p); + header_free(p); port_out_free(p); port_in_free(p); + struct_free(p); free(p); } @@ -468,12 +866,27 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = struct_build(p); + if (status) + goto error; + + status = header_build(p); + if (status) + goto error; + + status = metadata_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + metadata_build_free(p); + header_build_free(p); port_out_build_free(p); port_in_build_free(p); + struct_build_free(p); return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2be83bd35..4a7b679a4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -147,6 +147,91 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Packet headers and meta-data + */ + +/** Structure (struct) field. */ +struct rte_swx_field_params { + /** Struct field name. */ + const char *name; + + /** Struct field size (in bits). + * Restriction: All struct fields must be a multiple of 8 bits. + * Restriction: All struct fields must be no greater than 64 bits. + */ + uint32_t n_bits; +}; + +/** + * Pipeline struct type register + * + * Structs are used extensively in many part of the pipeline to define the size + * and layout of a specific memory piece such as: headers, meta-data, action + * data stored in a table entry, mailboxes for extern objects and functions. + * Similar to C language structs, they are a well defined sequence of fields, + * with each field having a unique name and a constant size. + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Struct type name. + * @param[in] fields + * The sequence of struct fields. + * @param[in] n_fields + * The number of struct fields. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Struct type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields); + +/** + * Pipeline packet header register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Header name. + * @param[in] struct_type_name + * The struct type instantiated by this packet header. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Header with this name already exists; + * -ENOSPC: Maximum number of headers reached for the pipeline. + */ +__rte_experimental +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name); + +/** + * Pipeline packet meta-data register + * + * @param[in] p + * Pipeline handle. + * @param[in] struct_type_name + * The struct type instantiated by the packet meta-data. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name); + + /** * Pipeline build * From patchwork Wed Aug 26 15:14:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76000 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F2780A04B4; Wed, 26 Aug 2020 17:16:03 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B18F21C0B0; Wed, 26 Aug 2020 17:15:12 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 13A6D14583 for ; Wed, 26 Aug 2020 17:15:03 +0200 (CEST) IronPort-SDR: /7TZ49Si1PmAGQY5+99P1VZOYt9jpITtTEsKvIXfiVG9BmES873nntDkaRdsPuJtyKDnCOCkIi 7v7eKB4C0FTw== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879496" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879496" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:52 -0700 IronPort-SDR: RyvykP2vJSLHqow8s2fmvxjI7VTZglq43tTpHtd5LE+56Cc3wrJVz3Pe9R1bMtb0Aq0yZ7amX/ IAPcjNunHXMA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081271" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:51 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:10 +0100 Message-Id: <20200826151445.51500-6-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 05/40] pipeline: add extern objects and functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add extern objects and functions to plug in functions that cannot be efficiently implemented with existing instructions, e.g. special checksum/ECC, crypto, meters, stats arrays, heuristics, etc. In/out arguments are passed through mailbox with format defined by struct. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/Makefile | 1 + lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_extern.h | 98 ++++ lib/librte_pipeline/rte_swx_pipeline.c | 477 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 113 +++++ 6 files changed, 695 insertions(+), 1 deletion(-) create mode 100644 lib/librte_pipeline/rte_swx_extern.h diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index 32582db9e..23bfd88e6 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -25,6 +25,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_extern.h SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_pipeline.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index 880c2b274..bea406848 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -8,5 +8,6 @@ sources = files('rte_pipeline.c', headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', - 'rte_swx_pipeline.h',) + 'rte_swx_pipeline.h', + 'rte_swx_extern.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 6a48c3666..4297e185d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,10 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_extern_type_register; + rte_swx_pipeline_extern_type_member_func_register; + rte_swx_pipeline_extern_object_config; + rte_swx_pipeline_extern_func_register; rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; diff --git a/lib/librte_pipeline/rte_swx_extern.h b/lib/librte_pipeline/rte_swx_extern.h new file mode 100644 index 000000000..e10e963d6 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_extern.h @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_EXTERN_H__ +#define __INCLUDE_RTE_SWX_EXTERN_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Extern objects and functions + * + * Extern object and extern function interfaces. The extern objects and extern + * functions provide the mechanisms to hook external functionality into the + * packet processing pipeline. + */ + +#include + +/* + * Extern type + */ + +/** + * Extern object constructor + * + * @param[in] args + * Extern object constructor arguments. It may be NULL. + * @return + * Extern object handle. + */ +typedef void * +(*rte_swx_extern_type_constructor_t)(const char *args); + +/** + * Extern object destructor + * + * @param[in] object + * Extern object handle. + */ +typedef void +(*rte_swx_extern_type_destructor_t)(void *object); + +/** + * Extern object member function + * + * The mailbox is used to pass input arguments to the member function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same member function for the same extern object. + * + * Multiple invocations of the same member function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the member + * function must be invoked again with exactly the same object and mailbox + * arguments. + * + * @param[in] object + * Extern object handle. + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_type_member_func_t)(void *object, void *mailbox); + +/* + * Extern function + */ + +/** The mailbox is used to pass input arguments to the extern function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same extern function. + * + * Multiple invocations of the same extern function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the extern + * function must be invoked again with exactly the same mailbox argument. + * + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_func_t)(void *mailbox); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index cb2e32b83..2335831bf 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -90,6 +90,70 @@ struct port_out_runtime { void *obj; }; +/* + * Extern object. + */ +struct extern_type_member_func { + TAILQ_ENTRY(extern_type_member_func) node; + char name[RTE_SWX_NAME_SIZE]; + rte_swx_extern_type_member_func_t func; + uint32_t id; +}; + +TAILQ_HEAD(extern_type_member_func_tailq, extern_type_member_func); + +struct extern_type { + TAILQ_ENTRY(extern_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_type_constructor_t constructor; + rte_swx_extern_type_destructor_t destructor; + struct extern_type_member_func_tailq funcs; + uint32_t n_funcs; +}; + +TAILQ_HEAD(extern_type_tailq, extern_type); + +struct extern_obj { + TAILQ_ENTRY(extern_obj) node; + char name[RTE_SWX_NAME_SIZE]; + struct extern_type *type; + void *obj; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_obj_tailq, extern_obj); + +#ifndef RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX +#define RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX 8 +#endif + +struct extern_obj_runtime { + void *obj; + uint8_t *mailbox; + rte_swx_extern_type_member_func_t funcs[RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX]; +}; + +/* + * Extern function. + */ +struct extern_func { + TAILQ_ENTRY(extern_func) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_func_t func; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_func_tailq, extern_func); + +struct extern_func_runtime { + uint8_t *mailbox; + rte_swx_extern_func_t func; +}; + /* * Header. */ @@ -130,6 +194,10 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + + /* Extern objects and functions. */ + struct extern_obj_runtime *extern_objs; + struct extern_func_runtime *extern_funcs; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -142,6 +210,9 @@ struct rte_swx_pipeline { struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct extern_type_tailq extern_types; + struct extern_obj_tailq extern_objs; + struct extern_func_tailq extern_funcs; struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; @@ -153,6 +224,8 @@ struct rte_swx_pipeline { uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_extern_objs; + uint32_t n_extern_funcs; uint32_t n_headers; int build_done; int numa_node; @@ -606,6 +679,395 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Extern object. + */ +static struct extern_type * +extern_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_type *elem; + + TAILQ_FOREACH(elem, &p->extern_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_type_member_func * +extern_type_member_func_find(struct extern_type *type, const char *name) +{ + struct extern_type_member_func *elem; + + TAILQ_FOREACH(elem, &type->funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_obj * +extern_obj_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_obj *elem; + + TAILQ_FOREACH(elem, &p->extern_objs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor) +{ + struct extern_type *elem; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_type_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(constructor, EINVAL); + CHECK(destructor, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct extern_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->mailbox_struct_type = mailbox_struct_type; + elem->constructor = constructor; + elem->destructor = destructor; + TAILQ_INIT(&elem->funcs); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_types, elem, node); + + return 0; +} + +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func) +{ + struct extern_type *type; + struct extern_type_member_func *type_member; + + CHECK(p, EINVAL); + + CHECK(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + CHECK(type->n_funcs < RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX, ENOSPC); + + CHECK(name, EINVAL); + CHECK(!extern_type_member_func_find(type, name), EEXIST); + + CHECK(member_func, EINVAL); + + /* Node allocation. */ + type_member = calloc(1, sizeof(struct extern_type_member_func)); + CHECK(type_member, ENOMEM); + + /* Node initialization. */ + strcpy(type_member->name, name); + type_member->func = member_func; + type_member->id = type->n_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&type->funcs, type_member, node); + type->n_funcs++; + + return 0; +} + +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args) +{ + struct extern_type *type; + struct extern_obj *obj; + void *obj_handle; + + CHECK(p, EINVAL); + + CHECK_NAME(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_obj_find(p, name), EEXIST); + + /* Node allocation. */ + obj = calloc(1, sizeof(struct extern_obj)); + CHECK(obj, ENOMEM); + + /* Object construction. */ + obj_handle = type->constructor(args); + if (!obj_handle) { + free(obj); + CHECK(0, ENODEV); + } + + /* Node initialization. */ + strcpy(obj->name, name); + obj->type = type; + obj->obj = obj_handle; + obj->struct_id = p->n_structs; + obj->id = p->n_extern_objs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_objs, obj, node); + p->n_extern_objs++; + p->n_structs++; + + return 0; +} + +static int +extern_obj_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_obj *obj; + + t->extern_objs = calloc(p->n_extern_objs, + sizeof(struct extern_obj_runtime)); + CHECK(t->extern_objs, ENOMEM); + + TAILQ_FOREACH(obj, &p->extern_objs, node) { + struct extern_obj_runtime *r = + &t->extern_objs[obj->id]; + struct extern_type_member_func *func; + uint32_t mailbox_size = + obj->type->mailbox_struct_type->n_bits / 8; + + r->obj = obj->obj; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + TAILQ_FOREACH(func, &obj->type->funcs, node) + r->funcs[func->id] = func->func; + + t->structs[obj->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_obj_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_objs) + continue; + + for (j = 0; j < p->n_extern_objs; j++) { + struct extern_obj_runtime *r = &t->extern_objs[j]; + + free(r->mailbox); + } + + free(t->extern_objs); + t->extern_objs = NULL; + } +} + +static void +extern_obj_free(struct rte_swx_pipeline *p) +{ + extern_obj_build_free(p); + + /* Extern objects. */ + for ( ; ; ) { + struct extern_obj *elem; + + elem = TAILQ_FIRST(&p->extern_objs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_objs, elem, node); + if (elem->obj) + elem->type->destructor(elem->obj); + free(elem); + } + + /* Extern types. */ + for ( ; ; ) { + struct extern_type *elem; + + elem = TAILQ_FIRST(&p->extern_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_types, elem, node); + + for ( ; ; ) { + struct extern_type_member_func *func; + + func = TAILQ_FIRST(&elem->funcs); + if (!func) + break; + + TAILQ_REMOVE(&elem->funcs, func, node); + free(func); + } + + free(elem); + } +} + +/* + * Extern function. + */ +static struct extern_func * +extern_func_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_func *elem; + + TAILQ_FOREACH(elem, &p->extern_funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func) +{ + struct extern_func *f; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_func_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(func, EINVAL); + + /* Node allocation. */ + f = calloc(1, sizeof(struct extern_func)); + CHECK(func, ENOMEM); + + /* Node initialization. */ + strcpy(f->name, name); + f->mailbox_struct_type = mailbox_struct_type; + f->func = func; + f->struct_id = p->n_structs; + f->id = p->n_extern_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_funcs, f, node); + p->n_extern_funcs++; + p->n_structs++; + + return 0; +} + +static int +extern_func_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_func *func; + + /* Memory allocation. */ + t->extern_funcs = calloc(p->n_extern_funcs, + sizeof(struct extern_func_runtime)); + CHECK(t->extern_funcs, ENOMEM); + + /* Extern function. */ + TAILQ_FOREACH(func, &p->extern_funcs, node) { + struct extern_func_runtime *r = + &t->extern_funcs[func->id]; + uint32_t mailbox_size = + func->mailbox_struct_type->n_bits / 8; + + r->func = func->func; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + t->structs[func->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_func_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_funcs) + continue; + + for (j = 0; j < p->n_extern_funcs; j++) { + struct extern_func_runtime *r = &t->extern_funcs[j]; + + free(r->mailbox); + } + + free(t->extern_funcs); + t->extern_funcs = NULL; + } +} + +static void +extern_func_free(struct rte_swx_pipeline *p) +{ + extern_func_build_free(p); + + for ( ; ; ) { + struct extern_func *elem; + + elem = TAILQ_FIRST(&p->extern_funcs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_funcs, elem, node); + free(elem); + } +} + /* * Header. */ @@ -826,6 +1288,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->extern_types); + TAILQ_INIT(&pipeline->extern_objs); + TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ @@ -843,6 +1308,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) metadata_free(p); header_free(p); + extern_func_free(p); + extern_obj_free(p); port_out_free(p); port_in_free(p); struct_free(p); @@ -870,6 +1337,14 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = extern_obj_build(p); + if (status) + goto error; + + status = extern_func_build(p); + if (status) + goto error; + status = header_build(p); if (status) goto error; @@ -884,6 +1359,8 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) error: metadata_build_free(p); header_build_free(p); + extern_func_build_free(p); + extern_obj_build_free(p); port_out_build_free(p); port_in_build_free(p); struct_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 4a7b679a4..2e8a6cdf8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include #include "rte_swx_port.h" +#include "rte_swx_extern.h" /** Name size. */ #ifndef RTE_SWX_NAME_SIZE @@ -147,6 +148,118 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Extern objects and functions + */ + +/** + * Pipeline extern type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern type name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * the extern objects that are instances of this type. Each extern object gets + * its own mailbox, which is used to pass the input arguments to the member + * functions and retrieve the output results. + * @param[in] constructor + * Function used to create the extern objects that are instances of this type. + * @param[in] destructor + * Function used to free the extern objects that are instances of this type. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor); + +/** + * Pipeline extern type member function register + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new member function to be added to the extern type. + * @param[in] member_func + * The new member function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Member function with this name already exists for this type; + * -ENOSPC: Maximum number of member functions reached for this type. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func); + +/** + * Pipeline extern object configure + * + * Instantiate a given extern type to create new extern object. + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new object instantiating the extern type. + * @param[in] args + * Extern object constructor arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern object with this name already exists; + * -ENODEV: Extern object constructor error. + */ +__rte_experimental +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args); + +/** + * Pipeline extern function register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern function name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * this extern function. The mailbox is used to pass the input arguments to + * the extern function and retrieve the output results. + * @param[in] func + * The extern function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern function with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func); + /* * Packet headers and meta-data */ From patchwork Wed Aug 26 15:14:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76001 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 06F3CA04B4; Wed, 26 Aug 2020 17:16:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E4E301C0BC; Wed, 26 Aug 2020 17:15:13 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id C3CFBAAB7 for ; Wed, 26 Aug 2020 17:15:04 +0200 (CEST) IronPort-SDR: IhSwC5Xafv++HEi1msY92e/jkGjAP90s/9RI3wbWGtldHorW6hhKnXosI+rH477dWBuSH6PNwz WK+tB0xO1QTA== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879502" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879502" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:53 -0700 IronPort-SDR: gWnBeM8MfRPdxynEPt+9CSRH03NXXTWzda5p3lGidewWf0AQUIxM/SFk8vVT69Rff9XzdjCbvG /oWhZNi7Rn9Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081275" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:53 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:11 +0100 Message-Id: <20200826151445.51500-7-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 06/40] pipeline: add action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add actions that are dynamically-defined through instructions as opposed to pre-defined. The actions are subroutines of the pipeline program that triggered by table lookup. The input arguments are the action data from the table entry (format defined by struct), the headers and meta-data are in/out. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 147 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 32 ++++ 3 files changed, 180 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 4297e185d..c701f158d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; + rte_swx_pipeline_action_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2335831bf..678700050 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -177,6 +177,26 @@ struct header_out_runtime { uint32_t n_bytes; }; +/* + * Instruction. + */ +struct instruction { +}; + +/* + * Action. + */ +struct action { + TAILQ_ENTRY(action) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + struct instruction *instructions; + uint32_t n_instructions; + uint32_t id; +}; + +TAILQ_HEAD(action_tailq, action); + /* * Pipeline. */ @@ -216,9 +236,11 @@ struct rte_swx_pipeline { struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; + struct action_tailq actions; struct port_in_runtime *in; struct port_out_runtime *out; + struct instruction **action_instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -226,6 +248,7 @@ struct rte_swx_pipeline { uint32_t n_ports_out; uint32_t n_extern_objs; uint32_t n_extern_funcs; + uint32_t n_actions; uint32_t n_headers; int build_done; int numa_node; @@ -1267,6 +1290,123 @@ metadata_free(struct rte_swx_pipeline *p) metadata_build_free(p); } +/* + * Instruction. + */ +static int +instruction_config(struct rte_swx_pipeline *p __rte_unused, + struct action *a __rte_unused, + const char **instructions __rte_unused, + uint32_t n_instructions __rte_unused) +{ + return 0; +} + +/* + * Action. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name) +{ + struct action *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->actions, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions) +{ + struct struct_type *args_struct_type; + struct action *a; + int err; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!action_find(p, name), EEXIST); + + if (args_struct_type_name) { + CHECK_NAME(args_struct_type_name, EINVAL); + args_struct_type = struct_type_find(p, args_struct_type_name); + CHECK(args_struct_type, EINVAL); + } else { + args_struct_type = NULL; + } + + /* Node allocation. */ + a = calloc(1, sizeof(struct action)); + CHECK(a, ENOMEM); + + /* Node initialization. */ + strcpy(a->name, name); + a->st = args_struct_type; + a->id = p->n_actions; + + /* Instruction translation. */ + err = instruction_config(p, a, instructions, n_instructions); + if (err) { + free(a); + return err; + } + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->actions, a, node); + p->n_actions++; + + return 0; +} + +static int +action_build(struct rte_swx_pipeline *p) +{ + struct action *action; + + p->action_instructions = calloc(p->n_actions, + sizeof(struct instruction *)); + CHECK(p->action_instructions, ENOMEM); + + TAILQ_FOREACH(action, &p->actions, node) + p->action_instructions[action->id] = action->instructions; + + return 0; +} + +static void +action_build_free(struct rte_swx_pipeline *p) +{ + free(p->action_instructions); + p->action_instructions = NULL; +} + +static void +action_free(struct rte_swx_pipeline *p) +{ + action_build_free(p); + + for ( ; ; ) { + struct action *action; + + action = TAILQ_FIRST(&p->actions); + if (!action) + break; + + TAILQ_REMOVE(&p->actions, action, node); + free(action->instructions); + free(action); + } +} + /* * Pipeline. */ @@ -1292,6 +1432,7 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_objs); TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); + TAILQ_INIT(&pipeline->actions); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1306,6 +1447,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + action_free(p); metadata_free(p); header_free(p); extern_func_free(p); @@ -1353,10 +1495,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = action_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + action_build_free(p); metadata_build_free(p); header_build_free(p); extern_func_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2e8a6cdf8..1b20293cb 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -344,6 +344,38 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Pipeline action + */ + +/** + * Pipeline action configure + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Action name. + * @param[in] args_struct_type_name + * The struct type instantiated by the action data. The action data represent + * the action arguments that are stored in the table entry together with the + * action ID. Set to NULL when the action does not have any arguments. + * @param[in] instructions + * Action instructions. + * @param[in] n_instructions + * Number of action instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Action with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions); /** * Pipeline build From patchwork Wed Aug 26 15:14:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76005 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2FA1AA04B1; Wed, 26 Aug 2020 17:17:25 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 560111C112; Wed, 26 Aug 2020 17:15:21 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 9820BAAB7 for ; Wed, 26 Aug 2020 17:15:05 +0200 (CEST) IronPort-SDR: hOfJTvTTkvcXeP7CpFgBEDtiPa3+iqrZZFDCWpidU/L/ujVxtGChu4RA9huhrSXZkLp4jpSueU HJZcgXpbuE9g== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879509" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879509" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:55 -0700 IronPort-SDR: u0sBBG/aaxCSrveVCRQvsQKugyejvNMuDv68Nfy3XVi6RlZNzb1JlY05e1bUHVWtZJjqAY5MI3 lxUhkfz9U2zw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081278" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:54 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:12 +0100 Message-Id: <20200826151445.51500-8-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 07/40] pipeline: add tables X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add tables to the pipeline. The match fields are flexibly selected from the headers and meta-data. The set of actions is flexibly selected per table from the pipeline set. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/Makefile | 1 + lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_ctl.h | 85 +++ lib/librte_pipeline/rte_swx_pipeline.c | 700 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 118 ++++ lib/librte_table/Makefile | 1 + lib/librte_table/meson.build | 3 +- lib/librte_table/rte_swx_table.h | 295 ++++++++ 9 files changed, 1208 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_table/rte_swx_table.h diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index 23bfd88e6..d214b1aeb 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -27,5 +27,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_extern.h SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_pipeline.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_ctl.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index bea406848..d5f4d16e5 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -9,5 +9,6 @@ headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', 'rte_swx_pipeline.h', - 'rte_swx_extern.h',) + 'rte_swx_extern.h', + 'rte_swx_ctl.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index c701f158d..b9e59bce2 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -68,6 +68,10 @@ EXPERIMENTAL { rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_action_config; + rte_swx_pipeline_table_type_register; + rte_swx_pipeline_table_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h new file mode 100644 index 000000000..c824ab56f --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_CTL_H__ +#define __INCLUDE_RTE_SWX_CTL_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline Control + */ + +#include +#include + +#include + +#include "rte_swx_table.h" + +/* + * Table Update API. + */ + +/** Table state. */ +struct rte_swx_table_state { + /** Table object. */ + void *obj; + + /** Action ID of the table default action. */ + uint64_t default_action_id; + + /** Action data of the table default action. Ignored when the action + * data size is zero; otherwise, action data size bytes are meaningful. + */ + uint8_t *default_action_data; +}; + +/** + * Pipeline table state get + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the *table_state* contains the pointer to the + * current pipeline table state, which is an array of *n_tables* elements, + * with array element i containing the state of the i-th pipeline table. The + * pipeline continues to own all the data structures directly or indirectly + * referenced by the *table_state* until the subsequent successful invocation + * of function *rte_swx_pipeline_table_state_set*. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state); + +/** + * Pipeline table state set + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the pipeline table state is updated to this + * *table_state*. The ownership of all the data structures directly or + * indirectly referenced by this *table_state* is passed from the caller to + * the pipeline. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 678700050..43cdb0f7c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -10,6 +10,7 @@ #include #include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" #define CHECK(condition, err_code) \ do { \ @@ -197,6 +198,55 @@ struct action { TAILQ_HEAD(action_tailq, action); +/* + * Table. + */ +struct table_type { + TAILQ_ENTRY(table_type) node; + char name[RTE_SWX_NAME_SIZE]; + enum rte_swx_table_match_type match_type; + struct rte_swx_table_ops ops; +}; + +TAILQ_HEAD(table_type_tailq, table_type); + +struct match_field { + enum rte_swx_table_match_type match_type; + struct field *field; +}; + +struct table { + TAILQ_ENTRY(table) node; + char name[RTE_SWX_NAME_SIZE]; + char args[RTE_SWX_NAME_SIZE]; + struct table_type *type; /* NULL when n_fields == 0. */ + + /* Match. */ + struct match_field *fields; + uint32_t n_fields; + int is_header; /* Only valid when n_fields > 0. */ + struct header *header; /* Only valid when n_fields > 0. */ + + /* Action. */ + struct action **actions; + struct action *default_action; + uint8_t *default_action_data; + uint32_t n_actions; + int default_action_is_const; + uint32_t action_data_size_max; + + uint32_t size; + uint32_t id; +}; + +TAILQ_HEAD(table_tailq, table); + +struct table_runtime { + rte_swx_table_lookup_t func; + void *mailbox; + uint8_t **key; +}; + /* * Pipeline. */ @@ -215,6 +265,12 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + /* Tables. */ + struct table_runtime *tables; + struct rte_swx_table_state *table_state; + uint64_t action_id; + int hit; /* 0 = Miss, 1 = Hit. */ + /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; @@ -237,10 +293,13 @@ struct rte_swx_pipeline { struct struct_type *metadata_st; uint32_t metadata_struct_id; struct action_tailq actions; + struct table_type_tailq table_types; + struct table_tailq tables; struct port_in_runtime *in; struct port_out_runtime *out; struct instruction **action_instructions; + struct rte_swx_table_state *table_state; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -249,6 +308,7 @@ struct rte_swx_pipeline { uint32_t n_extern_objs; uint32_t n_extern_funcs; uint32_t n_actions; + uint32_t n_tables; uint32_t n_headers; int build_done; int numa_node; @@ -269,6 +329,21 @@ struct_type_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +struct_type_field_find(struct struct_type *st, const char *name) +{ + uint32_t i; + + for (i = 0; i < st->n_fields; i++) { + struct field *f = &st->fields[i]; + + if (strcmp(f->name, name) == 0) + return f; + } + + return NULL; +} + int rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1106,6 +1181,50 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +header_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct header **header) +{ + struct header *h; + struct field *f; + char *header_name, *field_name; + + if ((name[0] != 'h') || (name[1] != '.')) + return NULL; + + header_name = strdup(&name[2]); + if (!header_name) + return NULL; + + field_name = strchr(header_name, '.'); + if (!field_name) { + free(header_name); + return NULL; + } + + *field_name = 0; + field_name++; + + h = header_find(p, header_name); + if (!h) { + free(header_name); + return NULL; + } + + f = struct_type_field_find(h->st, field_name); + if (!f) { + free(header_name); + return NULL; + } + + if (header) + *header = h; + + free(header_name); + return f; +} + int rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, const char *name, @@ -1229,6 +1348,18 @@ header_free(struct rte_swx_pipeline *p) /* * Meta-data. */ +static struct field * +metadata_field_parse(struct rte_swx_pipeline *p, const char *name) +{ + if (!p->metadata_st) + return NULL; + + if (name[0] != 'm' || name[1] != '.') + return NULL; + + return struct_type_field_find(p->metadata_st, &name[2]); +} + int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name) @@ -1407,6 +1538,536 @@ action_free(struct rte_swx_pipeline *p) } } +/* + * Table. + */ +static struct table_type * +table_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table_type *elem; + + TAILQ_FOREACH(elem, &p->table_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table_type * +table_type_resolve(struct rte_swx_pipeline *p, + const char *recommended_type_name, + enum rte_swx_table_match_type match_type) +{ + struct table_type *elem; + + /* Only consider the recommended type if the match type is correct. */ + if (recommended_type_name) + TAILQ_FOREACH(elem, &p->table_types, node) + if (!strcmp(elem->name, recommended_type_name) && + (elem->match_type == match_type)) + return elem; + + /* Ignore the recommended type and get the first element with this match + * type. + */ + TAILQ_FOREACH(elem, &p->table_types, node) + if (elem->match_type == match_type) + return elem; + + return NULL; +} + +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table *elem; + + TAILQ_FOREACH(elem, &p->tables, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table * +table_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct table *table = NULL; + + TAILQ_FOREACH(table, &p->tables, node) + if (table->id == id) + return table; + + return NULL; +} + +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops) +{ + struct table_type *elem; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_type_find(p, name), EEXIST); + + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->lkp, EINVAL); + CHECK(ops->free, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct table_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->match_type = match_type; + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->table_types, elem, node); + + return 0; +} + +static enum rte_swx_table_match_type +table_match_type_resolve(struct rte_swx_match_field_params *fields, + uint32_t n_fields) +{ + uint32_t i; + + for (i = 0; i < n_fields; i++) + if (fields[i].match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + + if (i == n_fields) + return RTE_SWX_TABLE_MATCH_EXACT; + + if ((i == n_fields - 1) && + (fields[i].match_type == RTE_SWX_TABLE_MATCH_LPM)) + return RTE_SWX_TABLE_MATCH_LPM; + + return RTE_SWX_TABLE_MATCH_WILDCARD; +} + +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size) +{ + struct table_type *type; + struct table *t; + struct action *default_action; + struct header *header = NULL; + int is_header = 0; + uint32_t offset_prev = 0, action_data_size_max = 0, i; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_find(p, name), EEXIST); + + CHECK(params, EINVAL); + + /* Match checks. */ + CHECK(!params->n_fields || params->fields, EINVAL); + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct header *h; + struct field *hf, *mf; + uint32_t offset; + + CHECK_NAME(field->name, EINVAL); + + hf = header_field_parse(p, field->name, &h); + mf = metadata_field_parse(p, field->name); + offset = hf ? hf->offset : mf->offset; + + CHECK(hf || mf, EINVAL); + + if (i == 0) { + is_header = hf ? 1 : 0; + header = hf ? h : NULL; + offset_prev = offset; + + continue; + } + + CHECK((is_header && hf && (h->id == header->id)) || + (!is_header && mf), EINVAL); + + CHECK(offset > offset_prev, EINVAL); + offset_prev = offset; + } + + /* Action checks. */ + CHECK(params->n_actions, EINVAL); + CHECK(params->action_names, EINVAL); + for (i = 0; i < params->n_actions; i++) { + const char *action_name = params->action_names[i]; + struct action *a; + uint32_t action_data_size; + + CHECK(action_name, EINVAL); + + a = action_find(p, action_name); + CHECK(a, EINVAL); + + action_data_size = a->st ? a->st->n_bits / 8 : 0; + if (action_data_size > action_data_size_max) + action_data_size_max = action_data_size; + } + + CHECK(params->default_action_name, EINVAL); + for (i = 0; i < p->n_actions; i++) + if (!strcmp(params->action_names[i], + params->default_action_name)) + break; + CHECK(i < params->n_actions, EINVAL); + default_action = action_find(p, params->default_action_name); + CHECK((default_action->st && params->default_action_data) || + !params->default_action_data, EINVAL); + + /* Table type checks. */ + if (params->n_fields) { + enum rte_swx_table_match_type match_type; + + match_type = table_match_type_resolve(params->fields, + params->n_fields); + type = table_type_resolve(p, + recommended_table_type_name, + match_type); + CHECK(type, EINVAL); + } else { + type = NULL; + } + + /* Memory allocation. */ + t = calloc(1, sizeof(struct table)); + CHECK(t, ENOMEM); + + t->fields = calloc(params->n_fields, sizeof(struct match_field)); + if (!t->fields) { + free(t); + CHECK(0, ENOMEM); + } + + t->actions = calloc(params->n_actions, sizeof(struct action *)); + if (!t->actions) { + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + + if (action_data_size_max) { + t->default_action_data = calloc(1, action_data_size_max); + if (!t->default_action_data) { + free(t->actions); + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + } + + /* Node initialization. */ + strcpy(t->name, name); + if (args && args[0]) + strcpy(t->args, args); + t->type = type; + + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct match_field *f = &t->fields[i]; + + f->match_type = field->match_type; + f->field = is_header ? + header_field_parse(p, field->name, NULL) : + metadata_field_parse(p, field->name); + } + t->n_fields = params->n_fields; + t->is_header = is_header; + t->header = header; + + for (i = 0; i < params->n_actions; i++) + t->actions[i] = action_find(p, params->action_names[i]); + t->default_action = default_action; + if (default_action->st) + memcpy(t->default_action_data, + params->default_action_data, + default_action->st->n_bits / 8); + t->n_actions = params->n_actions; + t->default_action_is_const = params->default_action_is_const; + t->action_data_size_max = action_data_size_max; + + t->size = size; + t->id = p->n_tables; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->tables, t, node); + p->n_tables++; + + return 0; +} + +static struct rte_swx_table_params * +table_params_get(struct table *table) +{ + struct rte_swx_table_params *params; + struct field *first, *last; + uint8_t *key_mask; + uint32_t key_size, key_offset, action_data_size, i; + + /* Memory allocation. */ + params = calloc(1, sizeof(struct rte_swx_table_params)); + if (!params) + return NULL; + + /* Key offset and size. */ + first = table->fields[0].field; + last = table->fields[table->n_fields - 1].field; + key_offset = first->offset / 8; + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* Memory allocation. */ + key_mask = calloc(1, key_size); + if (!key_mask) { + free(params); + return NULL; + } + + /* Key mask. */ + for (i = 0; i < table->n_fields; i++) { + struct field *f = table->fields[i].field; + uint32_t start = (f->offset - first->offset) / 8; + size_t size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + + /* Action data size. */ + action_data_size = 0; + for (i = 0; i < table->n_actions; i++) { + struct action *action = table->actions[i]; + uint32_t ads = action->st ? action->st->n_bits / 8 : 0; + + if (ads > action_data_size) + action_data_size = ads; + } + + /* Fill in. */ + params->match_type = table->type->match_type; + params->key_size = key_size; + params->key_offset = key_offset; + params->key_mask0 = key_mask; + params->action_data_size = action_data_size; + params->n_keys_max = table->size; + + return params; +} + +static void +table_params_free(struct rte_swx_table_params *params) +{ + if (!params) + return; + + free(params->key_mask0); + free(params); +} + +static int +table_state_build(struct rte_swx_pipeline *p) +{ + struct table *table; + + p->table_state = calloc(p->n_tables, + sizeof(struct rte_swx_table_state)); + CHECK(p->table_state, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct rte_swx_table_state *ts = &p->table_state[table->id]; + + if (table->type) { + struct rte_swx_table_params *params; + + /* ts->obj. */ + params = table_params_get(table); + CHECK(params, ENOMEM); + + ts->obj = table->type->ops.create(params, + NULL, + table->args, + p->numa_node); + + table_params_free(params); + CHECK(ts->obj, ENODEV); + } + + /* ts->default_action_data. */ + if (table->action_data_size_max) { + ts->default_action_data = + malloc(table->action_data_size_max); + CHECK(ts->default_action_data, ENOMEM); + + memcpy(ts->default_action_data, + table->default_action_data, + table->action_data_size_max); + } + + /* ts->default_action_id. */ + ts->default_action_id = table->default_action->id; + } + + return 0; +} + +static void +table_state_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + if (!p->table_state) + return; + + for (i = 0; i < p->n_tables; i++) { + struct rte_swx_table_state *ts = &p->table_state[i]; + struct table *table = table_find_by_id(p, i); + + /* ts->obj. */ + if (table->type && ts->obj) + table->type->ops.free(ts->obj); + + /* ts->default_action_data. */ + free(ts->default_action_data); + } + + free(p->table_state); + p->table_state = NULL; +} + +static void +table_state_free(struct rte_swx_pipeline *p) +{ + table_state_build_free(p); +} + +static int +table_stub_lkp(void *table __rte_unused, + void *mailbox __rte_unused, + uint8_t **key __rte_unused, + uint64_t *action_id __rte_unused, + uint8_t **action_data __rte_unused, + int *hit) +{ + *hit = 0; + return 1; /* DONE. */ +} + +static int +table_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct table *table; + + t->tables = calloc(p->n_tables, sizeof(struct table_runtime)); + CHECK(t->tables, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct table_runtime *r = &t->tables[table->id]; + + if (table->type) { + uint64_t size; + + size = table->type->ops.mailbox_size_get(); + + /* r->func. */ + r->func = table->type->ops.lkp; + + /* r->mailbox. */ + if (size) { + r->mailbox = calloc(1, size); + CHECK(r->mailbox, ENOMEM); + } + + /* r->key. */ + r->key = table->is_header ? + &t->structs[table->header->struct_id] : + &t->structs[p->metadata_struct_id]; + } else { + r->func = table_stub_lkp; + } + } + } + + return 0; +} + +static void +table_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->tables) + continue; + + for (j = 0; j < p->n_tables; j++) { + struct table_runtime *r = &t->tables[j]; + + free(r->mailbox); + } + + free(t->tables); + t->tables = NULL; + } +} + +static void +table_free(struct rte_swx_pipeline *p) +{ + table_build_free(p); + + /* Tables. */ + for ( ; ; ) { + struct table *elem; + + elem = TAILQ_FIRST(&p->tables); + if (!elem) + break; + + TAILQ_REMOVE(&p->tables, elem, node); + free(elem->fields); + free(elem->actions); + free(elem->default_action_data); + free(elem); + } + + /* Table types. */ + for ( ; ; ) { + struct table_type *elem; + + elem = TAILQ_FIRST(&p->table_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->table_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -1433,6 +2094,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); TAILQ_INIT(&pipeline->actions); + TAILQ_INIT(&pipeline->table_types); + TAILQ_INIT(&pipeline->tables); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1447,6 +2110,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + table_state_free(p); + table_free(p); action_free(p); metadata_free(p); header_free(p); @@ -1499,10 +2164,20 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = table_build(p); + if (status) + goto error; + + status = table_state_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + table_state_build_free(p); + table_build_free(p); action_build_free(p); metadata_build_free(p); header_build_free(p); @@ -1514,3 +2189,28 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } + +/* + * Control. + */ +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + *table_state = p->table_state; + return 0; +} + +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + p->table_state = table_state; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 1b20293cb..d7e3ba1ec 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include #include "rte_swx_port.h" +#include "rte_swx_table.h" #include "rte_swx_extern.h" /** Name size. */ @@ -377,6 +378,123 @@ rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char **instructions, uint32_t n_instructions); +/* + * Pipeline table + */ + +/** + * Pipeline table type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Table type name. + * @param[in] match type + * Match type implemented by the new table type. + * @param[in] ops + * Table type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops); + +/** Match field parameters. */ +struct rte_swx_match_field_params { + /** Match field name. Must be either a field of one of the registered + * packet headers ("h.header.field") or a field of the registered + * meta-data ("m.field"). + */ + const char *name; + + /** Match type of the field. */ + enum rte_swx_table_match_type match_type; +}; + +/** Pipeline table parameters. */ +struct rte_swx_pipeline_table_params { + /** The set of match fields for the current table. + * Restriction: All the match fields of the current table need to be + * part of the same struct, i.e. either all the match fields are part of + * the same header or all the match fields are part of the meta-data. + */ + struct rte_swx_match_field_params *fields; + + /** The number of match fields for the current table. If set to zero, no + * "regular" entries (i.e. entries other than the default entry) can be + * added to the current table and the match process always results in + * lookup miss. + */ + uint32_t n_fields; + + /** The set of actions for the current table. */ + const char **action_names; + + /** The number of actions for the current table. Must be at least one. + */ + uint32_t n_actions; + + /** The default table action that gets executed on lookup miss. Must be + * one of the table actions included in the *action_names*. + */ + const char *default_action_name; + + /** Default action data. The size of this array is the action data size + * of the default action. Must be NULL if the default action data size + * is zero. + */ + uint8_t *default_action_data; + + /** If non-zero (true), then the default action of the current table + * cannot be changed. If zero (false), then the default action can be + * changed in the future with another action from the *action_names* + * list. + */ + int default_action_is_const; +}; + +/** + * Pipeline table configure + * + * @param[out] p + * Pipeline handle. + * @param[in] name + * Table name. + * @param[in] params + * Table parameters. + * @param[in] recommended_table_type_name + * Recommended table type. Typically set to NULL. Useful as guidance when + * there are multiple table types registered for the match type of the table, + * as determined from the table match fields specification. Silently ignored + * if the recommended table type does not exist or it serves a different match + * type. + * @param[in] args + * Table creation arguments. + * @param[in] size + * Guideline on maximum number of table entries. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table with this name already exists; + * -ENODEV: Table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size); + /** * Pipeline build * diff --git a/lib/librte_table/Makefile b/lib/librte_table/Makefile index 6ad8a6b17..9df58698d 100644 --- a/lib/librte_table/Makefile +++ b/lib/librte_table/Makefile @@ -55,5 +55,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_lru_arm64.h endif SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_array.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_stub.h +SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_swx_table.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index 71d134768..b9d4fe3dc 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -22,7 +22,8 @@ headers = files('rte_table.h', 'rte_table_hash_func_arm64.h', 'rte_lru.h', 'rte_table_array.h', - 'rte_table_stub.h') + 'rte_table_stub.h', + 'rte_swx_table.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table.h b/lib/librte_table/rte_swx_table.h new file mode 100644 index 000000000..c5c202723 --- /dev/null +++ b/lib/librte_table/rte_swx_table.h @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_H__ +#define __INCLUDE_RTE_SWX_TABLE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Table + * + * Table interface. + */ + +#include +#include + +/** Match type. */ +enum rte_swx_table_match_type { + /** Wildcard Match (WM). */ + RTE_SWX_TABLE_MATCH_WILDCARD, + + /** Longest Prefix Match (LPM). */ + RTE_SWX_TABLE_MATCH_LPM, + + /** Exact Match (EM). */ + RTE_SWX_TABLE_MATCH_EXACT, +}; + +/** Table creation parameters. */ +struct rte_swx_table_params { + /** Table match type. */ + enum rte_swx_table_match_type match_type; + + /** Key size in bytes. */ + uint32_t key_size; + + /** Offset of the first byte of the key within the key buffer. */ + uint32_t key_offset; + + /** Mask of *key_size* bytes logically laid over the bytes at positions + * *key_offset* .. (*key_offset* + *key_size* - 1) of the key buffer in + * order to specify which bits from the key buffer are part of the key + * and which ones are not. A bit value of 1 in the *key_mask0* means the + * respective bit in the key buffer is part of the key, while a bit + * value of 0 means the opposite. A NULL value means that all the bits + * are part of the key, i.e. the *key_mask0* is an all-ones mask. + */ + uint8_t *key_mask0; + + /** Maximum size (in bytes) of the action data. The data stored in the + * table for each entry is equal to *action_data_size* plus 8 bytes, + * which are used to store the action ID. + */ + uint32_t action_data_size; + + /** Maximum number of keys to be stored in the table together with their + * associated data. + */ + uint32_t n_keys_max; +}; + +/** Table entry. */ +struct rte_swx_table_entry { + /** Used to faciliate the addition of the current table entry to a + * linked list. + */ + TAILQ_ENTRY(rte_swx_table_entry) node; + + /** Key value for the current entry. Array of *key_size* bytes or NULL + * if the *key_size* for the current table is 0. + */ + uint8_t *key; + + /** Key mask for the current entry. Array of *key_size* bytes that is + * logically and'ed with *key_mask0* of the current table. A NULL value + * means that all the key bits already enabled by *key_mask0* are part + * of the key of the current entry. + */ + uint8_t *key_mask; + + /** Placeholder for a possible compressed version of the *key* and + * *key_mask* of the current entry. Typically a hash signature, its main + * purpose is to the linked list search operation. Should be ignored by + * the API functions below. + */ + uint64_t key_signature; + + /** Action ID for the current entry. */ + uint64_t action_id; + + /** Action data for the current entry. Its size is defined by the action + * specified by the *action_id*. It must be NULL when the action data + * size of the *action_id* action is NULL. It must never exceed the + * *action_data_size* of the table. + */ + uint8_t *action_data; +}; + +/** List of table entries. */ +TAILQ_HEAD(rte_swx_table_entry_list, rte_swx_table_entry); + +/** + * Table memory footprint get + * + * @param[in] params + * Table create parameters. + * @param[in] entries + * Table entries. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, if successful, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_footprint_get_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args); + +/** + * Table mailbox size get + * + * The mailbox is used to store the context of a lookup operation that is in + * progress and it is passed as a parameter to the lookup operation. This allows + * for multiple concurrent lookup operations into the same table. + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, on success, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_mailbox_size_get_t)(void); + +/** + * Table create + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * Table handle, on success, or NULL, on error. + */ +typedef void * +(*rte_swx_table_create_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node); + +/** + * Table entry add + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_add_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table entry delete + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be deleted from the table. The entry *action_id* and *action_data* + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_delete_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table lookup + * + * The table lookup operation seaches a given key in the table and upon its + * completion it returns an indication of whether the key is found in the table + * (lookup hit) or not (lookup miss). In case of lookup hit, the action_id and + * the action_data associated with the key are also returned. + * + * Multiple invocations of this function may be required in order to complete a + * single table lookup operation for a given table and a given lookup key. The + * completion of the table lookup operation is flagged by a return value of 1; + * in case of a return value of 0, the function must be invoked again with + * exactly the same arguments. + * + * The mailbox argument is used to store the context of an on-going table lookup + * operation. The mailbox mechanism allows for multiple concurrent table lookup + * operations into the same table. + * + * The typical reason an implementation may choose to split the table lookup + * operation into multiple steps is to hide the latency of the inherrent memory + * read operations: before a read operation with the source data likely not in + * the CPU cache, the source data prefetch is issued and the table lookup + * operation is postponed in favor of some other unrelated work, which the CPU + * executes in parallel with the source data being fetched into the CPU cache; + * later on, the table lookup operation is resumed, this time with the source + * data likely to be read from the CPU cache with no CPU pipeline stall, which + * significantly improves the table lookup performance. + * + * @param[in] table + * Table handle. + * @param[in] mailbox + * Mailbox for the current table lookup operation. + * @param[in] key + * Lookup key. Its size mult be equal to the table *key_size*. If the latter + * is zero, then the lookup key must be NULL. + * @param[out] action_id + * ID of the action associated with the *key*. Must point to a valid 64-bit + * variable. Only valid when the function returns 1 and *hit* is set to true. + * @param[out] action_data + * Action data for the *action_id* action. Must point to a valid array of + * table *action_data_size* bytes. Only valid when the function returns 1 and + * *hit* is set to true. + * @param[out] hit + * Only valid when the function returns 1. Set to non-zero (true) on table + * lookup hit and to zero (false) on table lookup miss. + * @return + * 0 when the table lookup operation is not yet completed, and 1 when the + * table lookup operation is completed. No other return values are allowed. + */ +typedef int +(*rte_swx_table_lookup_t)(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit); + +/** + * Table free + * + * @param[in] table + * Table handle. + */ +typedef void +(*rte_swx_table_free_t)(void *table); + +/** Table operations. */ +struct rte_swx_table_ops { + /** Table memory footprint get. Set to NULL when not supported. */ + rte_swx_table_footprint_get_t footprint_get; + + /** Table mailbox size get. When NULL, the mailbox size is 0. */ + rte_swx_table_mailbox_size_get_t mailbox_size_get; + + /** Table create. Must be non-NULL. */ + rte_swx_table_create_t create; + + /** Incremental table entry add. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the new entry included. + */ + rte_swx_table_add_t add; + + /** Incremental table entry delete. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the entry excluded. + */ + rte_swx_table_delete_t del; + + /** Table lookup. Must be non-NULL. */ + rte_swx_table_lookup_t lkp; + + /** Table free. Must be non-NULL. */ + rte_swx_table_free_t free; +}; + +#ifdef __cplusplus +} +#endif + +#endif From patchwork Wed Aug 26 15:14:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76002 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C2BC9A04B4; Wed, 26 Aug 2020 17:16:41 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 42D321C0CE; Wed, 26 Aug 2020 17:15:16 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id E3C071B94F for ; Wed, 26 Aug 2020 17:15:04 +0200 (CEST) IronPort-SDR: UlBZ8Qp1kKIkQMMJyXcsHjrDxV2jwK8xEzjQNFeAs/X5oiCIYMMY8XJ8NpIs1qpOtBbs2ZoCK8 7FKnKNvHAdFA== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879512" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879512" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:56 -0700 IronPort-SDR: g+O07hmwE0h6wpOmnCecjzv3Ci9mjZRE5IoAtVgdZVsZk4xE+JutwDKt2ya3ItkAc0Eicqz3kx wPgW2DaxMPpg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081279" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:55 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:13 +0100 Message-Id: <20200826151445.51500-9-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 08/40] pipeline: add pipeline instructions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The pipeline instructions represent the main program that defines the life of the packet. As packets go through tables that trigger action subroutines, the headers and meta-data get transformed along the way. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 36 ++++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 20 +++++++++++ 3 files changed, 57 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index b9e59bce2..7139df0d3 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -70,6 +70,7 @@ EXPERIMENTAL { rte_swx_pipeline_action_config; rte_swx_pipeline_table_type_register; rte_swx_pipeline_table_config; + rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_table_state_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 43cdb0f7c..464f9faf8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -274,6 +274,10 @@ struct thread { /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; + + /* Instructions. */ + struct instruction *ip; + struct instruction *ret; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -300,6 +304,7 @@ struct rte_swx_pipeline { struct port_out_runtime *out; struct instruction **action_instructions; struct rte_swx_table_state *table_state; + struct instruction *instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -310,6 +315,7 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t n_instructions; int build_done; int numa_node; }; @@ -1424,6 +1430,12 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) +{ + t->ip = p->instructions; +} + static int instruction_config(struct rte_swx_pipeline *p __rte_unused, struct action *a __rte_unused, @@ -2110,6 +2122,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + free(p->instructions); + table_state_free(p); table_free(p); action_free(p); @@ -2124,6 +2138,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) free(p); } +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions) +{ + int err; + uint32_t i; + + err = instruction_config(p, NULL, instructions, n_instructions); + if (err) + return err; + + /* Thread instruction pointer reset. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + thread_ip_reset(p, t); + } + + return 0; +} + int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d7e3ba1ec..47a0f8dcc 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -495,6 +495,26 @@ rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, const char *args, uint32_t size); +/** + * Pipeline instructions configure + * + * @param[in] p + * Pipeline handle. + * @param[in] instructions + * Pipeline instructions. + * @param[in] n_instructions + * Number of pipeline instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions); + /** * Pipeline build * From patchwork Wed Aug 26 15:14:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76003 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 99A55A04B4; Wed, 26 Aug 2020 17:16:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EA34B1C0D9; Wed, 26 Aug 2020 17:15:17 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 272D814583 for ; Wed, 26 Aug 2020 17:15:05 +0200 (CEST) IronPort-SDR: 5WHTCxvaT6BLkdLhh23h0QgQpm3tt6fSRbwBadZ4PI91Kaqy/bPSwnjhFoIeLVgpG6up3k3SjU wmMhYtb2y+Dg== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879514" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879514" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:57 -0700 IronPort-SDR: UfB4pf8zPJRU6Xa1KPfpZFzlgOlXm+X9wAYTLVCM8HqqDORzrjFLmOU7+pXmxEpoSp7rMqjH0w ahePQuNksUcA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081280" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:56 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:14 +0100 Message-Id: <20200826151445.51500-10-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 09/40] pipeline: add rx and extract instructions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add packet reception and header extraction instructions. The RX must be the first pipeline instruction. Each extracted header is logically removed from the packet, then it can be read/written by instructions, emitted into the outgoing packet or discarded. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 564 ++++++++++++++++++- lib/librte_pipeline/rte_swx_pipeline.h | 13 + 3 files changed, 574 insertions(+), 4 deletions(-) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 7139df0d3..793957eb9 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -73,6 +73,7 @@ EXPERIMENTAL { rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 464f9faf8..98772de99 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -8,6 +8,7 @@ #include #include +#include #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -21,6 +22,16 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + /* * Struct. */ @@ -181,7 +192,64 @@ struct header_out_runtime { /* * Instruction. */ + +/* Packet headers are always in Network Byte Order (NBO), i.e. big endian. + * Packet meta-data fields are always assumed to be in Host Byte Order (HBO). + * Table entry fields can be in either NBO or HBO; they are assumed to be in HBO + * when transferred to packet meta-data and in NBO when transferred to packet + * headers. + */ + +/* Notation conventions: + * -Header field: H = h.header.field (dst/src) + * -Meta-data field: M = m.field (dst/src) + * -Extern object mailbox field: E = e.field (dst/src) + * -Extern function mailbox field: F = f.field (dst/src) + * -Table action data field: T = t.field (src only) + * -Immediate value: I = 32-bit unsigned value (src only) + */ + +enum instruction_type { + /* rx m.port_in */ + INSTR_RX, + + /* extract h.header */ + INSTR_HDR_EXTRACT, + INSTR_HDR_EXTRACT2, + INSTR_HDR_EXTRACT3, + INSTR_HDR_EXTRACT4, + INSTR_HDR_EXTRACT5, + INSTR_HDR_EXTRACT6, + INSTR_HDR_EXTRACT7, + INSTR_HDR_EXTRACT8, +}; + +struct instr_io { + struct { + uint8_t offset; + uint8_t n_bits; + uint8_t pad[2]; + } io; + + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + uint8_t n_bytes[8]; + } hdr; +}; + struct instruction { + enum instruction_type type; + union { + struct instr_io io; + }; +}; + +struct instruction_data { + char label[RTE_SWX_NAME_SIZE]; + char jmp_label[RTE_SWX_NAME_SIZE]; + uint32_t n_users; /* user = jmp instruction to this instruction. */ + int invalid; }; /* @@ -251,6 +319,10 @@ struct table_runtime { * Pipeline. */ struct thread { + /* Packet. */ + struct rte_swx_pkt pkt; + uint8_t *ptr; + /* Structures. */ uint8_t **structs; @@ -280,6 +352,29 @@ struct thread { struct instruction *ret; }; +#define MASK64_BIT_GET(mask, pos) ((mask) & (1LLU << (pos))) +#define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) +#define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) + +#define METADATA_READ(thread, offset, n_bits) \ +({ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + (m64 & m64_mask); \ +}) + +#define METADATA_WRITE(thread, offset, n_bits, value) \ +{ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + \ + uint64_t m_new = value; \ + \ + *m64_ptr = (m64 & ~m64_mask) | (m_new & m64_mask); \ +} + #ifndef RTE_SWX_PIPELINE_THREADS_MAX #define RTE_SWX_PIPELINE_THREADS_MAX 16 #endif @@ -315,6 +410,8 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t thread_id; + uint32_t port_id; uint32_t n_instructions; int build_done; int numa_node; @@ -1187,6 +1284,16 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct header * +header_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'h' || name[1] != '.') + return NULL; + + return header_find(p, &name[2]); +} + static struct field * header_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1430,19 +1537,459 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +pipeline_port_inc(struct rte_swx_pipeline *p) +{ + p->port_id = (p->port_id + 1) & (p->n_ports_in - 1); +} + static inline void thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) { t->ip = p->instructions; } +static inline void +thread_ip_inc(struct rte_swx_pipeline *p); + +static inline void +thread_ip_inc(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + t->ip++; +} + +static inline void +thread_ip_inc_cond(struct thread *t, int cond) +{ + t->ip += cond; +} + +static inline void +thread_yield(struct rte_swx_pipeline *p) +{ + p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + +/* + * rx. + */ +static int +instr_rx_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_RX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct port_in_runtime *port = &p->in[p->port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + int pkt_received; + + /* Packet. */ + pkt_received = port->pkt_rx(port->obj, pkt); + t->ptr = &pkt->pkt[pkt->offset]; + rte_prefetch0(t->ptr); + + TRACE("[Thread %2u] rx %s from port %u\n", + p->thread_id, + pkt_received ? "1 pkt" : "0 pkts", + p->port_id); + + /* Headers. */ + t->valid_headers = 0; + t->n_headers_out = 0; + + /* Meta-data. */ + METADATA_WRITE(t, ip->io.io.offset, ip->io.io.n_bits, p->port_id); + + /* Tables. */ + t->table_state = p->table_state; + + /* Thread. */ + pipeline_port_inc(p); + thread_ip_inc_cond(t, pkt_received); + thread_yield(p); +} + +/* + * extract. + */ +static int +instr_hdr_extract_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EXTRACT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract); + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + uint32_t offset = t->pkt.offset; + uint32_t length = t->pkt.length; + uint32_t i; + + for (i = 0; i < n_extract; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + TRACE("[Thread %2u]: extract header %u (%u bytes)\n", + p->thread_id, + header_id, + n_bytes); + + /* Headers. */ + t->structs[struct_id] = ptr; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + + /* Packet. */ + offset += n_bytes; + length -= n_bytes; + ptr += n_bytes; + } + + /* Headers. */ + t->valid_headers = valid_headers; + + /* Packet. */ + t->pkt.offset = offset; + t->pkt.length = length; + t->ptr = ptr; +} + +static inline void +instr_hdr_extract_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_extract_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + CHECK(0, EINVAL); +} + +static uint32_t +label_is_used(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t count = 0, i; + + if (!label[0]) + return 0; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].jmp_label)) + count++; + + return count; +} + static int -instruction_config(struct rte_swx_pipeline *p __rte_unused, - struct action *a __rte_unused, - const char **instructions __rte_unused, - uint32_t n_instructions __rte_unused) +instr_label_check(struct instruction_data *instruction_data, + uint32_t n_instructions) { + uint32_t i; + + /* Check that all instruction labels are unique. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + uint32_t j; + + if (!label[0]) + continue; + + for (j = i + 1; j < n_instructions; j++) + CHECK(strcmp(label, data[j].label), EINVAL); + } + + /* Get users for each instruction label. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + + data->n_users = label_is_used(instruction_data, + n_instructions, + label); + } + + return 0; +} + +static int +instruction_config(struct rte_swx_pipeline *p, + struct action *a, + const char **instructions, + uint32_t n_instructions) +{ + struct instruction *instr = NULL; + struct instruction_data *data = NULL; + char *string = NULL; + int err = 0; + uint32_t i; + + CHECK(n_instructions, EINVAL); + CHECK(instructions, EINVAL); + for (i = 0; i < n_instructions; i++) + CHECK(instructions[i], EINVAL); + + /* Memory allocation. */ + instr = calloc(n_instructions, sizeof(struct instruction)); + if (!instr) { + err = ENOMEM; + goto error; + } + + data = calloc(n_instructions, sizeof(struct instruction_data)); + if (!data) { + err = ENOMEM; + goto error; + } + + for (i = 0; i < n_instructions; i++) { + string = strdup(instructions[i]); + if (!string) { + err = ENOMEM; + goto error; + } + + err = instr_translate(p, a, string, &instr[i], &data[i]); + if (err) + goto error; + + free(string); + } + + err = instr_label_check(data, n_instructions); + if (err) + goto error; + + free(data); + + if (a) { + a->instructions = instr; + a->n_instructions = n_instructions; + } else { + p->instructions = instr; + p->n_instructions = n_instructions; + } + return 0; + +error: + free(string); + free(data); + free(instr); + return err; +} + +typedef void (*instr_exec_t)(struct rte_swx_pipeline *); + +static instr_exec_t instruction_table[] = { + [INSTR_RX] = instr_rx_exec, + + [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, + [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, + [INSTR_HDR_EXTRACT3] = instr_hdr_extract3_exec, + [INSTR_HDR_EXTRACT4] = instr_hdr_extract4_exec, + [INSTR_HDR_EXTRACT5] = instr_hdr_extract5_exec, + [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, + [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, + [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, +}; + +static inline void +instr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + instr_exec_t instr = instruction_table[ip->type]; + + instr(p); } /* @@ -2226,6 +2773,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) + instr_exec(p); +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 47a0f8dcc..fb83a8820 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -534,6 +534,19 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline run + * + * @param[in] p + * Pipeline handle. + * @param[in] n_instructions + * Number of instructions to execute. + */ +__rte_experimental +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, + uint32_t n_instructions); + /** * Pipeline free * From patchwork Wed Aug 26 15:14:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76004 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 01B90A04B4; Wed, 26 Aug 2020 17:17:09 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B132D1C0DC; Wed, 26 Aug 2020 17:15:19 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id AFB0F1BE51 for ; Wed, 26 Aug 2020 17:15:05 +0200 (CEST) IronPort-SDR: 2ckbH/mcP02HQMjS3eWNsgzDHtS/mNtxjD8ngUP0MWtuG90adUrxxQ61GztlaFM+KxMf4vnYZH QT377J9HWrOg== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879520" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879520" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:58 -0700 IronPort-SDR: BWrxY03wQhMzXAb/3a/YT6zIJwpYG2+JEeWxzBRk17kUlvUa3o3ZB6QbEljY/JlKn6ixk9g5Dw N0uCabiDjNJQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081286" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:57 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:15 +0100 Message-Id: <20200826151445.51500-11-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 10/40] pipeline: add tx and emit instructions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add header emit and packet transmission instructions. Emit adds to the output packet a header that is either generated (e.g. read from table entry by action) or extracted from the input packet. TX ends the pipeline processing; discard is implemented by tx to special port. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 328 +++++++++++++++++++++++++ 1 file changed, 328 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 98772de99..1a637068c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -213,6 +213,9 @@ enum instruction_type { /* rx m.port_in */ INSTR_RX, + /* tx m.port_out */ + INSTR_TX, + /* extract h.header */ INSTR_HDR_EXTRACT, INSTR_HDR_EXTRACT2, @@ -222,6 +225,17 @@ enum instruction_type { INSTR_HDR_EXTRACT6, INSTR_HDR_EXTRACT7, INSTR_HDR_EXTRACT8, + + /* emit h.header */ + INSTR_HDR_EMIT, + INSTR_HDR_EMIT_TX, + INSTR_HDR_EMIT2_TX, + INSTR_HDR_EMIT3_TX, + INSTR_HDR_EMIT4_TX, + INSTR_HDR_EMIT5_TX, + INSTR_HDR_EMIT6_TX, + INSTR_HDR_EMIT7_TX, + INSTR_HDR_EMIT8_TX, }; struct instr_io { @@ -1635,6 +1649,114 @@ instr_rx_exec(struct rte_swx_pipeline *p) thread_yield(p); } +/* + * tx. + */ +static int +instr_tx_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_TX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +emit_handler(struct thread *t) +{ + struct header_out_runtime *h0 = &t->headers_out[0]; + struct header_out_runtime *h1 = &t->headers_out[1]; + uint32_t offset = 0, i; + + /* No header change or header decapsulation. */ + if ((t->n_headers_out == 1) && + (h0->ptr + h0->n_bytes == t->ptr)) { + TRACE("Emit handler: no header change or header decap.\n"); + + t->pkt.offset -= h0->n_bytes; + t->pkt.length += h0->n_bytes; + + return; + } + + /* Header encapsulation (optionally, with prior header decasulation). */ + if ((t->n_headers_out == 2) && + (h1->ptr + h1->n_bytes == t->ptr) && + (h0->ptr == h0->ptr0)) { + uint32_t offset; + + TRACE("Emit handler: header encapsulation.\n"); + + offset = h0->n_bytes + h1->n_bytes; + memcpy(t->ptr - offset, h0->ptr, h0->n_bytes); + t->pkt.offset -= offset; + t->pkt.length += offset; + + return; + } + + /* Header insertion. */ + /* TBD */ + + /* Header extraction. */ + /* TBD */ + + /* For any other case. */ + TRACE("Emit handler: complex case.\n"); + + for (i = 0; i < t->n_headers_out; i++) { + struct header_out_runtime *h = &t->headers_out[i]; + + memcpy(&t->header_out_storage[offset], h->ptr, h->n_bytes); + offset += h->n_bytes; + } + + if (offset) { + memcpy(t->ptr - offset, t->header_out_storage, offset); + t->pkt.offset -= offset; + t->pkt.length += offset; + } +} + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t port_id = METADATA_READ(t, ip->io.io.offset, ip->io.io.n_bits); + struct port_out_runtime *port = &p->out[port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + + TRACE("[Thread %2u]: tx 1 pkt to port %u\n", + p->thread_id, + (uint32_t)port_id); + + /* Headers. */ + emit_handler(t); + + /* Packet. */ + port->pkt_tx(port->obj, pkt); + + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); +} + /* * extract. */ @@ -1797,6 +1919,185 @@ instr_hdr_extract8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * emit. + */ +static int +instr_hdr_emit_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EMIT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit); + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t n_headers_out = t->n_headers_out; + struct header_out_runtime *ho = &t->headers_out[n_headers_out - 1]; + uint8_t *ho_ptr = NULL; + uint32_t ho_nbytes = 0, i; + + for (i = 0; i < n_emit; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + struct header_runtime *hi = &t->headers[header_id]; + uint8_t *hi_ptr = t->structs[struct_id]; + + TRACE("[Thread %2u]: emit header %u\n", + p->thread_id, + header_id); + + /* Headers. */ + if (!i) { + if (!t->n_headers_out) { + ho = &t->headers_out[0]; + + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out = 1; + + continue; + } else { + ho_ptr = ho->ptr; + ho_nbytes = ho->n_bytes; + } + } + + if (ho_ptr + ho_nbytes == hi_ptr) { + ho_nbytes += n_bytes; + } else { + ho->n_bytes = ho_nbytes; + + ho++; + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out++; + } + } + + ho->n_bytes = ho_nbytes; + t->n_headers_out = n_headers_out; +} + +static inline void +instr_hdr_emit_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_emit_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_emit_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 1); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit2_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 2); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit3_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 3); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit4_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 4); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit5_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 5); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit6_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 6); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit7_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 7); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 9 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 8); + instr_tx_exec(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -1842,6 +2143,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + if (!strcmp(tokens[tpos], "extract")) return instr_hdr_extract_translate(p, action, @@ -1850,6 +2159,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -1971,6 +2288,7 @@ typedef void (*instr_exec_t)(struct rte_swx_pipeline *); static instr_exec_t instruction_table[] = { [INSTR_RX] = instr_rx_exec, + [INSTR_TX] = instr_tx_exec, [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, @@ -1980,6 +2298,16 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, + + [INSTR_HDR_EMIT] = instr_hdr_emit_exec, + [INSTR_HDR_EMIT_TX] = instr_hdr_emit_tx_exec, + [INSTR_HDR_EMIT2_TX] = instr_hdr_emit2_tx_exec, + [INSTR_HDR_EMIT3_TX] = instr_hdr_emit3_tx_exec, + [INSTR_HDR_EMIT4_TX] = instr_hdr_emit4_tx_exec, + [INSTR_HDR_EMIT5_TX] = instr_hdr_emit5_tx_exec, + [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, + [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, + [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, }; static inline void From patchwork Wed Aug 26 15:14:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76006 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BABEEA04B1; Wed, 26 Aug 2020 17:17:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CB4E41C115; Wed, 26 Aug 2020 17:15:22 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 1D39B1BE94 for ; Wed, 26 Aug 2020 17:15:05 +0200 (CEST) IronPort-SDR: EZrQblJiWyvboTAD5uJyv/9bFciXdXKrgNTBEiRG96yRRsSHr7yz84dVPRovGlAyK7lhMw0Pec qWcDnE+VuLww== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879522" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879522" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:59 -0700 IronPort-SDR: Rj8pu5n2H0N1rPyWXwrcFriR7OtAGciL2uRx+OR3oz3ikV+WovNxtyQ2QTiDXHEJy1bvXwTOC3 hNIUtDU+L2wg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081289" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:58 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:16 +0100 Message-Id: <20200826151445.51500-12-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 11/40] pipeline: add header validate and invalidate instructions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add instructions to flag a header as valid or invalid. This flag can be tested by the jmpv (jump if header valid) and jmpnv (jump if header not valid) instructions. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 108 +++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 1a637068c..8b1f290c0 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -236,6 +236,12 @@ enum instruction_type { INSTR_HDR_EMIT6_TX, INSTR_HDR_EMIT7_TX, INSTR_HDR_EMIT8_TX, + + /* validate h.header */ + INSTR_HDR_VALIDATE, + + /* invalidate h.header */ + INSTR_HDR_INVALIDATE, }; struct instr_io { @@ -252,10 +258,15 @@ struct instr_io { } hdr; }; +struct instr_hdr_validity { + uint8_t header_id; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; + struct instr_hdr_validity valid; }; }; @@ -2098,6 +2109,84 @@ instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) instr_tx_exec(p); } +/* + * validate. + */ +static int +instr_hdr_validate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_VALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_validate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] validate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_SET(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + +/* + * invalidate. + */ +static int +instr_hdr_invalidate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_INVALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] invalidate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_CLR(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2167,6 +2256,22 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2308,6 +2413,9 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, + + [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, + [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, }; static inline void From patchwork Wed Aug 26 15:14:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76009 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 719CAA04B1; Wed, 26 Aug 2020 17:18:18 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8C2B91C12C; Wed, 26 Aug 2020 17:15:26 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id E1E7E1BE51 for ; Wed, 26 Aug 2020 17:15:06 +0200 (CEST) IronPort-SDR: NEyJeC546ClGwu+RMp1XPBJMkXWq+7l9CO4hm/+6L+8uUyUCsP1zJzdN24S8Stfk1oxZlRiE0p ImshXkND/J1A== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879525" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879525" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:00 -0700 IronPort-SDR: MQ9Atk462CnbTsDAbwNwh+E0pKQ4ZduLi/mjModrlaseuLxeo1Gt+0rhsILSoXSgUMFrmrCtvL uvYBrk+xNAwQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081290" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:59 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:17 +0100 Message-Id: <20200826151445.51500-13-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 12/40] pipeline: add mov instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The mov (i.e. move) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 369 +++++++++++++++++++++++++ 1 file changed, 369 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 8b1f290c0..8963c1831 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6,9 +6,11 @@ #include #include #include +#include #include #include +#include #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -32,6 +34,9 @@ do { \ #define TRACE(...) #endif +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + /* * Struct. */ @@ -242,6 +247,21 @@ enum instruction_type { /* invalidate h.header */ INSTR_HDR_INVALIDATE, + + /* mov dst src + * dst = src + * dst = HMEF, src = HMEFTI + */ + INSTR_MOV, /* dst = MEF, src = MEFT */ + INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_MOV_I, /* dst = HMEF, src = I */ +}; + +struct instr_operand { + uint8_t struct_id; + uint8_t n_bits; + uint8_t offset; + uint8_t pad; }; struct instr_io { @@ -262,11 +282,20 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_dst_src { + struct instr_operand dst; + union { + struct instr_operand src; + uint32_t src_val; + }; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; + struct instr_dst_src mov; }; }; @@ -381,6 +410,57 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define MOV(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->mov.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define MOV_S(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->mov.src.n_bits); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#else + +#define MOV_S MOV + +#endif + +#define MOV_I(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint64_t src = (ip)->mov.src_val; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -944,6 +1024,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **object) +{ + struct extern_obj *obj; + struct field *f; + char *obj_name, *field_name; + + if ((name[0] != 'e') || (name[1] != '.')) + return NULL; + + obj_name = strdup(&name[2]); + if (!obj_name) + return NULL; + + field_name = strchr(obj_name, '.'); + if (!field_name) { + free(obj_name); + return NULL; + } + + *field_name = 0; + field_name++; + + obj = extern_obj_find(p, obj_name); + if (!obj) { + free(obj_name); + return NULL; + } + + f = struct_type_field_find(obj->type->mailbox_struct_type, field_name); + if (!f) { + free(obj_name); + return NULL; + } + + if (object) + *object = obj; + + free(obj_name); + return f; +} + int rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1182,6 +1306,50 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_func **function) +{ + struct extern_func *func; + struct field *f; + char *func_name, *field_name; + + if ((name[0] != 'f') || (name[1] != '.')) + return NULL; + + func_name = strdup(&name[2]); + if (!func_name) + return NULL; + + field_name = strchr(func_name, '.'); + if (!field_name) { + free(func_name); + return NULL; + } + + *field_name = 0; + field_name++; + + func = extern_func_find(p, func_name); + if (!func) { + free(func_name); + return NULL; + } + + f = struct_type_field_find(func->mailbox_struct_type, field_name); + if (!f) { + free(func_name); + return NULL; + } + + if (function) + *function = func; + + free(func_name); + return f; +} + int rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, const char *name, @@ -1562,6 +1730,82 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static struct field * +action_field_parse(struct action *action, const char *name); + +static struct field * +struct_field_parse(struct rte_swx_pipeline *p, + struct action *action, + const char *name, + uint32_t *struct_id) +{ + struct field *f; + + switch (name[0]) { + case 'h': + { + struct header *header; + + f = header_field_parse(p, name, &header); + if (!f) + return NULL; + + *struct_id = header->struct_id; + return f; + } + + case 'm': + { + f = metadata_field_parse(p, name); + if (!f) + return NULL; + + *struct_id = p->metadata_struct_id; + return f; + } + + case 't': + { + if (!action) + return NULL; + + f = action_field_parse(action, name); + if (!f) + return NULL; + + *struct_id = 0; + return f; + } + + case 'e': + { + struct extern_obj *obj; + + f = extern_obj_mailbox_field_parse(p, name, &obj); + if (!f) + return NULL; + + *struct_id = obj->struct_id; + return f; + } + + case 'f': + { + struct extern_func *func; + + f = extern_func_mailbox_field_parse(p, name, &func); + if (!f) + return NULL; + + *struct_id = func->struct_id; + return f; + } + + default: + return NULL; + } +} + static inline void pipeline_port_inc(struct rte_swx_pipeline *p) { @@ -2187,6 +2431,104 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * mov. + */ +static int +instr_mov_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* MOV or MOV_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_MOV; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_MOV_S; + + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src.struct_id = (uint8_t)src_struct_id; + instr->mov.src.n_bits = fsrc->n_bits; + instr->mov.src.offset = fsrc->offset / 8; + return 0; + } + + /* MOV_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_MOV_I; + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_mov_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov\n", + p->thread_id); + + MOV(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov (s)\n", + p->thread_id); + + MOV_S(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov m.f %x\n", + p->thread_id, + ip->mov.src_val); + + MOV_I(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2272,6 +2614,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2416,6 +2766,10 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, + + [INSTR_MOV] = instr_mov_exec, + [INSTR_MOV_S] = instr_mov_s_exec, + [INSTR_MOV_I] = instr_mov_i_exec, }; static inline void @@ -2446,6 +2800,21 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +action_field_find(struct action *a, const char *name) +{ + return a->st ? struct_type_field_find(a->st, name) : NULL; +} + +static struct field * +action_field_parse(struct action *action, const char *name) +{ + if (name[0] != 't' || name[1] != '.') + return NULL; + + return action_field_find(action, &name[2]); +} + int rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char *name, From patchwork Wed Aug 26 15:14:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76007 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2145CA04B1; Wed, 26 Aug 2020 17:17:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1C2431C11D; Wed, 26 Aug 2020 17:15:24 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 9F64814583 for ; Wed, 26 Aug 2020 17:15:06 +0200 (CEST) IronPort-SDR: R/aEJO+VNJwNTMQ5E60aTTSq+1SerFurYLh6WWHfZ9lwy19bAkBCS2Nt+dGG7nNVWybPdAOOP+ OUyV2Urnn0tg== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879528" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879528" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:01 -0700 IronPort-SDR: fSHJCJBwQlXs7/neKPFHxzgOLWfCTurYMhgtXitUWwdGaGpEwbponhrAWso8St08g0A11Ih7S6 9JqiPkvNGRlw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081292" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:00 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:18 +0100 Message-Id: <20200826151445.51500-14-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 13/40] pipeline: add dma instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The DMA instruction handles the bulk read transfer of one header from the table entry action data. Typically used to generate headers, i.e. headers that are not extracted from the input packet. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 207 +++++++++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 8963c1831..07b9f7ef4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -255,6 +255,18 @@ enum instruction_type { INSTR_MOV, /* dst = MEF, src = MEFT */ INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_MOV_I, /* dst = HMEF, src = I */ + + /* dma h.header t.field + * memcpy(h.header, t.field, sizeof(h.header)) + */ + INSTR_DMA_HT, + INSTR_DMA_HT2, + INSTR_DMA_HT3, + INSTR_DMA_HT4, + INSTR_DMA_HT5, + INSTR_DMA_HT6, + INSTR_DMA_HT7, + INSTR_DMA_HT8, }; struct instr_operand { @@ -290,12 +302,26 @@ struct instr_dst_src { }; }; +struct instr_dma { + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + } dst; + + struct { + uint8_t offset[8]; + } src; + + uint16_t n_bytes[8]; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; struct instr_dst_src mov; + struct instr_dma dma; }; }; @@ -2529,6 +2555,170 @@ instr_mov_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * dma. + */ +static int +instr_dma_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1]; + char *src = tokens[2]; + struct header *h; + struct field *tf; + + CHECK(action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + h = header_parse(p, dst); + CHECK(h, EINVAL); + + tf = action_field_parse(action, src); + CHECK(tf, EINVAL); + + instr->type = INSTR_DMA_HT; + instr->dma.dst.header_id[0] = h->id; + instr->dma.dst.struct_id[0] = h->struct_id; + instr->dma.n_bytes[0] = h->st->n_bits / 8; + instr->dma.src.offset[0] = tf->offset / 8; + + return 0; +} + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma); + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *action_data = t->structs[0]; + uint64_t valid_headers = t->valid_headers; + uint32_t i; + + for (i = 0; i < n_dma; i++) { + uint32_t header_id = ip->dma.dst.header_id[i]; + uint32_t struct_id = ip->dma.dst.struct_id[i]; + uint32_t offset = ip->dma.src.offset[i]; + uint32_t n_bytes = ip->dma.n_bytes[i]; + + struct header_runtime *h = &t->headers[header_id]; + uint8_t *h_ptr0 = h->ptr0; + uint8_t *h_ptr = t->structs[struct_id]; + + void *dst = MASK64_BIT_GET(valid_headers, header_id) ? + h_ptr : h_ptr0; + void *src = &action_data[offset]; + + TRACE("[Thread %2u] dma h.s t.f\n", p->thread_id); + + /* Headers. */ + memcpy(dst, src, n_bytes); + t->structs[struct_id] = dst; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + } + + t->valid_headers = valid_headers; +} + +static inline void +instr_dma_ht_exec(struct rte_swx_pipeline *p) +{ + __instr_dma_ht_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2622,6 +2812,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2770,6 +2968,15 @@ static instr_exec_t instruction_table[] = { [INSTR_MOV] = instr_mov_exec, [INSTR_MOV_S] = instr_mov_s_exec, [INSTR_MOV_I] = instr_mov_i_exec, + + [INSTR_DMA_HT] = instr_dma_ht_exec, + [INSTR_DMA_HT2] = instr_dma_ht2_exec, + [INSTR_DMA_HT3] = instr_dma_ht3_exec, + [INSTR_DMA_HT4] = instr_dma_ht4_exec, + [INSTR_DMA_HT5] = instr_dma_ht5_exec, + [INSTR_DMA_HT6] = instr_dma_ht6_exec, + [INSTR_DMA_HT7] = instr_dma_ht7_exec, + [INSTR_DMA_HT8] = instr_dma_ht8_exec, }; static inline void From patchwork Wed Aug 26 15:14:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76008 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 699CAA04B1; Wed, 26 Aug 2020 17:18:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 562CB1C125; Wed, 26 Aug 2020 17:15:25 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id E1F981BE9B for ; Wed, 26 Aug 2020 17:15:06 +0200 (CEST) IronPort-SDR: kbNOlw8BJG8ckz0unmG/MzTJ1O42k2iiZafjeNk5PCf/iZazQE/broHn9SQ5lHkrgflXOpubIK vtWaDfhbj3vA== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879532" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879532" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:02 -0700 IronPort-SDR: PnpJrFmkZwN4+l7buI+wVZ2RpL8yD/j3zU5HYIIk22/7uFp1/bjLia0TCclyT4LRPCjUhIyYoD 3MO2siVdJ/Yg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081300" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:01 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:19 +0100 Message-Id: <20200826151445.51500-15-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 14/40] pipeline: introduce add instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The add instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 302 +++++++++++++++++++++++++ 1 file changed, 302 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 07b9f7ef4..f7569ad6f 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -267,6 +267,17 @@ enum instruction_type { INSTR_DMA_HT6, INSTR_DMA_HT7, INSTR_DMA_HT8, + + /* add dst src + * dst += src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_ADD, /* dst = MEF, src = MEF */ + INSTR_ALU_ADD_MH, /* dst = MEF, src = H */ + INSTR_ALU_ADD_HM, /* dst = H, src = MEF */ + INSTR_ALU_ADD_HH, /* dst = H, src = H */ + INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ + INSTR_ALU_ADD_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -322,6 +333,7 @@ struct instruction { struct instr_hdr_validity valid; struct instr_dst_src mov; struct instr_dma dma; + struct instr_dst_src alu; }; }; @@ -436,6 +448,136 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define ALU(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_S(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MH ALU_S + +#define ALU_HM(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#define ALU_HH(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_S ALU +#define ALU_MH ALU +#define ALU_HM ALU +#define ALU_HH ALU + +#endif + +#define ALU_I(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MI ALU_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_HI(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_HI ALU_I + +#endif + #define MOV(thread, ip) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ @@ -2719,6 +2861,151 @@ instr_dma_ht8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * alu. + */ +static int +instr_alu_add_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* ADD, ADD_HM, ADD_MH, ADD_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_ADD; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_ADD_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* ADD_MI, ADD_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_ADD_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_ADD_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_alu_add_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2820,6 +3107,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2977,6 +3272,13 @@ static instr_exec_t instruction_table[] = { [INSTR_DMA_HT6] = instr_dma_ht6_exec, [INSTR_DMA_HT7] = instr_dma_ht7_exec, [INSTR_DMA_HT8] = instr_dma_ht8_exec, + + [INSTR_ALU_ADD] = instr_alu_add_exec, + [INSTR_ALU_ADD_MH] = instr_alu_add_mh_exec, + [INSTR_ALU_ADD_HM] = instr_alu_add_hm_exec, + [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, + [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, + [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, }; static inline void From patchwork Wed Aug 26 15:14:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76010 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6AD76A04B1; Wed, 26 Aug 2020 17:18:28 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 236A91C133; Wed, 26 Aug 2020 17:15:28 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 657D91BE94 for ; Wed, 26 Aug 2020 17:15:07 +0200 (CEST) IronPort-SDR: FC9Q1OfMww3s+DUs40Gef16b8B2FHojfBuQKjRBUtMygYwC2lVwsbA79jCr/nC1S4WdDjJhM6i G7p0Er4eWLCw== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879536" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879536" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:03 -0700 IronPort-SDR: H5PMO06whigODA5wwufLZU55xhTMOki4i3InwO+78Gj2EvL1/ScDiRvg3WYHXusSTZq8AmtPm2 HE8DYq5p+vVw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081314" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:02 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:20 +0100 Message-Id: <20200826151445.51500-16-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 15/40] pipeline: introduce sub instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The sub (i.e. subtract) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index f7569ad6f..79629a15e 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -278,6 +278,17 @@ enum instruction_type { INSTR_ALU_ADD_HH, /* dst = H, src = H */ INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ INSTR_ALU_ADD_HI, /* dst = H, src = I */ + + /* sub dst src + * dst -= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SUB, /* dst = MEF, src = MEF */ + INSTR_ALU_SUB_MH, /* dst = MEF, src = H */ + INSTR_ALU_SUB_HM, /* dst = H, src = MEF */ + INSTR_ALU_SUB_HH, /* dst = H, src = H */ + INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ + INSTR_ALU_SUB_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -2916,6 +2927,58 @@ instr_alu_add_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_sub_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SUB, SUB_HM, SUB_MH, SUB_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SUB; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SUB_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SUB_MI, SUB_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SUB_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SUB_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3006,6 +3069,96 @@ instr_alu_add_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_sub_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3115,6 +3268,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3279,6 +3440,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, + + [INSTR_ALU_SUB] = instr_alu_sub_exec, + [INSTR_ALU_SUB_MH] = instr_alu_sub_mh_exec, + [INSTR_ALU_SUB_HM] = instr_alu_sub_hm_exec, + [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, + [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, + [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, }; static inline void From patchwork Wed Aug 26 15:14:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76012 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E1F07A04B1; Wed, 26 Aug 2020 17:18:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E0E611C192; Wed, 26 Aug 2020 17:15:30 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id D3F1314583 for ; Wed, 26 Aug 2020 17:15:07 +0200 (CEST) IronPort-SDR: zdHk8NFkJbt6i5kG+X8/G4xedUr6EcotLKQnafitL5J/OvFrZ7wA0D3HmzvONTFzxehZiFMSLw ek9SESMJED7A== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879545" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879545" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:04 -0700 IronPort-SDR: 7hvG2yi5pxwfDwAqL+t0GQdkBozKdZSj7eRlXfAbioKRwSxb9HE5MbcEQ1BO8LrBK5/VfXSm0B vTzoaglnz6mg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081323" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:03 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:21 +0100 Message-Id: <20200826151445.51500-17-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 16/40] pipeline: introduce ckadd instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The ckadd (i.e. checksum add) instruction is used to either compute, verify or update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 230 +++++++++++++++++++++++++ 1 file changed, 230 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 79629a15e..be5758a4a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -289,6 +289,14 @@ enum instruction_type { INSTR_ALU_SUB_HH, /* dst = H, src = H */ INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ INSTR_ALU_SUB_HI, /* dst = H, src = I */ + + /* ckadd dst src + * dst = dst '+ src[0:1] '+ src[2:3] + ... + * dst = H, src = {H, h.header} + */ + INSTR_ALU_CKADD_FIELD, /* src = H */ + INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ + INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ }; struct instr_operand { @@ -2979,6 +2987,53 @@ instr_alu_sub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_ckadd_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + /* CKADD_FIELD. */ + fsrc = header_field_parse(p, src, &hsrc); + if (fsrc) { + instr->type = INSTR_ALU_CKADD_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* CKADD_STRUCT, CKADD_STRUCT20. */ + hsrc = header_parse(p, src); + CHECK(hsrc, EINVAL); + + instr->type = INSTR_ALU_CKADD_STRUCT; + if ((hsrc->st->n_bits / 8) == 20) + instr->type = INSTR_ALU_CKADD_STRUCT20; + + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = hsrc->st->n_bits; + instr->alu.src.offset = 0; /* Unused. */ + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3159,6 +3214,169 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] ckadd (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* The first input (r) is a 16-bit number. The second and the third + * inputs are 32-bit numbers. In the worst case scenario, the sum of the + * three numbers (output r) is a 34-bit number. + */ + r += (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is an 18-bit + * number. In the worst case scenario, the sum of the two numbers is a + * 19-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 3-bit number (0 .. 7). Their sum is a 17-bit number (0 .. 0x10006). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x10006), the output r is (0 .. 7). So no carry bit can be generated, + * therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r0, r1; + + TRACE("[Thread %2u] ckadd (struct of 20 bytes)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + r0 = src32_ptr[0]; /* r0 is a 32-bit number. */ + r1 = src32_ptr[1]; /* r1 is a 32-bit number. */ + r0 += src32_ptr[2]; /* The output r0 is a 33-bit number. */ + r1 += src32_ptr[3]; /* The output r1 is a 33-bit number. */ + r0 += r1 + src32_ptr[4]; /* The output r0 is a 35-bit number. */ + + /* The first input is a 16-bit number. The second input is a 19-bit + * number. Their sum is a 20-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 4-bit number (0 .. 15). The sum is a 17-bit number (0 .. 0x1000E). + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1000E), the output r is (0 .. 15). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + r0 = ~r0 & 0xFFFF; + r0 = r0 ? r0 : 0xFFFF; + + *dst16_ptr = (uint16_t)r0; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r = 0; + uint32_t i; + + TRACE("[Thread %2u] ckadd (struct)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + /* The max number of 32-bit words in a 256-byte header is 8 = 2^3. + * Therefore, in the worst case scenario, a 35-bit number is added to a + * 16-bit number (the input r), so the output r is 36-bit number. + */ + for (i = 0; i < ip->alu.src.n_bits / 32; i++, src32_ptr++) + r += *src32_ptr; + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1000E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3276,6 +3494,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3447,6 +3673,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, + + [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, + [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, + [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, }; static inline void From patchwork Wed Aug 26 15:14:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76011 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 90442A04B1; Wed, 26 Aug 2020 17:18:41 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 682AE1C139; Wed, 26 Aug 2020 17:15:29 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id D56391BE9E for ; Wed, 26 Aug 2020 17:15:07 +0200 (CEST) IronPort-SDR: dvg9Oc9iGdp7ICz/53pzkEuT+MarZ0C4gqDWI8iUmyhwzl9hIc+O2PkTmjMYcTrjaQIPWtwSOW DfZMwGfknEbQ== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879549" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879549" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:05 -0700 IronPort-SDR: wBt9AR9qtGHKlUmcLYgdwGgN2OQ2LTNWZIyzGZ43ZIumelR47xhWimnCR++qa6jcUrf3+A61Rr viJ4/lpa8UCA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081327" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:04 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:22 +0100 Message-Id: <20200826151445.51500-18-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 17/40] pipeline: introduce cksub instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The cksub (i.e. checksum subtract) instruction is used to update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 116 +++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index be5758a4a..bb58aea7f 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -297,6 +297,12 @@ enum instruction_type { INSTR_ALU_CKADD_FIELD, /* src = H */ INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ + + /* cksub dst src + * dst = dst '- src + * dst = H, src = H + */ + INSTR_ALU_CKSUB_FIELD, }; struct instr_operand { @@ -3034,6 +3040,36 @@ instr_alu_ckadd_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_cksub_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + fsrc = header_field_parse(p, src, &hsrc); + CHECK(fsrc, EINVAL); + + instr->type = INSTR_ALU_CKSUB_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3273,6 +3309,77 @@ instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_cksub_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] cksub (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* Subtraction in 1's complement arithmetic (i.e. a '- b) is the same as + * the following sequence of operations in 2's complement arithmetic: + * a '- b = (a - b) % 0xFFFF. + * + * In order to prevent an underflow for the below subtraction, in which + * a 33-bit number (the subtrahend) is taken out of a 16-bit number (the + * minuend), we first add a multiple of the 0xFFFF modulus to the + * minuend. The number we add to the minuend needs to be a 34-bit number + * or higher, so for readability reasons we picked the 36-bit multiple. + * We are effectively turning the 16-bit minuend into a 36-bit number: + * (a - b) % 0xFFFF = (a + 0xFFFF00000 - b) % 0xFFFF. + */ + r += 0xFFFF00000ULL; /* The output r is a 36-bit number. */ + + /* A 33-bit number is subtracted from a 36-bit number (the input r). The + * result (the output r) is a 36-bit number. + */ + r -= (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1001E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) { @@ -3502,6 +3609,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3677,6 +3792,7 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, + [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, }; static inline void From patchwork Wed Aug 26 15:14:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76013 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C1B55A04B1; Wed, 26 Aug 2020 17:19:03 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 671211C199; Wed, 26 Aug 2020 17:15:32 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 2CE7C1BEBA for ; Wed, 26 Aug 2020 17:15:08 +0200 (CEST) IronPort-SDR: I3Tl1a7OvdsbROzw2QTgqwQyQvu00dpx2awlku7pHG5h/2YXcjST3ryRLuKa35A9I3FoEM9neb zOCgX/AmNS2w== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879553" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879553" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:06 -0700 IronPort-SDR: K84Nsw2dR9z2YHtdPoiuolMD57whXIvlGNYSJLIwqfbHWWU2+7Z6XDy8+9VOx2E/sG5nR0t4Wv 4oPckXFdZQvw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081335" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:05 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:23 +0100 Message-Id: <20200826151445.51500-19-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 18/40] pipeline: introduce and instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The and (i.e. bitwise and) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index bb58aea7f..24c08ef67 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -303,6 +303,14 @@ enum instruction_type { * dst = H, src = H */ INSTR_ALU_CKSUB_FIELD, + + /* and dst src + * dst &= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_AND, /* dst = MEF, src = MEFT */ + INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_AND_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3070,6 +3078,55 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_and_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* AND or AND_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_AND; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_AND_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* AND_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_AND_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3250,6 +3307,51 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_and_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3617,6 +3719,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3793,6 +3903,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, + + [INSTR_ALU_AND] = instr_alu_and_exec, + [INSTR_ALU_AND_S] = instr_alu_and_s_exec, + [INSTR_ALU_AND_I] = instr_alu_and_i_exec, }; static inline void From patchwork Wed Aug 26 15:14:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76014 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6CCB8A04B1; Wed, 26 Aug 2020 17:19:17 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9CD861C19E; Wed, 26 Aug 2020 17:15:33 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 9D5381BFD7 for ; Wed, 26 Aug 2020 17:15:08 +0200 (CEST) IronPort-SDR: kRTnDaYNd+7gMzcbfhQuoFsY2Gd6ybmkJKPOvwkURFY53uoXyQb2W28msgm/pVI2nCDK8RQ1Wg fnP+nr1usijA== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879556" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879556" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:06 -0700 IronPort-SDR: pa9kW3vqEbhh6FXJVKR3ItqC5WQtmam73WRO6y6ijNEJXesH6O2fF06x7XxnOLd+Ool+7O55Rq e16yQse7+P5Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081344" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:06 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:24 +0100 Message-Id: <20200826151445.51500-20-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 19/40] pipeline: introduce or instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The or (i.e. bitwise or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 24c08ef67..317eedaad 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -311,6 +311,14 @@ enum instruction_type { INSTR_ALU_AND, /* dst = MEF, src = MEFT */ INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_AND_I, /* dst = HMEF, src = I */ + + /* or dst src + * dst |= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_OR, /* dst = MEF, src = MEFT */ + INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_OR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3127,6 +3135,55 @@ instr_alu_and_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_or_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* OR or OR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_OR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_OR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* OR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_OR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3352,6 +3409,51 @@ instr_alu_and_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_or_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3727,6 +3829,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "or")) + return instr_alu_or_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3907,6 +4017,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_AND] = instr_alu_and_exec, [INSTR_ALU_AND_S] = instr_alu_and_s_exec, [INSTR_ALU_AND_I] = instr_alu_and_i_exec, + + [INSTR_ALU_OR] = instr_alu_or_exec, + [INSTR_ALU_OR_S] = instr_alu_or_s_exec, + [INSTR_ALU_OR_I] = instr_alu_or_i_exec, }; static inline void From patchwork Wed Aug 26 15:14:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76015 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3366EA04B1; Wed, 26 Aug 2020 17:19:28 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AA3E51C1A5; Wed, 26 Aug 2020 17:15:34 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id C2A1B1BE9E for ; Wed, 26 Aug 2020 17:15:08 +0200 (CEST) IronPort-SDR: 3GNT48YpogHQZ3n/vBdJCOIrbYWN6V0zfckT5XYqehrc4Ipw1haw/MHUqx8nSJbyIMWkTE7jcj KmGHLP3hZ3SQ== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879560" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879560" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:08 -0700 IronPort-SDR: MTF8tGG3dMTQt2NRGEpr5SEpJsyJf5sO/yUAG+y3ZFFVXHt88l89432mD/31ZTGeFDO9Hs4Cge 8qsXsOQ6weRg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081366" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:07 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:25 +0100 Message-Id: <20200826151445.51500-21-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 20/40] pipeline: introduce xor instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The xor (i.e. bitwise exclusive or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 317eedaad..20a831fb4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -319,6 +319,14 @@ enum instruction_type { INSTR_ALU_OR, /* dst = MEF, src = MEFT */ INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_OR_I, /* dst = HMEF, src = I */ + + /* xor dst src + * dst ^= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ + INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3184,6 +3192,55 @@ instr_alu_or_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_xor_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* XOR or XOR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_XOR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_XOR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* XOR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_XOR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3454,6 +3511,51 @@ instr_alu_or_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_xor_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3837,6 +3939,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "xor")) + return instr_alu_xor_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4021,6 +4131,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_OR] = instr_alu_or_exec, [INSTR_ALU_OR_S] = instr_alu_or_s_exec, [INSTR_ALU_OR_I] = instr_alu_or_i_exec, + + [INSTR_ALU_XOR] = instr_alu_xor_exec, + [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, + [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, }; static inline void From patchwork Wed Aug 26 15:14:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76016 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B732BA04B1; Wed, 26 Aug 2020 17:19:40 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BA7A11C1AB; Wed, 26 Aug 2020 17:15:35 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 5D51414583 for ; Wed, 26 Aug 2020 17:15:09 +0200 (CEST) IronPort-SDR: qpWGwJZGu57zJR854N4R403LICoulk3Wk9wpyya59h3cFwYuvT1M+GLGuBKe3kW3cODTld+YCp tfE16fKarmnQ== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879564" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879564" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:08 -0700 IronPort-SDR: 06w5SqNfGCjznC4NZyXSPMyK/1eLQQ7v3uNtNdZFA3ME8qAGOm4qY5CGCARYgQcCHKY/2o/P2L B09HrPHsPrjQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081379" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:08 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:26 +0100 Message-Id: <20200826151445.51500-22-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 21/40] pipeline: introduce shl instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The shl (i.e. shift left) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 20a831fb4..c22bc007c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -327,6 +327,17 @@ enum instruction_type { INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ + + /* shl dst src + * dst <<= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHL, /* dst = MEF, src = MEF */ + INSTR_ALU_SHL_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHL_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHL_HH, /* dst = H, src = H */ + INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHL_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3094,6 +3105,58 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shl_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHL, SHL_HM, SHL_MH, SHL_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHL; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHL_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHL_MI, SHL_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHL_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHL_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3421,6 +3484,96 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shl_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -3947,6 +4100,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shl")) + return instr_alu_shl_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4135,6 +4296,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_XOR] = instr_alu_xor_exec, [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, + + [INSTR_ALU_SHL] = instr_alu_shl_exec, + [INSTR_ALU_SHL_MH] = instr_alu_shl_mh_exec, + [INSTR_ALU_SHL_HM] = instr_alu_shl_hm_exec, + [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, + [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, + [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, }; static inline void From patchwork Wed Aug 26 15:14:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76017 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9B475A04B1; Wed, 26 Aug 2020 17:19:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9615A1C1B2; Wed, 26 Aug 2020 17:15:36 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 6A4DC1C002 for ; Wed, 26 Aug 2020 17:15:10 +0200 (CEST) IronPort-SDR: Tt0MSxsDuTja69wSK63tYtmhwtWuvkBRyYdxOUc6gwZqJ5yFPuhKqmPxSphckLBpREWK5+d4F7 nx14CIhOBouQ== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879569" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879569" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:09 -0700 IronPort-SDR: svStI7RL9dnuaBN7Vcx/1wgWdvPPwM3kBlJwdSSpTPePdY21T3JGe/sVT6f9rp6KEWYZ7pz/Km BOuWKHdMOUMw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081389" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:09 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:27 +0100 Message-Id: <20200826151445.51500-23-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 22/40] pipeline: introduce shr instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The shr (i.e. shift right) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index c22bc007c..0c0490eef 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -338,6 +338,17 @@ enum instruction_type { INSTR_ALU_SHL_HH, /* dst = H, src = H */ INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ INSTR_ALU_SHL_HI, /* dst = H, src = I */ + + /* shr dst src + * dst >>= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHR, /* dst = MEF, src = MEF */ + INSTR_ALU_SHR_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHR_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHR_HH, /* dst = H, src = H */ + INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHR_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3157,6 +3168,58 @@ instr_alu_shl_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shr_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHR, SHR_HM, SHR_MH, SHR_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHR; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHR_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHR_MI, SHR_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHR_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHR_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3574,6 +3637,96 @@ instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -4108,6 +4261,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shr")) + return instr_alu_shr_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4303,6 +4464,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, + + [INSTR_ALU_SHR] = instr_alu_shr_exec, + [INSTR_ALU_SHR_MH] = instr_alu_shr_mh_exec, + [INSTR_ALU_SHR_HM] = instr_alu_shr_hm_exec, + [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, + [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, + [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, }; static inline void From patchwork Wed Aug 26 15:14:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76018 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id AFFB0A04B1; Wed, 26 Aug 2020 17:20:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 90C191C1BC; Wed, 26 Aug 2020 17:15:37 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 603801C00D for ; Wed, 26 Aug 2020 17:15:11 +0200 (CEST) IronPort-SDR: oxlzlBLeFTaobGrdU45hhwYaNFn1ta2dZOvyCpnE59vjVy4GXvpT++GAAbgBwMSGkZWHI2sOJ8 CGyYzVTneYVQ== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879572" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879572" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:10 -0700 IronPort-SDR: LvgjIhLbgMSCqjUYbUgJmlHjVh4eHFaTAdkXeCQ34+iDh+v8FijrThCqtljaAWoq/D9zBvZWSf aGosogYediSg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081399" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:10 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:28 +0100 Message-Id: <20200826151445.51500-24-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 23/40] pipeline: introduce table instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The table instruction looks up the input key into the table and then it triggers the execution of the action found in the table entry. On lookup miss, the default table action is executed. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 100 +++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 0c0490eef..51741dc99 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -349,6 +349,9 @@ enum instruction_type { INSTR_ALU_SHR_HH, /* dst = H, src = H */ INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ INSTR_ALU_SHR_HI, /* dst = H, src = I */ + + /* table TABLE */ + INSTR_TABLE, }; struct instr_operand { @@ -376,6 +379,10 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_table { + uint8_t table_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -405,6 +412,7 @@ struct instruction { struct instr_dst_src mov; struct instr_dma dma; struct instr_dst_src alu; + struct instr_table table; }; }; @@ -2057,6 +2065,15 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_action_call(struct rte_swx_pipeline *p, + struct thread *t, + uint32_t action_id) +{ + t->ret = t->ip + 1; + t->ip = p->action_instructions[action_id]; +} + static inline void thread_ip_inc(struct rte_swx_pipeline *p); @@ -2670,6 +2687,79 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * table. + */ +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name); + +static int +instr_table_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct table *t; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + t = table_find(p, tokens[1]); + CHECK(t, EINVAL); + + instr->type = INSTR_TABLE; + instr->table.table_id = t->id; + return 0; +} + +static inline void +instr_table_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t table_id = ip->table.table_id; + struct rte_swx_table_state *ts = &t->table_state[table_id]; + struct table_runtime *table = &t->tables[table_id]; + uint64_t action_id; + uint8_t *action_data; + int done, hit; + + /* Table. */ + done = table->func(ts->obj, + table->mailbox, + table->key, + &action_id, + &action_data, + &hit); + if (!done) { + /* Thread. */ + TRACE("[Thread %2u] table %u (not finalized)\n", + p->thread_id, + table_id); + + thread_yield(p); + return; + } + + action_id = hit ? action_id : ts->default_action_id; + action_data = hit ? action_data : ts->default_action_data; + + TRACE("[Thread %2u] table %u (%s, action %u)\n", + p->thread_id, + table_id, + hit ? "hit" : "miss", + (uint32_t)action_id); + + t->action_id = action_id; + t->structs[0] = action_data; + t->hit = hit; + + /* Thread. */ + thread_ip_action_call(p, t, action_id); +} + /* * mov. */ @@ -4269,6 +4359,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "table")) + return instr_table_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4471,6 +4569,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, + + [INSTR_TABLE] = instr_table_exec, }; static inline void From patchwork Wed Aug 26 15:14:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76019 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0F5ADA04B1; Wed, 26 Aug 2020 17:20:35 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7DCAA1C1E7; Wed, 26 Aug 2020 17:15:40 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 3D8DF1C0AE for ; Wed, 26 Aug 2020 17:15:12 +0200 (CEST) IronPort-SDR: ni7HXysn8K/uH9S1rJiGEet+NEkXB0m+64AJmbchl7598hEQaaHb+nM/mO/w+D8wz8KxOrZccA 4rF3+Pv4MSvA== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879577" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879577" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:11 -0700 IronPort-SDR: ARudv/UlZcaGYYY04kxtfsb1xBtPCIA5neJ+3u0XFq1AAtZj6Wm/DgpqrN6sprC1ljunIdvlkc fqytT1NEFNAg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081406" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:11 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:29 +0100 Message-Id: <20200826151445.51500-25-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 24/40] pipeline: introduce extern instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The extern instruction calls one of the member functions of a given extern object or it calls the given extern function. The function arguments must be written in advance in the maibox. The results are available in the same place after execution. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 175 +++++++++++++++++++++++++ 1 file changed, 175 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 51741dc99..1fb3a24af 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -352,6 +352,12 @@ enum instruction_type { /* table TABLE */ INSTR_TABLE, + + /* extern e.obj.func */ + INSTR_EXTERN_OBJ, + + /* extern f.func */ + INSTR_EXTERN_FUNC, }; struct instr_operand { @@ -383,6 +389,15 @@ struct instr_table { uint8_t table_id; }; +struct instr_extern_obj { + uint8_t ext_obj_id; + uint8_t func_id; +}; + +struct instr_extern_func { + uint8_t ext_func_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -413,6 +428,8 @@ struct instruction { struct instr_dma dma; struct instr_dst_src alu; struct instr_table table; + struct instr_extern_obj ext_obj; + struct instr_extern_func ext_func; }; }; @@ -1271,6 +1288,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_type_member_func * +extern_obj_member_func_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **obj) +{ + struct extern_obj *object; + struct extern_type_member_func *func; + char *object_name, *func_name; + + if (name[0] != 'e' || name[1] != '.') + return NULL; + + object_name = strdup(&name[2]); + if (!object_name) + return NULL; + + func_name = strchr(object_name, '.'); + if (!func_name) { + free(object_name); + return NULL; + } + + *func_name = 0; + func_name++; + + object = extern_obj_find(p, object_name); + if (!object) { + free(object_name); + return NULL; + } + + func = extern_type_member_func_find(object->type, func_name); + if (!func) { + free(object_name); + return NULL; + } + + if (obj) + *obj = object; + + free(object_name); + return func; +} + static struct field * extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1553,6 +1614,16 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_func * +extern_func_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'f' || name[1] != '.') + return NULL; + + return extern_func_find(p, &name[2]); +} + static struct field * extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -2097,6 +2168,12 @@ thread_yield(struct rte_swx_pipeline *p) p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); } +static inline void +thread_yield_cond(struct rte_swx_pipeline *p, int cond) +{ + p->thread_id = (p->thread_id + cond) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + /* * rx. */ @@ -2760,6 +2837,94 @@ instr_table_exec(struct rte_swx_pipeline *p) thread_ip_action_call(p, t, action_id); } +/* + * extern. + */ +static int +instr_extern_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *token = tokens[1]; + + CHECK(n_tokens == 2, EINVAL); + + if (token[0] == 'e') { + struct extern_obj *obj; + struct extern_type_member_func *func; + + func = extern_obj_member_func_parse(p, token, &obj); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_OBJ; + instr->ext_obj.ext_obj_id = obj->id; + instr->ext_obj.func_id = func->id; + + return 0; + } + + if (token[0] == 'f') { + struct extern_func *func; + + func = extern_func_parse(p, token); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_FUNC; + instr->ext_func.ext_func_id = func->id; + + return 0; + } + + CHECK(0, EINVAL); +} + +static inline void +instr_extern_obj_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t obj_id = ip->ext_obj.ext_obj_id; + uint32_t func_id = ip->ext_obj.func_id; + struct extern_obj_runtime *obj = &t->extern_objs[obj_id]; + rte_swx_extern_type_member_func_t func = obj->funcs[func_id]; + + TRACE("[Thread %2u] extern obj %u member func %u\n", + p->thread_id, + obj_id, + func_id); + + /* Extern object member function execute. */ + uint32_t done = func(obj->obj, obj->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + +static inline void +instr_extern_func_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t ext_func_id = ip->ext_func.ext_func_id; + struct extern_func_runtime *ext_func = &t->extern_funcs[ext_func_id]; + rte_swx_extern_func_t func = ext_func->func; + + TRACE("[Thread %2u] extern func %u\n", + p->thread_id, + ext_func_id); + + /* Extern function execute. */ + uint32_t done = func(ext_func->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + /* * mov. */ @@ -4367,6 +4532,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "extern")) + return instr_extern_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4571,6 +4744,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, [INSTR_TABLE] = instr_table_exec, + [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, + [INSTR_EXTERN_FUNC] = instr_extern_func_exec, }; static inline void From patchwork Wed Aug 26 15:14:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76020 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 20246A04B1; Wed, 26 Aug 2020 17:20:47 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9A3241C1F2; Wed, 26 Aug 2020 17:15:41 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id A23F41C0B4 for ; Wed, 26 Aug 2020 17:15:13 +0200 (CEST) IronPort-SDR: RToGsDJkAGbg+gUMhJNkTb7zkZ+89xpVR1s08eSJ9nENEielgcY5ABcVulEzBLyB/fezY3ZXEJ QzZrIT/Lh8zQ== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879579" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879579" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:13 -0700 IronPort-SDR: p2A+uMhr5cjySuQvyhfUHl1v8q/g2kBClj9TqGhw3NmMQOD00o0S0hSYOEQn2lOyr0zKr04bWH ZTgj+OZGR4CA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081410" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:11 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:30 +0100 Message-Id: <20200826151445.51500-26-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 25/40] pipeline: introduce jmp and return instructions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The jump instructions are either unconditional (jmp) or conditional on positive/negative tests such as header validity (jmpv/jmpnv), table lookup hit/miss (jmph/jmpnh), executed action (jmpa/jmpna), equality (jmpeq/jmpneq), comparison result (jmplt/jmpgt). The return instruction resumes the pipeline execution after action subroutine. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 1323 ++++++++++++++++++++++-- 1 file changed, 1211 insertions(+), 112 deletions(-) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 1fb3a24af..4eb1f4228 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -358,6 +358,84 @@ enum instruction_type { /* extern f.func */ INSTR_EXTERN_FUNC, + + /* jmp LABEL + * Unconditional jump + */ + INSTR_JMP, + + /* jmpv LABEL h.header + * Jump if header is valid + */ + INSTR_JMP_VALID, + + /* jmpnv LABEL h.header + * Jump if header is invalid + */ + INSTR_JMP_INVALID, + + /* jmph LABEL + * Jump if table lookup hit + */ + INSTR_JMP_HIT, + + /* jmpnh LABEL + * Jump if table lookup miss + */ + INSTR_JMP_MISS, + + /* jmpa LABEL ACTION + * Jump if action run + */ + INSTR_JMP_ACTION_HIT, + + /* jmpna LABEL ACTION + * Jump if action not run + */ + INSTR_JMP_ACTION_MISS, + + /* jmpeq LABEL a b + * Jump is a is equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_EQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_EQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_EQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmpneq LABEL a b + * Jump is a is not equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_NEQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_NEQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_NEQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmplt LABEL a b + * Jump if a is less than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_LT, /* a = MEF, b = MEF */ + INSTR_JMP_LT_MH, /* a = MEF, b = H */ + INSTR_JMP_LT_HM, /* a = H, b = MEF */ + INSTR_JMP_LT_HH, /* a = H, b = H */ + INSTR_JMP_LT_MI, /* a = MEF, b = I */ + INSTR_JMP_LT_HI, /* a = H, b = I */ + + /* jmpgt LABEL a b + * Jump if a is greater than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_GT, /* a = MEF, b = MEF */ + INSTR_JMP_GT_MH, /* a = MEF, b = H */ + INSTR_JMP_GT_HM, /* a = H, b = MEF */ + INSTR_JMP_GT_HH, /* a = H, b = H */ + INSTR_JMP_GT_MI, /* a = MEF, b = I */ + INSTR_JMP_GT_HI, /* a = H, b = I */ + + /* return + * Return from action + */ + INSTR_RETURN, }; struct instr_operand { @@ -419,6 +497,21 @@ struct instr_dma { uint16_t n_bytes[8]; }; +struct instr_jmp { + struct instruction *ip; + + union { + struct instr_operand a; + uint8_t header_id; + uint8_t action_id; + }; + + union { + struct instr_operand b; + uint32_t b_val; + }; +}; + struct instruction { enum instruction_type type; union { @@ -430,6 +523,7 @@ struct instruction { struct instr_table table; struct instr_extern_obj ext_obj; struct instr_extern_func ext_func; + struct instr_jmp jmp; }; }; @@ -544,6 +638,9 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define HEADER_VALID(thread, header_id) \ + MASK64_BIT_GET((thread)->valid_headers, header_id) + #define ALU(thread, ip, operator) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ @@ -725,6 +822,118 @@ struct thread { *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ } +#define JMP_CMP(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_S(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MH JMP_CMP_S + +#define JMP_CMP_HM(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_HH(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_S JMP_CMP +#define JMP_CMP_MH JMP_CMP +#define JMP_CMP_HM JMP_CMP +#define JMP_CMP_HH JMP_CMP + +#endif + +#define JMP_CMP_I(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MI JMP_CMP_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_HI(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_HI JMP_CMP_I + +#endif + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -2048,6 +2257,42 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static int +instruction_is_jmp(struct instruction *instr) +{ + switch (instr->type) { + case INSTR_JMP: + case INSTR_JMP_VALID: + case INSTR_JMP_INVALID: + case INSTR_JMP_HIT: + case INSTR_JMP_MISS: + case INSTR_JMP_ACTION_HIT: + case INSTR_JMP_ACTION_MISS: + case INSTR_JMP_EQ: + case INSTR_JMP_EQ_S: + case INSTR_JMP_EQ_I: + case INSTR_JMP_NEQ: + case INSTR_JMP_NEQ_S: + case INSTR_JMP_NEQ_I: + case INSTR_JMP_LT: + case INSTR_JMP_LT_MH: + case INSTR_JMP_LT_HM: + case INSTR_JMP_LT_HH: + case INSTR_JMP_LT_MI: + case INSTR_JMP_LT_HI: + case INSTR_JMP_GT: + case INSTR_JMP_GT_MH: + case INSTR_JMP_GT_HM: + case INSTR_JMP_GT_HH: + case INSTR_JMP_GT_MI: + case INSTR_JMP_GT_HI: + return 1; + + default: + return 0; + } +} + static struct field * action_field_parse(struct action *action, const char *name); @@ -2136,6 +2381,12 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_set(struct thread *t, struct instruction *ip) +{ + t->ip = ip; +} + static inline void thread_ip_action_call(struct rte_swx_pipeline *p, struct thread *t, @@ -4351,141 +4602,819 @@ instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } -#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 +/* + * jmp. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name); static int -instr_translate(struct rte_swx_pipeline *p, - struct action *action, - char *string, - struct instruction *instr, - struct instruction_data *data) +instr_jmp_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) { - char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; - int n_tokens = 0, tpos = 0; + CHECK(n_tokens == 2, EINVAL); - /* Parse the instruction string into tokens. */ - for ( ; ; ) { - char *token; + strcpy(data->jmp_label, tokens[1]); - token = strtok_r(string, " \t\v", &string); - if (!token) - break; + instr->type = INSTR_JMP; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); +static int +instr_jmp_valid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - tokens[n_tokens] = token; - n_tokens++; - } + CHECK(n_tokens == 3, EINVAL); - CHECK(n_tokens, EINVAL); + strcpy(data->jmp_label, tokens[1]); - /* Handle the optional instruction label. */ - if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { - strcpy(data->label, tokens[0]); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - tpos += 2; - CHECK(n_tokens - tpos, EINVAL); - } + instr->type = INSTR_JMP_VALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - /* Identify the instruction type. */ - if (!strcmp(tokens[tpos], "rx")) - return instr_rx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_invalid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - if (!strcmp(tokens[tpos], "tx")) - return instr_tx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "extract")) - return instr_hdr_extract_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "emit")) - return instr_hdr_emit_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - if (!strcmp(tokens[tpos], "validate")) - return instr_hdr_validate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_INVALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - if (!strcmp(tokens[tpos], "invalidate")) - return instr_hdr_invalidate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_hit_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "mov")) - return instr_mov_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "dma")) - return instr_dma_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "add")) - return instr_alu_add_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_miss_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "sub")) - return instr_alu_sub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "ckadd")) - return instr_alu_ckadd_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "cksub")) - return instr_alu_cksub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_action_hit_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; - if (!strcmp(tokens[tpos], "and")) - return instr_alu_and_translate(p, + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_action_miss_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_eq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_EQ or JMP_EQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_EQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_EQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_EQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_EQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_neq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_NEQ or JMP_NEQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_NEQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_NEQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_NEQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_NEQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_lt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_LT, JMP_LT_MH, JMP_LT_HM, JMP_LT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_LT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_LT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_LT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_LT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_LT_MI, JMP_LT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_LT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_LT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_gt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_GT, JMP_GT_MH, JMP_GT_HM, JMP_GT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_GT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_GT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_GT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_GT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_GT_MI, JMP_GT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_GT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_GT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static inline void +instr_jmp_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmp\n", p->thread_id); + + thread_ip_set(t, ip->jmp.ip); +} + +static inline void +instr_jmp_valid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_invalid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpnv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {t->ip + 1, ip->jmp.ip}; + + TRACE("[Thread %2u] jmph\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {ip->jmp.ip, t->ip + 1}; + + TRACE("[Thread %2u] jmpnh\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_action_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpa\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_action_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpna\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_eq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq\n", p->thread_id); + + JMP_CMP(t, ip, ==); +} + +static inline void +instr_jmp_eq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, ==); +} + +static inline void +instr_jmp_eq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, ==); +} + +static inline void +instr_jmp_neq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq\n", p->thread_id); + + JMP_CMP(t, ip, !=); +} + +static inline void +instr_jmp_neq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, !=); +} + +static inline void +instr_jmp_neq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, !=); +} + +static inline void +instr_jmp_lt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt\n", p->thread_id); + + JMP_CMP(t, ip, <); +} + +static inline void +instr_jmp_lt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, <); +} + +static inline void +instr_jmp_lt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, <); +} + +static inline void +instr_jmp_lt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, <); +} + +static inline void +instr_jmp_lt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, <); +} + +static inline void +instr_jmp_lt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, <); +} + +static inline void +instr_jmp_gt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt\n", p->thread_id); + + JMP_CMP(t, ip, >); +} + +static inline void +instr_jmp_gt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, >); +} + +static inline void +instr_jmp_gt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, >); +} + +static inline void +instr_jmp_gt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, >); +} + +static inline void +instr_jmp_gt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, >); +} + +static inline void +instr_jmp_gt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, >); +} + +/* + * return. + */ +static int +instr_return_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens __rte_unused, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + CHECK(action, EINVAL); + CHECK(n_tokens == 1, EINVAL); + + instr->type = INSTR_RETURN; + return 0; +} + +static inline void +instr_return_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + TRACE("[Thread %2u] return\n", p->thread_id); + + t->ip = t->ret; +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, action, &tokens[tpos], n_tokens - tpos, @@ -4540,9 +5469,117 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "jmp")) + return instr_jmp_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpv")) + return instr_jmp_valid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnv")) + return instr_jmp_invalid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmph")) + return instr_jmp_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnh")) + return instr_jmp_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpa")) + return instr_jmp_action_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpna")) + return instr_jmp_action_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpeq")) + return instr_jmp_eq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpneq")) + return instr_jmp_neq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmplt")) + return instr_jmp_lt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpgt")) + return instr_jmp_gt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "return")) + return instr_return_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } +static struct instruction_data * +label_find(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t i; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].label)) + return &data[i]; + + return NULL; +} + static uint32_t label_is_used(struct instruction_data *data, uint32_t n, const char *label) { @@ -4590,6 +5627,32 @@ instr_label_check(struct instruction_data *instruction_data, return 0; } +static int +instr_jmp_resolve(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + struct instruction_data *found; + + if (!instruction_is_jmp(instr)) + continue; + + found = label_find(instruction_data, + n_instructions, + data->jmp_label); + CHECK(found, EINVAL); + + instr->jmp.ip = &instr[found - instruction_data]; + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -4638,6 +5701,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_jmp_resolve(instr, data, n_instructions); + if (err) + goto error; + free(data); if (a) { @@ -4746,6 +5813,38 @@ static instr_exec_t instruction_table[] = { [INSTR_TABLE] = instr_table_exec, [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, [INSTR_EXTERN_FUNC] = instr_extern_func_exec, + + [INSTR_JMP] = instr_jmp_exec, + [INSTR_JMP_VALID] = instr_jmp_valid_exec, + [INSTR_JMP_INVALID] = instr_jmp_invalid_exec, + [INSTR_JMP_HIT] = instr_jmp_hit_exec, + [INSTR_JMP_MISS] = instr_jmp_miss_exec, + [INSTR_JMP_ACTION_HIT] = instr_jmp_action_hit_exec, + [INSTR_JMP_ACTION_MISS] = instr_jmp_action_miss_exec, + + [INSTR_JMP_EQ] = instr_jmp_eq_exec, + [INSTR_JMP_EQ_S] = instr_jmp_eq_s_exec, + [INSTR_JMP_EQ_I] = instr_jmp_eq_i_exec, + + [INSTR_JMP_NEQ] = instr_jmp_neq_exec, + [INSTR_JMP_NEQ_S] = instr_jmp_neq_s_exec, + [INSTR_JMP_NEQ_I] = instr_jmp_neq_i_exec, + + [INSTR_JMP_LT] = instr_jmp_lt_exec, + [INSTR_JMP_LT_MH] = instr_jmp_lt_mh_exec, + [INSTR_JMP_LT_HM] = instr_jmp_lt_hm_exec, + [INSTR_JMP_LT_HH] = instr_jmp_lt_hh_exec, + [INSTR_JMP_LT_MI] = instr_jmp_lt_mi_exec, + [INSTR_JMP_LT_HI] = instr_jmp_lt_hi_exec, + + [INSTR_JMP_GT] = instr_jmp_gt_exec, + [INSTR_JMP_GT_MH] = instr_jmp_gt_mh_exec, + [INSTR_JMP_GT_HM] = instr_jmp_gt_hm_exec, + [INSTR_JMP_GT_HH] = instr_jmp_gt_hh_exec, + [INSTR_JMP_GT_MI] = instr_jmp_gt_mi_exec, + [INSTR_JMP_GT_HI] = instr_jmp_gt_hi_exec, + + [INSTR_RETURN] = instr_return_exec, }; static inline void From patchwork Wed Aug 26 15:14:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76021 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 99457A04B1; Wed, 26 Aug 2020 17:21:00 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C89EA1C1FB; Wed, 26 Aug 2020 17:15:42 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id BA5641C0B4 for ; Wed, 26 Aug 2020 17:15:14 +0200 (CEST) IronPort-SDR: vwLolyuiCbep1jV1AomJ8WN0x8mTNHj0nBZNbHzDYBgoSUK8OQ7JlglzPBi6Kcsp3cFylgXLdA uyFwvIU8aKSg== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879583" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879583" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:13 -0700 IronPort-SDR: hhK69dFNKUgTIX/vhxC/7s22D3YEp57oQaRs3B7v0mOZLdwArxBjnXgHyUqpP+I4XieYVdEAGP LoRLgetaYI4g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081419" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:13 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:31 +0100 Message-Id: <20200826151445.51500-27-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 26/40] pipeline: add instruction verifier X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Instruction verifier. Executes at instruction translation time during pipeline build, i.e. initialization instead of run-time. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 51 ++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 4eb1f4228..0016e0f0a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5653,6 +5653,53 @@ instr_jmp_resolve(struct instruction *instructions, return 0; } +static int +instr_verify(struct rte_swx_pipeline *p __rte_unused, + struct action *a, + struct instruction *instr, + struct instruction_data *data __rte_unused, + uint32_t n_instructions) +{ + if (!a) { + enum instruction_type type; + uint32_t i; + + /* Check that the first instruction is rx. */ + CHECK(instr[0].type == INSTR_RX, EINVAL); + + /* Check that there is at least one tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if (instr[i].type == INSTR_TX) + break; + } + CHECK(i < n_instructions, EINVAL); + + /* Check that the last instruction is either tx or unconditional + * jump. + */ + type = instr[n_instructions - 1].type; + CHECK((type == INSTR_TX) || (type == INSTR_JMP), EINVAL); + } + + if (a) { + enum instruction_type type; + uint32_t i; + + /* Check that there is at least one return or tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if ((type == INSTR_RETURN) || (type == INSTR_TX)) + break; + } + CHECK(i < n_instructions, EINVAL); + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5701,6 +5748,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_verify(p, a, instr, data, n_instructions); + if (err) + goto error; + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; From patchwork Wed Aug 26 15:14:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76022 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4CB0BA04B1; Wed, 26 Aug 2020 17:21:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F23001C206; Wed, 26 Aug 2020 17:15:43 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 576781C0C3 for ; Wed, 26 Aug 2020 17:15:15 +0200 (CEST) IronPort-SDR: LGCG2AnehwrwsCoDwMJeToRkRX36qRytYwhX3zvxfuDPGsdyeuN76yk9UnVPmssJUMJArbSj+J 4UMiznoeuDmA== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879587" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879587" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:14 -0700 IronPort-SDR: pp7AF0oxhvacTsp+LTcS1ccB2krrBSs+iYCvHsb/HrRb2omvXvkHY+SVm/YTvPyN9H63N1ed3b 0GQPL3/9bkEQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081422" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:13 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:32 +0100 Message-Id: <20200826151445.51500-28-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 27/40] pipeline: add instruction optimizer X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Instruction optimizer. Detects frequent patterns and replaces them with some more powerful vector-like pipeline instructions without any user effort. Executes at instruction translation, not at run-time. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.c | 226 +++++++++++++++++++++++++ 1 file changed, 226 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 0016e0f0a..d5c2a9c5d 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5700,6 +5700,230 @@ instr_verify(struct rte_swx_pipeline *p __rte_unused, return 0; } +static int +instr_pattern_extract_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EXTRACT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_extract_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static int +instr_pattern_emit_many_tx_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EMIT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (!i) + return 0; + + if (instr[i].type != INSTR_TX) + return 0; + + i++; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_emit_many_tx_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + /* Any emit instruction in addition to the first one. */ + for (i = 1; i < n_instr - 1; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } + + /* The TX instruction is the last one in the pattern. */ + instr[0].type++; + instr[0].io.io.offset = instr[i].io.io.offset; + instr[0].io.io.n_bits = instr[i].io.io.n_bits; + data[i].invalid = 1; +} + +static int +instr_pattern_dma_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_DMA_HT) + break; + + if (i == RTE_DIM(instr->dma.dst.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_dma_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].dma.dst.header_id[i] = instr[i].dma.dst.header_id[0]; + instr[0].dma.dst.struct_id[i] = instr[i].dma.dst.struct_id[0]; + instr[0].dma.src.offset[i] = instr[i].dma.src.offset[0]; + instr[0].dma.n_bytes[i] = instr[i].dma.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static uint32_t +instr_optimize(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i, pos = 0; + + for (i = 0; i < n_instructions; ) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + uint32_t n_instr = 0; + int detected; + + /* Extract many. */ + detected = instr_pattern_extract_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_extract_many_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* Emit many + TX. */ + detected = instr_pattern_emit_many_tx_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_emit_many_tx_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* DMA many. */ + detected = instr_pattern_dma_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_dma_many_optimize(instr, data, n_instr); + i += n_instr; + continue; + } + + /* No pattern starting at the current instruction. */ + i++; + } + + /* Eliminate the invalid instructions that have been optimized out. */ + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + + if (data->invalid) + continue; + + if (i != pos) { + memcpy(&instructions[pos], instr, sizeof(*instr)); + memcpy(&instruction_data[pos], data, sizeof(*data)); + } + + pos++; + } + + return pos; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5752,6 +5976,8 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + n_instructions = instr_optimize(instr, data, n_instructions); + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; From patchwork Wed Aug 26 15:14:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76023 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BFEF4A04B1; Wed, 26 Aug 2020 17:21:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0D2AD1C20A; Wed, 26 Aug 2020 17:15:45 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id DCF831C0D1 for ; Wed, 26 Aug 2020 17:15:16 +0200 (CEST) IronPort-SDR: LpkQzg5Nz5tmE3K3srhtC6cmhBP0v4bRfFZBQys22kaJ4YpQkkjJfsOF8b22vkpJF77pUECziN oIvGr4gIySgw== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879591" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879591" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:15 -0700 IronPort-SDR: nMCDz58zxyItUZbC6OEkpuMDz71GAtj+f2EGQo6AsS4u/flHOqRhTfhtSNCpssVScYFDznGhM9 rnLwiYEQAKFQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081428" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:15 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:33 +0100 Message-Id: <20200826151445.51500-29-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 28/40] pipeline: add pipeline query API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Query API to be used by the control plane to detect the configuration and state of the pipeline and its internal objects. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_pipeline_version.map | 10 + lib/librte_pipeline/rte_swx_ctl.h | 313 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.c | 219 +++++++++++++ 3 files changed, 542 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 793957eb9..bb992fdd0 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -76,4 +76,14 @@ EXPERIMENTAL { rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_info_get; + rte_swx_ctl_pipeline_numa_node_get; + rte_swx_ctl_pipeline_port_in_stats_read; + rte_swx_ctl_pipeline_port_out_stats_read; + rte_swx_ctl_action_info_get; + rte_swx_ctl_action_arg_info_get; + rte_swx_ctl_table_info_get; + rte_swx_ctl_table_match_field_info_get; + rte_swx_ctl_table_action_info_get; + rte_swx_ctl_table_ops_get; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index c824ab56f..bdcc24cee 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -18,8 +18,321 @@ extern "C" { #include +#include "rte_swx_port.h" #include "rte_swx_table.h" +struct rte_swx_pipeline; + +/** Name size. */ +#ifndef RTE_SWX_CTL_NAME_SIZE +#define RTE_SWX_CTL_NAME_SIZE 64 +#endif + +/* + * Pipeline Query API. + */ + +/** Pipeline info. */ +struct rte_swx_ctl_pipeline_info { + /** Number of input ports. */ + uint32_t n_ports_in; + + /** Number of input ports. */ + uint32_t n_ports_out; + + /** Number of actions. */ + uint32_t n_actions; + + /** Number of tables. */ + uint32_t n_tables; +}; + +/** + * Pipeline info get + * + * @param[in] p + * Pipeline handle. + * @param[out] pipeline + * Pipeline info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline); + +/** + * Pipeline NUMA node get + * + * @param[in] p + * Pipeline handle. + * @param[out] numa_node + * Pipeline NUMA node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, + int *numa_node); + +/* + * Ports Query API. + */ + +/** + * Input port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_in* - 1). + * @param[out] stats + * Input port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats); + +/** + * Output port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_out* - 1). + * @param[out] stats + * Output port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats); + +/* + * Action Query API. + */ + +/** Action info. */ +struct rte_swx_ctl_action_info { + /** Action name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of action arguments. */ + uint32_t n_args; +}; + +/** + * Action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[out] action + * Action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action); + +/** Action argument info. */ +struct rte_swx_ctl_action_arg_info { + /** Action argument name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Action argument size (in bits). */ + uint32_t n_bits; +}; + +/** + * Action argument info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[in] action_arg_id + * Action ID (0 .. *n_args* - 1). + * @param[out] action + * Action argument info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg); + +/* + * Table Query API. + */ + +/** Table info. */ +struct rte_swx_ctl_table_info { + /** Table name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Table creation arguments. */ + char args[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of match fields. */ + uint32_t n_match_fields; + + /** Number of actions. */ + uint32_t n_actions; + + /** Non-zero (true) when the default action is constant, therefore it + * cannot be changed; zero (false) when the default action not constant, + * therefore it can be changed. + */ + int default_action_is_const; + + /** Table size parameter. */ + uint32_t size; +}; + +/** + * Table info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables* - 1). + * @param[out] table + * Table info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table); + +/** Table match field info. + * + * If (n_bits, offset) are known for all the match fields of the table, then the + * table (key_offset, key_size, key_mask0) can be computed. + */ +struct rte_swx_ctl_table_match_field_info { + /** Match type of the current match field. */ + enum rte_swx_table_match_type match_type; + + /** Non-zero (true) when the current match field is part of a registered + * header, zero (false) when it is part of the registered meta-data. + */ + int is_header; + + /** Match field size (in bits). */ + uint32_t n_bits; + + /** Match field offset within its parent struct (one of the headers or + * the meta-data). + */ + uint32_t offset; +}; + +/** + * Table match field info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] match_field_id + * Match field ID (0 .. *n_match_fields* - 1). + * @param[out] match_field + * Table match field info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field); + +/** Table action info. */ +struct rte_swx_ctl_table_action_info { + /** Action ID. */ + uint32_t action_id; +}; + +/** + * Table action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] table_action_id + * Action index within the set of table actions (0 .. table n_actions - 1). + * Not to be confused with the action ID, which works at the pipeline level + * (0 .. pipeline n_actions - 1), which is precisely what this function + * returns as part of *table_action*. + * @param[out] table_action + * Table action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action); + +/** + * Table operations get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[out] table_ops + * Table operations. Only valid when function returns success and *is_stub* is + * zero (false). + * @param[out] is_stub + * A stub table is a table with no match fields. No "regular" table entries + * (i.e. entries other than the default entry) can be added to such a table, + * therefore the lookup opeation always results in lookup miss. Non-zero + * (true) when the current table is a stub table, zero (false) otherwise. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub); + /* * Table Update API. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d5c2a9c5d..6963f63a2 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6152,6 +6152,18 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct action * +action_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct action *action = NULL; + + TAILQ_FOREACH(action, &p->actions, node) + if (action->id == id) + return action; + + return NULL; +} + static struct field * action_field_find(struct action *a, const char *name) { @@ -6942,6 +6954,177 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) /* * Control. */ +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline) +{ + struct action *action; + struct table *table; + uint32_t n_actions = 0, n_tables = 0; + + if (!p || !pipeline) + return -EINVAL; + + TAILQ_FOREACH(action, &p->actions, node) + n_actions++; + + TAILQ_FOREACH(table, &p->tables, node) + n_tables++; + + pipeline->n_ports_in = p->n_ports_in; + pipeline->n_ports_out = p->n_ports_out; + pipeline->n_actions = n_actions; + pipeline->n_tables = n_tables; + + return 0; +} + +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, int *numa_node) +{ + if (!p || !numa_node) + return -EINVAL; + + *numa_node = p->numa_node; + return 0; +} + +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action) +{ + struct action *a = NULL; + + if (!p || (action_id >= p->n_actions) || !action) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a) + return -EINVAL; + + strcpy(action->name, a->name); + action->n_args = a->st ? a->st->n_fields : 0; + return 0; +} + +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg) +{ + struct action *a = NULL; + struct field *arg = NULL; + + if (!p || (action_id >= p->n_actions) || !action_arg) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a || !a->st || (action_arg_id >= a->st->n_fields)) + return -EINVAL; + + arg = &a->st->fields[action_arg_id]; + strcpy(action_arg->name, arg->name); + action_arg->n_bits = arg->n_bits; + + return 0; +} + +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table) +{ + struct table *t = NULL; + + if (!p || !table) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + strcpy(table->name, t->name); + strcpy(table->args, t->args); + table->n_match_fields = t->n_fields; + table->n_actions = t->n_actions; + table->default_action_is_const = t->default_action_is_const; + table->size = t->size; + return 0; +} + +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field) +{ + struct table *t; + struct match_field *f; + + if (!p || (table_id >= p->n_tables) || !match_field) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (match_field_id >= t->n_fields)) + return -EINVAL; + + f = &t->fields[match_field_id]; + match_field->match_type = f->match_type; + match_field->is_header = t->is_header; + match_field->n_bits = f->field->n_bits; + match_field->offset = f->field->offset; + + return 0; +} + +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables) || !table_action) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (table_action_id >= t->n_actions)) + return -EINVAL; + + table_action->action_id = t->actions[table_action_id]->id; + + return 0; +} + +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables)) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + if (t->type) { + if (table_ops) + memcpy(table_ops, &t->type->ops, sizeof(*table_ops)); + *is_stub = 0; + } else { + *is_stub = 1; + } + + return 0; +} + int rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, struct rte_swx_table_state **table_state) @@ -6963,3 +7146,39 @@ rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, p->table_state = table_state; return 0; } + +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats) +{ + struct port_in *port; + + if (!p || !stats) + return -EINVAL; + + port = port_in_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} + +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats) +{ + struct port_out *port; + + if (!p || !stats) + return -EINVAL; + + port = port_out_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} From patchwork Wed Aug 26 15:14:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76024 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 51FA8A04B1; Wed, 26 Aug 2020 17:21:40 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E58AB1C211; Wed, 26 Aug 2020 17:15:45 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id CCEC61C0D7 for ; Wed, 26 Aug 2020 17:15:17 +0200 (CEST) IronPort-SDR: QHr/3VdTAI7QTRj9F1UE0IReKwQsw1HnO+eGtN674TIinKX09OKA23hWfsQVJgHhr0m6/C14go 8sF72SAndq7A== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879595" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879595" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:16 -0700 IronPort-SDR: 8PzGV3+vP+zCUI/t3uZYt0nyNMawPPztKMpdfH2QNvcdLymgAw9MEp+1NnwbpyUDUKwR5LJLhH Ti/RdUd8flNg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081437" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:16 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:34 +0100 Message-Id: <20200826151445.51500-30-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 29/40] pipeline: add pipeline flush X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Flush the packets currently buffered by the pipeline output ports. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 13 +++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 12 ++++++++++++ 3 files changed, 26 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index bb992fdd0..730e11a0c 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -74,6 +74,7 @@ EXPERIMENTAL { rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_run; + rte_swx_pipeline_flush; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6963f63a2..64fc187c4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6951,6 +6951,19 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) instr_exec(p); } +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < p->n_ports_out; i++) { + struct port_out_runtime *port = &p->out[i]; + + if (port->flush) + port->flush(port->obj); + } +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index fb83a8820..203e394d6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -547,6 +547,18 @@ void rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions); +/** + * Pipeline flush + * + * Flush all output ports of the pipeline. + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p); + /** * Pipeline free * From patchwork Wed Aug 26 15:14:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76025 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A61FEA04B1; Wed, 26 Aug 2020 17:21:52 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 04B9B1C217; Wed, 26 Aug 2020 17:15:47 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 7407E1C0D1 for ; Wed, 26 Aug 2020 17:15:18 +0200 (CEST) IronPort-SDR: eq14DP8lKIMyDXt7iURIp4Z6MsvWTOOFRbnTIeAX0xD+b8WEqorVlE0U9V1SQa5D7niKArvha5 ZMRdTswqzQoQ== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879597" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879597" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:17 -0700 IronPort-SDR: WWR2Kf/9ojQ9LFfTq0VTv5F0jw7an69dL6lUPmdK6XS2A1nTG+rQpNhnSCgNkdfgEzwjxD5ZiO Sez+rC74oMMQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081443" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:16 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:35 +0100 Message-Id: <20200826151445.51500-31-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 30/40] pipeline: add instruction description X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Added instruction set reference table. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/rte_swx_pipeline.h | 109 +++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 203e394d6..6da5710af 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -345,6 +345,115 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Instructions + */ + +/** + * Instruction operands: + * + *
+-----+---------------------------+------------------+-----+-----+
+ *
|     | Description               | Format           | DST | SRC |
+ *
+-----+---------------------------+------------------+-----+-----+
+ *
| hdr | Header                    | h.header         |     |     |
+ *
+-----+---------------------------+------------------+-----+-----+
+ *
| act | Action                    | ACTION           |     |     |
+ *
+-----+---------------------------+------------------+-----+-----+
+ *
| tbl | Table                     | TABLE            |     |     |
+ *
+-----+---------------------------+------------------+-----+-----+
+ *
| H   | Header field              | h.header.field   | YES | YES |
+ *
+-----+---------------------------+------------------+-----+-----+
+ *
| M   | Meta-data field           | m.field          | YES | YES |
+ *
+-----+---------------------------+------------------+-----+-----+
+ *
| E   | Extern obj mailbox field  | e.ext_obj.field  | YES | YES |
+ *
+-----+---------------------------+------------------+-----+-----+
+ *
| F   | Extern func mailbox field | f.ext_func.field | YES | YES |
+ *
+-----+---------------------------+------------------+-----+-----+
+ *
| T   | Table action data field   | t.header.field   | NO  | YES |
+ *
+-----+---------------------------+------------------+-----+-----+
+ *
| I   | Immediate value (32-bit)  | h.header.field   | NO  | YES |
+ *
+-----+---------------------------+------------------+-----+-----+
+ * + * Instruction set: + * + *
+------------+----------------------+-------------------+------+--------+
+ *
| Instr.     | Instruction          | Instruction       | 1st  | 2nd    |
+ *
| Name       | Description          | Format            | opnd.| opnd.  |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| rx         | Receive one pkt      | rx m.port_in      | M    |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| tx         | Transmit one pkt     | tx m.port_out     | M    |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| extract    | Extract one hdr      | extract h.hdr     | hdr  |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| emit       | Emit one hdr         | emit h.hdr        | hdr  |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| validate   | Validate one hdr     | validate h.hdr    | hdr  |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| invalidate | Invalidate one hdr   | invalidate h.hdr  | hdr  |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| mov        | dst = src            | mov dst src       | HMEF | HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| dma        | memcpy(h.hdr,        | dma h.hdr t.field | hdr  | T      |
+ *
|            |    &t.field,         |                   |      |        |
+ *
|            |    sizeof(h.hdr)     |                   |      |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| add        | dst += src           | add dst src       | HMEF | HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| sub        | dst -= src           | add dst src       | HMEF | HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| ckadd      | Checksum add: dst =  | add dst src       | HMEF | HMEFTI |
+ *
|            | dst '+ src[0:1] '+   |                   |      | or hdr |
+ *
|            | src[2:3] '+ ...      |                   |      |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| cksub      | Checksum subtract:   | add dst src       | HMEF | HMEFTI |
+ *
|            | dst = dst '- src     |                   |      |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| and        | dst &= src           | and dst src       | HMEF | HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| or         | dst |= src           | or  dst src       | HMEF | HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| xor        | dst ^= src           | xor dst src       | HMEF | HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| shl        | dst <<= src          | shl dst src       | HMEF | HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| shr        | dst >>= src          | shr dst src       | HMEF | HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| table      | Table lookup         | table TABLE       | tbl  |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| extern     | Ext obj member func  | extern e.obj.mfunc| ext  |        |
+ *
|            | call or ext func call| extern f.func     |      |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmp        | Unconditional jump   | jmp LABEL         |      |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmpv       | Jump if hdr is valid | jmpv LABEL h.hdr  | hdr  |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmpnv      | Jump if hdr is inval | jmpnv LABEL h.hdr | hdr  |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmph       | Jump if tbl lkp hit  | jmph LABEL        |      |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmpnh      | Jump if tbl lkp miss | jmpnh LABEL       |      |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmpa       | Jump if action run   | jmpa LABEL ACTION | act  |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmpna      | Jump if act not run  | jmpna LABEL ACTION| act  |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmpeq      | Jump if (a == b)     | jmpeq LABEL a b   | HMEFT| HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmpneq     | Jump if (a != b)     | jmpneq LABEL a b  | HMEFT| HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmplt      | Jump if (a < b)      | jmplt LABEL a b   | HMEFT| HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| jmpgt      | Jump if (a > b)      | jmpgt LABEL a b   | HMEFT| HMEFTI |
+ *
+------------+----------------------+-------------------+------+--------+
+ *
| return     | Return from action   | return            |      |        |
+ *
+------------+----------------------+-------------------+------+--------+
+ * + * At initialization time, the pipeline and action instructions (including the + * symbolic name operands) are translated to internal data structures that are + * used at run-time. + */ + /* * Pipeline action */ From patchwork Wed Aug 26 15:14:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76026 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A659AA04B1; Wed, 26 Aug 2020 17:22:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E7BA91C21F; Wed, 26 Aug 2020 17:15:47 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 8CD2F1C0D1 for ; Wed, 26 Aug 2020 17:15:19 +0200 (CEST) IronPort-SDR: EcpXWqIXx/rr5nI+GYpQVokHf15sxewhQ07oJbiXe+il4joIlpTJStWC8kX0D/peQQva6KTg6P s2nmzIPq84tA== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879602" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879602" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:18 -0700 IronPort-SDR: 7W7jEJhIBncw1gMsf9FJSfd+lshLrseNbgTAFgKmoTA46YcsJ8iiqxjbULBSQOwCm8ki9XKXym bCp6Hq3BrVuQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081450" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:17 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:36 +0100 Message-Id: <20200826151445.51500-32-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 31/40] pipeline: add table update high level API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" High-level transaction-oriented API for pipeline table updates. It supports multi-table atomic updates, i.e. multiple tables can be updated in a single step with only the before and after table set visible to the packets. Uses the lower-level table update mechanisms. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/Makefile | 1 + lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 15 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++++++++++++++++ lib/librte_pipeline/rte_swx_ctl.h | 170 ++ 5 files changed, 1737 insertions(+), 4 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.c diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index d214b1aeb..d4a5f77f7 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -22,6 +22,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := rte_pipeline.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_port_in_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_table_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c +SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_ctl.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d5f4d16e5..be1d9c3a4 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -4,7 +4,8 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', - 'rte_swx_pipeline.c',) + 'rte_swx_pipeline.c', + 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 730e11a0c..ec38f0eef 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -1,4 +1,4 @@ -DPDK_21 { +DPDK_20.0 { global: rte_pipeline_ah_packet_drop; @@ -75,8 +75,6 @@ EXPERIMENTAL { rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; - rte_swx_pipeline_table_state_get; - rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; rte_swx_ctl_pipeline_numa_node_get; rte_swx_ctl_pipeline_port_in_stats_read; @@ -87,4 +85,15 @@ EXPERIMENTAL { rte_swx_ctl_table_match_field_info_get; rte_swx_ctl_table_action_info_get; rte_swx_ctl_table_ops_get; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_create; + rte_swx_ctl_pipeline_free; + rte_swx_ctl_pipeline_table_entry_add; + rte_swx_ctl_pipeline_table_default_entry_add; + rte_swx_ctl_pipeline_table_entry_delete; + rte_swx_ctl_pipeline_commit; + rte_swx_ctl_pipeline_abort; + rte_swx_ctl_pipeline_table_entry_read; + rte_swx_ctl_pipeline_table_fprintf; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.c b/lib/librte_pipeline/rte_swx_ctl.c new file mode 100644 index 000000000..44da678c1 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.c @@ -0,0 +1,1552 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include +#include +#include +#include +#include + +#include +#include + +#include "rte_swx_ctl.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN +#define field_ntoh(val, n_bits) (ntoh64((val) << (64 - n_bits))) +#define field_hton(val, n_bits) (hton64((val) << (64 - n_bits))) +#else +#define field_ntoh(val, n_bits) (val) +#define field_hton(val, n_bits) (val) +#endif + +struct action { + struct rte_swx_ctl_action_info info; + struct rte_swx_ctl_action_arg_info *args; + uint32_t data_size; +}; + +struct table { + struct rte_swx_ctl_table_info info; + struct rte_swx_ctl_table_match_field_info *mf; + struct rte_swx_ctl_table_action_info *actions; + struct rte_swx_table_ops ops; + struct rte_swx_table_params params; + + struct rte_swx_table_entry_list entries; + struct rte_swx_table_entry_list pending_add; + struct rte_swx_table_entry_list pending_modify0; + struct rte_swx_table_entry_list pending_modify1; + struct rte_swx_table_entry_list pending_delete; + struct rte_swx_table_entry *pending_default; + + int is_stub; + uint32_t n_add; + uint32_t n_modify; + uint32_t n_delete; +}; + +struct rte_swx_ctl_pipeline { + struct rte_swx_ctl_pipeline_info info; + struct rte_swx_pipeline *p; + struct action *actions; + struct table *tables; + struct rte_swx_table_state *ts; + struct rte_swx_table_state *ts_next; + int numa_node; +}; + +static struct action * +action_find(struct rte_swx_ctl_pipeline *ctl, const char *action_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + + if (!strcmp(action_name, a->info.name)) + return a; + } + + return NULL; +} + +static void +action_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->actions) + return; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *action = &ctl->actions[i]; + + free(action->args); + } + + free(ctl->actions); + ctl->actions = NULL; +} + +static struct table * +table_find(struct rte_swx_ctl_pipeline *ctl, const char *table_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + if (!strcmp(table_name, table->info.name)) + return table; + } + + return NULL; +} + +static int +table_params_get(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + uint8_t *key_mask = NULL; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + uint32_t key_size = 0, key_offset = 0, action_data_size = 0, i; + + if (table->info.n_match_fields) { + struct rte_swx_ctl_table_match_field_info *first, *last; + uint32_t i; + + first = &table->mf[0]; + last = &table->mf[table->info.n_match_fields - 1]; + + /* match_type. */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + + f = &table->mf[i]; + if (f->match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + } + + if (i == table->info.n_match_fields) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + else if ((i == table->info.n_match_fields - 1) && + (last->match_type == RTE_SWX_TABLE_MATCH_LPM)) + match_type = RTE_SWX_TABLE_MATCH_LPM; + + /* key_offset. */ + key_offset = first->offset / 8; + + /* key_size. */ + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* key_mask. */ + key_mask = calloc(1, key_size); + CHECK(key_mask, ENOMEM); + + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + uint32_t start; + size_t size; + + f = &table->mf[i]; + start = (f->offset - first->offset) / 8; + size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + } + + /* action_data_size. */ + for (i = 0; i < table->info.n_actions; i++) { + uint32_t action_id = table->actions[i].action_id; + struct action *a = &ctl->actions[action_id]; + + if (a->data_size > action_data_size) + action_data_size = a->data_size; + } + + /* Fill in. */ + table->params.match_type = match_type; + table->params.key_size = key_size; + table->params.key_offset = key_offset; + table->params.key_mask0 = key_mask; + table->params.action_data_size = action_data_size; + table->params.n_keys_max = table->info.size; + + return 0; +} + +static void +table_entry_free(struct rte_swx_table_entry *entry) +{ + if (!entry) + return; + + free(entry->key); + free(entry->key_mask); + free(entry->action_data); + free(entry); +} + +static struct rte_swx_table_entry * +table_entry_alloc(struct table *table) +{ + struct rte_swx_table_entry *entry; + + entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!entry) + goto error; + + /* key, key_mask. */ + if (!table->is_stub) { + entry->key = calloc(1, table->params.key_size); + if (!entry->key) + goto error; + + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + entry->key_mask = calloc(1, table->params.key_size); + if (!entry->key_mask) + goto error; + } + } + + /* action_data. */ + if (table->params.action_data_size) { + entry->action_data = calloc(1, table->params.action_data_size); + if (!entry->action_data) + goto error; + } + + return entry; + +error: + table_entry_free(entry); + return NULL; +} + +static int +table_entry_check(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_check, + int data_check) +{ + struct table *table = &ctl->tables[table_id]; + + CHECK(entry, EINVAL); + + if (key_check) { + if (table->is_stub) { + /* key. */ + CHECK(!entry->key, EINVAL); + + /* key_mask. */ + CHECK(!entry->key_mask, EINVAL); + } else { + /* key. */ + CHECK(entry->key, EINVAL); + + /* key_mask. */ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_WILDCARD: + break; + + case RTE_SWX_TABLE_MATCH_LPM: + /* TBD Check that key mask is prefix. */ + break; + + case RTE_SWX_TABLE_MATCH_EXACT: + CHECK(!entry->key_mask, EINVAL); + break; + + default: + CHECK(0, EINVAL); + } + } + } + + if (data_check) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + CHECK(i < table->info.n_actions, EINVAL); + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + CHECK((a->data_size && entry->action_data) || + (!a->data_size && !entry->action_data), EINVAL); + } + + return 0; +} + +static struct rte_swx_table_entry * +table_entry_duplicate(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_duplicate, + int data_duplicate) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_entry *new_entry = NULL; + + if (!entry) + goto error; + + new_entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!new_entry) + goto error; + + if (key_duplicate && !table->is_stub) { + /* key. */ + if (!entry->key) + goto error; + + new_entry->key = malloc(table->params.key_size); + if (!new_entry->key) + goto error; + + memcpy(new_entry->key, entry->key, table->params.key_size); + + /* key_signature. */ + new_entry->key_signature = entry->key_signature; + + /* key_mask. */ + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + if (!entry->key_mask) + goto error; + + new_entry->key_mask = malloc(table->params.key_size); + if (!new_entry->key_mask) + goto error; + + memcpy(new_entry->key_mask, + entry->key_mask, + table->params.key_size); + } + } + + if (data_duplicate) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + if (i >= table->info.n_actions) + goto error; + + new_entry->action_id = entry->action_id; + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + if (a->data_size) { + if (!entry->action_data) + goto error; + + new_entry->action_data = malloc(a->data_size); + if (!new_entry->action_data) + goto error; + + memcpy(new_entry->action_data, + entry->action_data, + a->data_size); + } + } + + return entry; + +error: + table_entry_free(new_entry); + return NULL; +} + +static int +entry_keycmp_em(struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1, + uint32_t key_size) +{ + if (e0->key_signature != e1->key_signature) + return 1; /* Not equal. */ + + if (memcmp(e0->key, e1->key, key_size)) + return 1; /* Not equal. */ + + return 0; /* Equal */ +} + +static int +entry_keycmp_wm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +entry_keycmp_lpm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +table_entry_keycmp(struct table *table, + struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1) +{ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_EXACT: + return entry_keycmp_em(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_WILDCARD: + return entry_keycmp_wm(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_LPM: + return entry_keycmp_lpm(e0, e1, table->params.key_size); + + default: + return 1; /* Not equal. */ + } +} + +static struct rte_swx_table_entry * +table_entries_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->entries, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_entries_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->entries); + if (!entry) + break; + + TAILQ_REMOVE(&table->entries, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_add_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_add, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_add_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_add, node); +} + +static void +table_pending_add_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_add); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_add, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify0_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify0, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify0_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify0, node); +} + +static void +table_pending_modify0_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify0); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify0, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify1_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify1, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify1_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify1, node); +} + +static void +table_pending_modify1_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify1); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify1, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_delete_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_delete, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_delete_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_delete, node); +} + +static void +table_pending_delete_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_delete); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_delete, entry, node); + table_entry_free(entry); + } +} + +static void +table_pending_default_free(struct table *table) +{ + if (!table->pending_default) + return; + + free(table->pending_default->action_data); + free(table->pending_default); + table->pending_default = NULL; +} + +static void +table_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->tables) + return; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + free(table->mf); + free(table->actions); + free(table->params.key_mask0); + + table_entries_free(table); + table_pending_add_free(table); + table_pending_modify0_free(table); + table_pending_modify1_free(table); + table_pending_delete_free(table); + table_pending_default_free(table); + } + + free(ctl->tables); + ctl->tables = NULL; +} + +static void +table_state_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->ts_next) + return; + + /* For each table, free its table state. */ + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts_next[i]; + + /* Default action data. */ + free(ts->default_action_data); + + /* Table object. */ + if (!table->is_stub && table->ops.free && ts->obj) + table->ops.free(ts->obj); + } + + free(ctl->ts_next); + ctl->ts_next = NULL; +} + +static int +table_state_create(struct rte_swx_ctl_pipeline *ctl) +{ + int status = 0; + uint32_t i; + + ctl->ts_next = calloc(ctl->info.n_tables, + sizeof(struct rte_swx_table_state)); + if (!ctl->ts_next) { + status = -ENOMEM; + goto error; + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts[i]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[i]; + + /* Table object. */ + if (!table->is_stub) { + ts_next->obj = table->ops.create(&table->params, + &table->entries, + table->info.args, + ctl->numa_node); + if (!ts_next->obj) { + status = -ENODEV; + goto error; + } + } + + /* Default action data: duplicate from current table state. */ + ts_next->default_action_data = + malloc(table->params.action_data_size); + if (!ts_next->default_action_data) { + status = -ENOMEM; + goto error; + } + + memcpy(ts_next->default_action_data, + ts->default_action_data, + table->params.action_data_size); + + ts_next->default_action_id = ts->default_action_id; + } + + return 0; + +error: + table_state_free(ctl); + return status; +} + +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl) +{ + if (!ctl) + return; + + action_free(ctl); + + table_state_free(ctl); + + table_free(ctl); + + free(ctl); +} + +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p) +{ + struct rte_swx_ctl_pipeline *ctl = NULL; + uint32_t i; + int status; + + if (!p) + goto error; + + ctl = calloc(1, sizeof(struct rte_swx_ctl_pipeline)); + if (!ctl) + goto error; + + /* info. */ + status = rte_swx_ctl_pipeline_info_get(p, &ctl->info); + if (status) + goto error; + + /* numa_node. */ + status = rte_swx_ctl_pipeline_numa_node_get(p, &ctl->numa_node); + if (status) + goto error; + + /* p. */ + ctl->p = p; + + /* actions. */ + ctl->actions = calloc(ctl->info.n_actions, sizeof(struct action)); + if (!ctl->actions) + goto error; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_action_info_get(p, i, &a->info); + if (status) + goto error; + + /* args. */ + a->args = calloc(a->info.n_args, + sizeof(struct rte_swx_ctl_action_arg_info)); + if (!a->args) + goto error; + + for (j = 0; j < a->info.n_args; j++) { + status = rte_swx_ctl_action_arg_info_get(p, + i, + j, + &a->args[j]); + if (status) + goto error; + } + + /* data_size. */ + for (j = 0; j < a->info.n_args; j++) { + struct rte_swx_ctl_action_arg_info *info = &a->args[j]; + + a->data_size += info->n_bits; + } + + a->data_size = (a->data_size + 7) / 8; + } + + /* tables. */ + ctl->tables = calloc(ctl->info.n_tables, sizeof(struct table)); + if (!ctl->tables) + goto error; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + + TAILQ_INIT(&t->entries); + TAILQ_INIT(&t->pending_add); + TAILQ_INIT(&t->pending_modify0); + TAILQ_INIT(&t->pending_modify1); + TAILQ_INIT(&t->pending_delete); + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_table_info_get(p, i, &t->info); + if (status) + goto error; + + /* mf. */ + t->mf = calloc(t->info.n_match_fields, + sizeof(struct rte_swx_ctl_table_match_field_info)); + if (!t->mf) + goto error; + + for (j = 0; j < t->info.n_match_fields; j++) { + status = rte_swx_ctl_table_match_field_info_get(p, + i, + j, + &t->mf[j]); + if (status) + goto error; + } + + /* actions. */ + t->actions = calloc(t->info.n_actions, + sizeof(struct rte_swx_ctl_table_action_info)); + if (!t->actions) + goto error; + + for (j = 0; j < t->info.n_actions; j++) { + status = rte_swx_ctl_table_action_info_get(p, + i, + j, + &t->actions[j]); + if (status || + t->actions[j].action_id >= ctl->info.n_actions) + goto error; + } + + /* ops, is_stub. */ + status = rte_swx_ctl_table_ops_get(p, i, &t->ops, &t->is_stub); + if (status) + goto error; + + if ((t->is_stub && t->info.n_match_fields) || + (!t->is_stub && !t->info.n_match_fields)) + goto error; + + /* params. */ + status = table_params_get(ctl, i); + if (status) + goto error; + } + + /* ts. */ + status = rte_swx_pipeline_table_state_get(p, &ctl->ts); + if (status) + goto error; + + /* ts_next. */ + status = table_state_create(ctl); + if (status) + goto error; + + return ctl; + +error: + rte_swx_ctl_pipeline_free(ctl); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry, *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + CHECK(table_name && table_name[0], EINVAL); + + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + new_entry = table_entry_duplicate(ctl, table_id, entry, 1, 1); + CHECK(new_entry, ENOMEM); + + /* The new entry is found in the table->entries list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->entries list to the + * table->pending_modify0 list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is found in the table->pending_add list: + * - Replace the entry in the table->pending_add list with the new entry + * (and free the replaced entry). + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_add, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_modify1 list: + * - Replace the entry in the table->pending_modify1 list with the new + * entry (and free the replaced entry). + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_modify1, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_delete list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->pending_delete list to the + * table->pending_modify0 list. + */ + existing_entry = table_pending_delete_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_delete, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is not found in any of the above lists: + * - Add the new entry to the table->pending_add list. + */ + TAILQ_INSERT_TAIL(&table->pending_add, new_entry, node); + + return 0; +} + +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + CHECK(entry, EINVAL); + CHECK(!table_entry_check(ctl, table_id, entry, 1, 0), EINVAL); + + /* The entry is found in the table->entries list: + * - Move the existing entry from the table->entries list to to the + * table->pending_delete list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_add list: + * - Remove the entry from the table->pending_add list and free it. + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + } + + /* The entry is found in the table->pending_modify1 list: + * - Free the entry in the table->pending_modify1 list; + * - Move the existing entry from the table->pending_modify0 list to the + * table->pending_delete list. + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + struct rte_swx_table_entry *real_existing_entry; + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + real_existing_entry = table_pending_modify0_find(table, entry); + CHECK(real_existing_entry, EINVAL); /* Coverity. */ + + TAILQ_REMOVE(&table->pending_modify0, + real_existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + real_existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_delete list: + * - Do nothing: the existing entry is already in the + * table->pending_delete list, i.e. already marked for delete, so + * simply keep it there as it is. + */ + + /* The entry is not found in any of the above lists: + * - Do nothing: no existing entry to delete. + */ + + return 0; +} + +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + CHECK(!table->info.default_action_is_const, EINVAL); + + new_entry = table_entry_duplicate(ctl, table_id, entry, 0, 1); + CHECK(new_entry, ENOMEM); + + table_pending_default_free(table); + + table->pending_default = new_entry; + return 0; +} + +static int +table_rollfwd0(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Reset counters. */ + table->n_add = 0; + table->n_modify = 0; + table->n_delete = 0; + + /* Add pending rules. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_add++; + } + + /* Modify pending rules. */ + TAILQ_FOREACH(entry, &table->pending_modify1, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_modify++; + } + + /* Delete pending rules. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + int status; + + status = table->ops.del(ts_next->obj, entry); + if (status) + return status; + + table->n_delete++; + } + + return 0; +} + +static void +table_rollfwd1(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct action *a; + uint8_t *action_data; + uint64_t action_id; + + /* Copy the pending default entry. */ + if (!table->pending_default) + return; + + action_id = table->pending_default->action_id; + action_data = table->pending_default->action_data; + a = &ctl->actions[action_id]; + + memcpy(ts_next->default_action_data, + action_data, + a->data_size); + + ts_next->default_action_id = action_id; +} + +static void +table_rollfwd2(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Move all the pending add entries to the table, as they are now part + * of the table. + */ + table_pending_add_admit(table); + + /* Move all the pending modify1 entries to table, are they are now part + * of the table. Free up all the pending modify0 entries, as they are no + * longer part of the table. + */ + table_pending_modify1_admit(table); + table_pending_modify0_free(table); + + /* Free up all the pending delete entries, as they are no longer part of + * the table. + */ + table_pending_delete_free(table); + + /* Free up the pending default entry, as it is now part of the table. */ + table_pending_default_free(table); +} + +static void +table_rollback(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Add back all the entries that were just deleted. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + if (!table->n_delete) + break; + + table->ops.add(ts_next->obj, entry); + table->n_delete--; + } + + /* Add back the old copy for all the entries that were just + * modified. + */ + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + if (!table->n_modify) + break; + + table->ops.add(ts_next->obj, entry); + table->n_modify--; + } + + /* Delete all the entries that were just added. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + if (!table->n_add) + break; + + table->ops.del(ts_next->obj, entry); + table->n_add--; + } +} + +static void +table_abort(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Free up all the pending add entries, as none of them is part of the + * table. + */ + table_pending_add_free(table); + + /* Free up all the pending modify1 entries, as none of them made it to + * the table. Add back all the pending modify0 entries, as none of them + * was deleted from the table. + */ + table_pending_modify1_free(table); + table_pending_modify0_admit(table); + + /* Add back all the pending delete entries, as none of them was deleted + * from the table. + */ + table_pending_delete_admit(table); + + /* Free up the pending default entry, as it is no longer going to be + * added to the table. + */ + table_pending_default_free(table); +} + +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, int abort_on_fail) +{ + struct rte_swx_table_state *ts; + int status = 0; + uint32_t i; + + CHECK(ctl, EINVAL); + + /* Operate the changes on the current ts_next before it becomes the new + * ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + status = table_rollfwd0(ctl, i); + if (status) + goto rollback; + } + + for (i = 0; i < ctl->info.n_tables; i++) + table_rollfwd1(ctl, i); + + /* Swap the table state for the data plane. The current ts and ts_next + * become the new ts_next and ts, respectively. + */ + rte_swx_pipeline_table_state_set(ctl->p, ctl->ts_next); + usleep(100); + ts = ctl->ts; + ctl->ts = ctl->ts_next; + ctl->ts_next = ts; + + /* Operate the changes on the current ts_next, which is the previous ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollfwd0(ctl, i); + table_rollfwd1(ctl, i); + table_rollfwd2(ctl, i); + } + + return 0; + +rollback: + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollback(ctl, i); + if (abort_on_fail) + table_abort(ctl, i); + } + + return status; +} + +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl) + return; + + for (i = 0; i < ctl->info.n_tables; i++) + table_abort(ctl, i); +} + +#define RTE_SWX_CTL_ENTRY_TOKENS_MAX 256 + +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string) +{ + char *tokens[RTE_SWX_CTL_ENTRY_TOKENS_MAX]; + struct table *table; + struct action *action; + struct rte_swx_table_entry *entry = NULL; + char *s0 = NULL, *s; + uint32_t n_tokens = 0, arg_offset = 0, i; + + /* Check input arguments. */ + if (!ctl) + goto error; + + if (!table_name || !table_name[0]) + goto error; + + table = table_find(ctl, table_name); + if (!table) + goto error; + + if (!string || !string[0]) + goto error; + + /* Memory allocation. */ + s0 = strdup(string); + if (!s0) + goto error; + + entry = table_entry_alloc(table); + if (!entry) + goto error; + + /* Parse the string into tokens. */ + for (s = s0; ; ) { + char *token; + + token = strtok_r(s, " \f\n\r\t\v", &s); + if (!token) + break; + + if (n_tokens >= RTE_SWX_CTL_ENTRY_TOKENS_MAX) + goto error; + + tokens[n_tokens] = token; + n_tokens++; + } + + if ((n_tokens < 3 + table->info.n_match_fields) || + strcmp(tokens[0], "match") || + strcmp(tokens[1 + table->info.n_match_fields], "action")) + goto error; + + action = action_find(ctl, tokens[2 + table->info.n_match_fields]); + if (!action) + goto error; + + if (n_tokens != 3 + table->info.n_match_fields + + action->info.n_args * 2) + goto error; + + /* + * Match. + */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *mf = &table->mf[i]; + char *mf_val = tokens[1 + i]; + uint64_t val; + + val = strtoull(mf_val, &mf_val, 0); + if (mf_val[0]) + goto error; + + /* Endianness conversion. */ + if (mf->is_header) + val = field_hton(val, mf->n_bits); + + /* Copy key and key_mask to entry. */ + memcpy(&entry->key[(mf->offset - table->mf[0].offset) / 8], + (uint8_t *)&val, + mf->n_bits / 8); + + /* TBD Set entry->key_mask for wilcard and LPM match tables. */ + } + + /* + * Action. + */ + /* action_id. */ + entry->action_id = action - ctl->actions; + + /* action_data. */ + for (i = 0; i < action->info.n_args; i++) { + struct rte_swx_ctl_action_arg_info *arg = &action->args[i]; + char *arg_name, *arg_val; + uint64_t val; + int is_nbo = 0; + + arg_name = tokens[3 + table->info.n_match_fields + i * 2]; + arg_val = tokens[3 + table->info.n_match_fields + i * 2 + 1]; + + if (strcmp(arg_name, arg->name) || + (strlen(arg_val) < 4) || + ((arg_val[0] != 'H') && (arg_val[0] != 'N')) || + (arg_val[1] != '(') || + (arg_val[strlen(arg_val) - 1] != ')')) + goto error; + + if (arg_val[0] == 'N') + is_nbo = 1; + + arg_val[strlen(arg_val) - 1] = 0; /* Remove the ')'. */ + arg_val += 2; /* Remove the "H(" or "N(". */ + + val = strtoull(arg_val, &arg_val, 0); + if (arg_val[0]) + goto error; + + /* Endianness conversion. */ + if (is_nbo) + val = field_hton(val, arg->n_bits); + + /* Copy to entry. */ + memcpy(&entry->action_data[arg_offset], + (uint8_t *)&val, + arg->n_bits / 8); + + arg_offset += arg->n_bits / 8; + } + + return entry; + +error: + table_entry_free(entry); + free(s0); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name) +{ + struct table *table; + struct rte_swx_table_entry *entry; + uint32_t n_entries = 0, i; + + if (!f || !ctl || !table_name || !table_name[0]) + return -EINVAL; + + table = table_find(ctl, table_name); + if (!table) + return -EINVAL; + + /* Table. */ + fprintf(f, "# Table %s: key size %u bytes, key offset %u, key mask [", + table->info.name, + table->params.key_size, + table->params.key_offset); + + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", table->params.key_mask0[i]); + + fprintf(f, "], action data size %u bytes\n", + table->params.action_data_size); + + /* Table entries. */ + TAILQ_FOREACH(entry, &table->entries, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_delete, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + fprintf(f, "# Table %s currently has %u entries.\n", + table_name, + n_entries); + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index bdcc24cee..786adedb2 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -391,6 +391,176 @@ int rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, struct rte_swx_table_state *table_state); +/* + * High Level Reference Table Update API. + */ + +/** Pipeline control opaque data structure. */ +struct rte_swx_ctl_pipeline; + +/** + * Pipeline control create + * + * @param[in] p + * Pipeline handle. + * @return + * Pipeline control handle, on success, or NULL, on error. + */ +__rte_experimental +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p); + +/** + * Pipeline table entry add + * + * Schedule entry for addition to table or update as part of the next commit + * operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table default entry add + * + * Schedule table default entry update as part of the next commit operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * The new table default entry. The *key* and *key_mask* entry fields are + * ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table entry delete + * + * Schedule entry for deletion from table as part of the next commit operation. + * Request is silently discarded if no such entry exists. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be deleted from the table. The *action_id* and *action_data* entry + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline commit + * + * Perform all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] abort_on_fail + * When non-zero (false), all the scheduled work is discarded after a failed + * commit. Otherwise, the scheduled work is still kept pending for the next + * commit. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, + int abort_on_fail); + +/** + * Pipeline abort + * + * Discard all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl); + +/** + * Pipeline table entry read + * + * Read table entry from string. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] string + * String containing the table entry. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string); + +/** + * Pipeline table print to file + * + * Print all the table entries to file. + * + * @param[in] f + * Output file. + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name); + +/** + * Pipeline control free + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl); + #ifdef __cplusplus } #endif From patchwork Wed Aug 26 15:14:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76027 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 311CEA04B1; Wed, 26 Aug 2020 17:22:22 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DF5001C221; Wed, 26 Aug 2020 17:15:48 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id CAF7F1C0D2 for ; Wed, 26 Aug 2020 17:15:20 +0200 (CEST) IronPort-SDR: 2x9toj/AFpVsdUjO7wTjDucOCUgSvZKHDwbOmZY1c4+mT6asFBynyxYTrvzfcSgd/UQ2dbf4Sy cyZnQ6JGn1SQ== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879606" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879606" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:20 -0700 IronPort-SDR: aaOv3WSJSdTdSaegC6Z37fsgEpL/6KJSN80Z9eOUu7g1C4dqkTiNor/+uXBZRlIacp+VovUJNt kOOi5k0I1JFw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081458" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:19 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:37 +0100 Message-Id: <20200826151445.51500-33-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 32/40] port: add ethernet device port X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the Ethernet device input/output pipeline port type. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu --- lib/librte_port/Makefile | 2 + lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 3 +- lib/librte_port/rte_swx_port_ethdev.c | 313 ++++++++++++++++++++++++++ lib/librte_port/rte_swx_port_ethdev.h | 54 +++++ 5 files changed, 375 insertions(+), 3 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile index 4221618b3..682336c01 100644 --- a/lib/librte_port/Makefile +++ b/lib/librte_port/Makefile @@ -38,6 +38,7 @@ endif SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_source_sink.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_sym_crypto.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_eventdev.c +SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_swx_port_ethdev.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port.h @@ -56,5 +57,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_source_sink.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_sym_crypto.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_eventdev.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port_ethdev.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 5b5fbf6c4..3d7f309bb 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -10,7 +10,8 @@ sources = files( 'rte_port_sched.c', 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', - 'rte_port_eventdev.c') + 'rte_port_eventdev.c', + 'rte_swx_port_ethdev.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -22,7 +23,8 @@ headers = files( 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', - 'rte_swx_port.h',) + 'rte_swx_port.h', + 'rte_swx_port_ethdev.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index bd1fbb66b..6da5c8074 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -37,5 +37,6 @@ EXPERIMENTAL { rte_port_eventdev_reader_ops; rte_port_eventdev_writer_ops; rte_port_eventdev_writer_nodrop_ops; - + rte_swx_port_ethdev_reader_ops; + rte_swx_port_ethdev_writer_ops; }; diff --git a/lib/librte_port/rte_swx_port_ethdev.c b/lib/librte_port/rte_swx_port_ethdev.c new file mode 100644 index 000000000..18d1c0b5d --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.c @@ -0,0 +1,313 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include +#include +#include +#include + +#include +#include +#include + +#include "rte_swx_port_ethdev.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port ETHDEV Reader + */ +struct reader { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + int n_pkts; + int pos; +}; + +static void * +reader_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_reader_params *params = args; + struct reader *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_rx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct reader)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static int +reader_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct reader *p = port; + struct rte_mbuf *m; + + if (p->pos == p->n_pkts) { + int n_pkts; + + n_pkts = rte_eth_rx_burst(p->params.port_id, + p->params.queue_id, + p->pkts, + p->params.burst_size); + if (!n_pkts) { + p->stats.n_empty++; + return 0; + } + + TRACE("[Ethdev RX port %u queue %u] %d packets in\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + p->n_pkts = n_pkts; + p->pos = 0; + } + + m = p->pkts[p->pos++]; + pkt->handle = m; + pkt->pkt = m->buf_addr; + pkt->offset = m->data_off; + pkt->length = m->pkt_len; + + TRACE("[Ethdev RX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->pos - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, + NULL, + &((uint8_t *)m->buf_addr)[m->data_off], + m->data_len); + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + return 1; +} + +static void +reader_free(void *port) +{ + struct reader *p = port; + int i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) { + struct rte_mbuf *pkt = p->pkts[i]; + + rte_pktmbuf_free(pkt); + } + + free(p->pkts); + free(p); +} + +static void +reader_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct reader *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Port ETHDEV Writer + */ +struct writer { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_out_stats stats; + + struct rte_mbuf **pkts; + int n_pkts; +}; + +static void * +writer_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_writer_params *params = args; + struct writer *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_tx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct writer)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static void +__writer_flush(struct writer *p) +{ + int n_pkts; + + for (n_pkts = 0; ; ) { + n_pkts += rte_eth_tx_burst(p->params.port_id, + p->params.queue_id, + p->pkts + n_pkts, + p->n_pkts - n_pkts); + + TRACE("[Ethdev TX port %u queue %u] %d packets out\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + if (n_pkts == p->n_pkts) + break; + } + + p->n_pkts = 0; +} + +static void +writer_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct writer *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Ethdev TX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->n_pkts - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + p->pkts[p->n_pkts++] = m; + if (p->n_pkts == (int)p->params.burst_size) + __writer_flush(p); +} + +static void +writer_flush(void *port) +{ + struct writer *p = port; + + if (p->n_pkts) + __writer_flush(p); +} + +static void +writer_free(void *port) +{ + struct writer *p = port; + + if (!p) + return; + + writer_flush(p); + free(p->pkts); + free(port); +} + +static void +writer_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct writer *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops = { + .create = reader_create, + .free = reader_free, + .pkt_rx = reader_pkt_rx, + .stats_read = reader_stats_read, +}; + +struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops = { + .create = writer_create, + .free = writer_free, + .pkt_tx = writer_pkt_tx, + .flush = writer_flush, + .stats_read = writer_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_ethdev.h b/lib/librte_port/rte_swx_port_ethdev.h new file mode 100644 index 000000000..cbc2d7b21 --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ +#define __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Ethernet Device Input and Output Ports + */ + +#include + +#include "rte_swx_port.h" + +/** Ethernet device input port (reader) creation parameters. */ +struct rte_swx_port_ethdev_reader_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device receive queue ID. */ + uint16_t queue_id; + + /** Ethernet device receive burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device reader operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops; + +/** Ethernet device output port (writer) creation parameters. */ +struct rte_swx_port_ethdev_writer_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device transmit queue ID. */ + uint16_t queue_id; + + /** Ethernet device transmit burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device writer operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops; + +#ifdef __cplusplus +} +#endif + +#endif From patchwork Wed Aug 26 15:14:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76028 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B8237A04B1; Wed, 26 Aug 2020 17:22:37 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DA8051C229; Wed, 26 Aug 2020 17:15:49 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 973C81C115 for ; Wed, 26 Aug 2020 17:15:21 +0200 (CEST) IronPort-SDR: CDUrS4NhIXqpbtqO+4hlz/5TK4OD3YFFarzjWmyP+CXjOzOjJ8T5fJwlxl8msNbVvwbRKDZobT nxdoAqCBOH3A== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879609" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879609" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:21 -0700 IronPort-SDR: tJhmWhRMUIJf7sTNdXWH4ocWF21VVpocGTzbf/IxHewNEtQ2yr5d4ILA81zaKpRFJEUaJPKA/E 9d236bFrFpUQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081461" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:20 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:38 +0100 Message-Id: <20200826151445.51500-34-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 33/40] port: add source and sink ports X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the PCAP file-based source (input) and sink (output) pipeline port types. The sink port is typically used to implement the packet drop pipeline action. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu --- lib/librte_port/Makefile | 2 + lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 2 + lib/librte_port/rte_swx_port_source_sink.c | 335 +++++++++++++++++++++ lib/librte_port/rte_swx_port_source_sink.h | 57 ++++ 5 files changed, 400 insertions(+), 2 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile index 682336c01..e4a9865d9 100644 --- a/lib/librte_port/Makefile +++ b/lib/librte_port/Makefile @@ -39,6 +39,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_source_sink.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_sym_crypto.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_eventdev.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_swx_port_ethdev.c +SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_swx_port_source_sink.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port.h @@ -58,5 +59,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_sym_crypto.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_eventdev.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port_ethdev.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port_source_sink.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 3d7f309bb..9bbae28b7 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -11,7 +11,8 @@ sources = files( 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', 'rte_port_eventdev.c', - 'rte_swx_port_ethdev.c',) + 'rte_swx_port_ethdev.c', + 'rte_swx_port_source_sink.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -24,7 +25,8 @@ headers = files( 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', 'rte_swx_port.h', - 'rte_swx_port_ethdev.h',) + 'rte_swx_port_ethdev.h', + 'rte_swx_port_source_sink.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index 6da5c8074..eb4dd9347 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -39,4 +39,6 @@ EXPERIMENTAL { rte_port_eventdev_writer_nodrop_ops; rte_swx_port_ethdev_reader_ops; rte_swx_port_ethdev_writer_ops; + rte_swx_port_source_ops; + rte_swx_port_sink_ops; }; diff --git a/lib/librte_port/rte_swx_port_source_sink.c b/lib/librte_port/rte_swx_port_source_sink.c new file mode 100644 index 000000000..dcec9025a --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.c @@ -0,0 +1,335 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include +#include +#include +#ifdef RTE_PORT_PCAP +#include +#endif +#include + +#include +#include + +#include "rte_swx_port_source_sink.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port SOURCE + */ +#ifdef RTE_PORT_PCAP + +struct source { + struct { + struct rte_mempool *pool; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + uint32_t n_pkts; + uint32_t pos; +}; + +static void +source_free(void *port) +{ + struct source *p = port; + uint32_t i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) + rte_pktmbuf_free(p->pkts[i]); + + free(p->pkts); + + free(p); +} + +static void * +source_create(void *args) +{ + char pcap_errbuf[PCAP_ERRBUF_SIZE]; + struct rte_swx_port_source_params *params = args; + struct source *p = NULL; + pcap_t *f = NULL; + uint32_t n_pkts_max, i; + + /* Check input arguments. */ + CHECK(params); + CHECK(params->pool); + CHECK(params->file_name && params->file_name[0]); + n_pkts_max = params->n_pkts_max ? + params->n_pkts_max : + RTE_SWX_PORT_SOURCE_PKTS_MAX; + + /* Resource allocation. */ + f = pcap_open_offline(params->file_name, pcap_errbuf); + if (!f) + goto error; + + p = calloc(1, sizeof(struct source)); + if (!p) + goto error; + + p->pkts = calloc(n_pkts_max, sizeof(struct rte_mbuf *)); + if (!p->pkts) + goto error; + + /* Initialization. */ + p->params.pool = params->pool; + + /* PCAP file. */ + for (i = 0; i < n_pkts_max; i++) { + struct pcap_pkthdr pcap_pkthdr; + const uint8_t *pcap_pktdata; + struct rte_mbuf *m; + uint8_t *m_data; + + /* Read new packet from PCAP file. */ + pcap_pktdata = pcap_next(f, &pcap_pkthdr); + if (!pcap_pktdata) + break; + + /* Allocate new buffer from pool. */ + m = rte_pktmbuf_alloc(params->pool); + if (!m) + goto error; + m_data = rte_pktmbuf_mtod(m, uint8_t *); + + rte_memcpy(m_data, pcap_pktdata, pcap_pkthdr.caplen); + m->data_len = pcap_pkthdr.caplen; + m->pkt_len = pcap_pkthdr.caplen; + + p->pkts[p->n_pkts] = m; + p->n_pkts++; + } + + if (!p->n_pkts) + goto error; + + pcap_close(f); + return p; + +error: + if (p) + source_free(p); + if (f) + pcap_close(f); + return NULL; +} + +static int +source_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct source *p = port; + struct rte_mbuf *m_dst, *m_src; + uint8_t *m_dst_data, *m_src_data; + + /* m_src identification. */ + m_src = p->pkts[p->pos]; + m_src_data = rte_pktmbuf_mtod(m_src, uint8_t *); + + /* m_dst allocation from pool. */ + m_dst = rte_pktmbuf_alloc(p->params.pool); + if (!m_dst) + return 0; + + /* m_dst initialization. */ + m_dst->data_len = m_src->data_len; + m_dst->pkt_len = m_src->pkt_len; + m_dst->data_off = m_src->data_off; + + m_dst_data = rte_pktmbuf_mtod(m_dst, uint8_t *); + rte_memcpy(m_dst_data, m_src_data, m_src->data_len); + + /* pkt initialization. */ + pkt->handle = m_dst; + pkt->pkt = m_dst->buf_addr; + pkt->offset = m_dst->data_off; + pkt->length = m_dst->pkt_len; + + TRACE("[Source port] Pkt RX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + /* port stats update. */ + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + /* m_src next. */ + p->pos++; + if (p->pos == p->n_pkts) + p->pos = 0; + + return 1; +} + +static void +source_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct source *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = source_create, + .free = source_free, + .pkt_rx = source_pkt_rx, + .stats_read = source_stats_read, +}; + +#else + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = NULL, + .free = NULL, + .pkt_rx = NULL, + .stats_read = NULL, +}; + +#endif + +/* + * Port SINK + */ +#ifdef RTE_PORT_PCAP + +struct sink { + struct rte_swx_port_out_stats stats; + pcap_t *f_pcap; + pcap_dumper_t *f_dump; +}; + +static void +sink_free(void *port) +{ + struct sink *p = port; + + if (!p) + return; + + if (p->f_dump) + pcap_dump_close(p->f_dump); + if (p->f_pcap) + pcap_close(p->f_pcap); + free(p); +} + +static void * +sink_create(void *args) +{ + struct rte_swx_port_sink_params *params = args; + struct sink *p; + + /* Memory allocation. */ + p = calloc(1, sizeof(struct sink)); + if (!p) + goto error; + + if (params && params->file_name && params->file_name[0]) { + p->f_pcap = pcap_open_dead(DLT_EN10MB, 65535); + if (!p->f_pcap) + goto error; + + p->f_dump = pcap_dump_open(p->f_pcap, params->file_name); + if (!p->f_dump) + goto error; + } + + return p; + +error: + sink_free(p); + return NULL; +} + +static void +sink_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct sink *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Sink port] Pkt TX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + if (p->f_dump) { + struct pcap_pkthdr pcap_pkthdr; + uint8_t *m_data = rte_pktmbuf_mtod(m, uint8_t *); + + pcap_pkthdr.len = m->pkt_len; + pcap_pkthdr.caplen = m->data_len; + gettimeofday(&pcap_pkthdr.ts, NULL); + + pcap_dump((uint8_t *)p->f_dump, &pcap_pkthdr, m_data); + pcap_dump_flush(p->f_dump); + } + + rte_pktmbuf_free(m); +} + +static void +sink_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct sink *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_out_ops rte_swx_port_sink_ops = { + .create = sink_create, + .free = sink_free, + .pkt_tx = sink_pkt_tx, + .flush = NULL, + .stats_read = sink_stats_read, +}; + +#else + +struct rte_swx_port_out_ops rte_swx_port_sink_ops = { + .create = NULL, + .free = NULL, + .pkt_tx = NULL, + .flush = NULL, + .stats_read = NULL, +}; + +#endif diff --git a/lib/librte_port/rte_swx_port_source_sink.h b/lib/librte_port/rte_swx_port_source_sink.h new file mode 100644 index 000000000..88a890c5a --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ +#define __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Source and Sink Ports + */ + +#include "rte_swx_port.h" + +/** Maximum number of packets to read from the PCAP file. */ +#ifndef RTE_SWX_PORT_SOURCE_PKTS_MAX +#define RTE_SWX_PORT_SOURCE_PKTS_MAX 1024 +#endif + +/** Source port creation parameters. */ +struct rte_swx_port_source_params { + /** Buffer pool. Must be valid. */ + struct rte_mempool *pool; + + /** Name of a valid PCAP file to read the input packets from. */ + const char *file_name; + + /** Maximum number of packets to read from the PCAP file. When 0, it is + * internally set to RTE_SWX_PORT_SOURCE_PKTS_MAX. Once read from the + * PCAP file, the same packets are looped forever. + */ + uint32_t n_pkts_max; +}; + +/** Source port operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_source_ops; + +/** Sink port creation parameters. */ +struct rte_swx_port_sink_params { + /** Name of a valid PCAP file to write the output packets to. When NULL, + * all the output packets are dropped instead of being saved to a PCAP + * file. + */ + const char *file_name; +}; + +/** Sink port operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_sink_ops; + +#ifdef __cplusplus +} +#endif + +#endif From patchwork Wed Aug 26 15:14:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76029 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 34C14A04B1; Wed, 26 Aug 2020 17:22:52 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D5BAE1C231; Wed, 26 Aug 2020 17:15:50 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 1E5F61C121 for ; Wed, 26 Aug 2020 17:15:24 +0200 (CEST) IronPort-SDR: YzFDchq6e8wgjdSN+/ewc5pn90RqrfVA+049sF2DOGmbAYzs/AeKo3Oxz6q4z6ew4azbO95wP2 cBbWyni3EhvA== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879619" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879619" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:24 -0700 IronPort-SDR: 5ATBC3UdsLA5jOURwOrvaEw1NBx+k6Uxfk8LIa1C4tAC/u1FqzQUROmM0OnIMM0rJJtI/IfxSa K1r2WjQUKuzQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081466" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:23 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:39 +0100 Message-Id: <20200826151445.51500-35-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 34/40] table: add exact match table X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the exact match pipeline table type. Used under the hood by the pipeline table instruction. Signed-off-by: Cristian Dumitrescu --- lib/librte_table/Makefile | 2 + lib/librte_table/meson.build | 6 +- lib/librte_table/rte_swx_table_em.c | 851 +++++++++++++++++++++++++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 5 files changed, 894 insertions(+), 2 deletions(-) create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h diff --git a/lib/librte_table/Makefile b/lib/librte_table/Makefile index 9df58698d..f544fd5af 100644 --- a/lib/librte_table/Makefile +++ b/lib/librte_table/Makefile @@ -34,6 +34,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_hash_ext.c SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_hash_lru.c SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_array.c SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_stub.c +SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_swx_table_em.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table.h @@ -56,5 +57,6 @@ endif SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_array.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_stub.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_swx_table.h +SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_swx_table_em.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index b9d4fe3dc..d69678386 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -11,7 +11,8 @@ sources = files('rte_table_acl.c', 'rte_table_hash_ext.c', 'rte_table_hash_lru.c', 'rte_table_array.c', - 'rte_table_stub.c') + 'rte_table_stub.c', + 'rte_swx_table_em.c',) headers = files('rte_table.h', 'rte_table_acl.h', 'rte_table_lpm.h', @@ -23,7 +24,8 @@ headers = files('rte_table.h', 'rte_lru.h', 'rte_table_array.h', 'rte_table_stub.h', - 'rte_swx_table.h',) + 'rte_swx_table.h', + 'rte_swx_table_em.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table_em.c b/lib/librte_table/rte_swx_table_em.c new file mode 100644 index 000000000..85c77ad03 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.c @@ -0,0 +1,851 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include +#include +#include +#include + +#include +#include + +#include "rte_swx_table_em.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#ifndef RTE_SWX_TABLE_EM_USE_HUGE_PAGES +#define RTE_SWX_TABLE_EM_USE_HUGE_PAGES 1 +#endif + +#if RTE_SWX_TABLE_EM_USE_HUGE_PAGES + +#include + +static void * +env_malloc(size_t size, size_t alignment, int numa_node) +{ + return rte_zmalloc_socket(NULL, size, alignment, numa_node); +} + +static void +env_free(void *start, size_t size __rte_unused) +{ + rte_free(start); +} + +#else + +#include + +static void * +env_malloc(size_t size, size_t alignment __rte_unused, int numa_node) +{ + return numa_alloc_onnode(size, numa_node); +} + +static void +env_free(void *start, size_t size) +{ + numa_free(start, size); +} + +#endif + +#if defined(RTE_ARCH_X86_64) + +#include + +#define crc32_u64(crc, v) _mm_crc32_u64(crc, v) + +#else + +static inline uint64_t +crc32_u64_generic(uint64_t crc, uint64_t value) +{ + int i; + + crc = (crc & 0xFFFFFFFFLLU) ^ value; + for (i = 63; i >= 0; i--) { + uint64_t mask; + + mask = -(crc & 1LLU); + crc = (crc >> 1LLU) ^ (0x82F63B78LLU & mask); + } + + return crc; +} + +#define crc32_u64(crc, v) crc32_u64_generic(crc, v) + +#endif + +/* Key size needs to be one of: 8, 16, 32 or 64. */ +static inline uint32_t +hash(void *key, void *key_mask, uint32_t key_size, uint32_t seed) +{ + uint64_t *k = key; + uint64_t *m = key_mask; + uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5; + + switch (key_size) { + case 8: + crc0 = crc32_u64(seed, k[0] & m[0]); + return crc0; + + case 16: + k0 = k[0] & m[0]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc0 ^= crc1; + + return crc0; + + case 32: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = k2 >> 32; + + crc0 = crc32_u64(crc0, crc1); + crc1 = crc32_u64(crc2, crc3); + + crc0 ^= crc1; + + return crc0; + + case 64: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + k5 = k[5] & m[5]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = crc32_u64(k2 >> 32, k[4] & m[4]); + + crc4 = crc32_u64(k5, k[6] & m[6]); + crc5 = crc32_u64(k5 >> 32, k[7] & m[7]); + + crc0 = crc32_u64(crc0, (crc1 << 32) ^ crc2); + crc1 = crc32_u64(crc3, (crc4 << 32) ^ crc5); + + crc0 ^= crc1; + + return crc0; + + default: + crc0 = 0; + return crc0; + } +} + +/* n_bytes needs to be a multiple of 8 bytes. */ +static void +keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes) +{ + uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask; + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + dst64[i] = src64[i] & src_mask64[i]; +} + +/* + * Return: 0 = Keys are NOT equal; 1 = Keys are equal. + */ +static inline uint32_t +keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes) +{ + uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask; + + switch (n_bytes) { + case 8: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint32_t result = 1; + + if (xor0) + result = 0; + return result; + } + + case 16: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t or = xor0 | xor1; + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 32: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t or = (xor0 | xor1) | (xor2 | xor3); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 64: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t xor4 = a64[4] ^ (b64[4] & b_mask64[4]); + uint64_t xor5 = a64[5] ^ (b64[5] & b_mask64[5]); + uint64_t xor6 = a64[6] ^ (b64[6] & b_mask64[6]); + uint64_t xor7 = a64[7] ^ (b64[7] & b_mask64[7]); + uint64_t or = ((xor0 | xor1) | (xor2 | xor3)) | + ((xor4 | xor5) | (xor6 | xor7)); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + default: { + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + if (a64[i] != (b64[i] & b_mask64[i])) + return 0; + return 1; + } + } +} + +#define KEYS_PER_BUCKET 4 + +struct bucket_extension { + struct bucket_extension *next; + uint16_t sig[KEYS_PER_BUCKET]; + uint32_t key_id[KEYS_PER_BUCKET]; +}; + +struct table { + /* Input parameters */ + struct rte_swx_table_params params; + + /* Internal. */ + uint32_t key_size; + uint32_t data_size; + uint32_t key_size_shl; + uint32_t data_size_shl; + uint32_t n_buckets; + uint32_t n_buckets_ext; + uint32_t key_stack_tos; + uint32_t bkt_ext_stack_tos; + uint64_t total_size; + + /* Memory arrays. */ + uint8_t *key_mask; + struct bucket_extension *buckets; + struct bucket_extension *buckets_ext; + uint8_t *keys; + uint32_t *key_stack; + uint32_t *bkt_ext_stack; + uint8_t *data; +}; + +static inline uint8_t * +table_key(struct table *t, uint32_t key_id) +{ + return &t->keys[(uint64_t)key_id << t->key_size_shl]; +} + +static inline uint64_t * +table_key_data(struct table *t, uint32_t key_id) +{ + return (uint64_t *)&t->data[(uint64_t)key_id << t->data_size_shl]; +} + +static inline int +bkt_is_empty(struct bucket_extension *bkt) +{ + return (!bkt->sig[0] && !bkt->sig[1] && !bkt->sig[2] && !bkt->sig[2]) ? + 1 : 0; +} + +/* Return: + * 0 = Bucket key position is NOT empty; + * 1 = Bucket key position is empty. + */ +static inline int +bkt_key_is_empty(struct bucket_extension *bkt, uint32_t bkt_pos) +{ + return bkt->sig[bkt_pos] ? 0 : 1; +} + +/* Return: 0 = Keys are NOT equal; 1 = Keys are equal. */ +static inline int +bkt_keycmp(struct table *t, + struct bucket_extension *bkt, + uint8_t *input_key, + uint32_t bkt_pos, + uint32_t input_sig) +{ + uint32_t bkt_key_id; + uint8_t *bkt_key; + + /* Key signature comparison. */ + if (input_sig != bkt->sig[bkt_pos]) + return 0; + + /* Key comparison. */ + bkt_key_id = bkt->key_id[bkt_pos]; + bkt_key = table_key(t, bkt_key_id); + return keycmp(bkt_key, input_key, t->key_mask, t->key_size); +} + +static inline void +bkt_key_install(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos, + uint32_t bkt_key_id, + uint32_t input_sig) +{ + uint8_t *bkt_key; + uint64_t *bkt_data; + + /* Key signature. */ + bkt->sig[bkt_pos] = (uint16_t)input_sig; + + /* Key. */ + bkt->key_id[bkt_pos] = bkt_key_id; + bkt_key = table_key(t, bkt_key_id); + keycpy(bkt_key, input->key, t->key_mask, t->key_size); + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +static inline void +bkt_key_data_update(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos) +{ + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[bkt_pos]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +#define CL RTE_CACHE_LINE_ROUNDUP + +static int +__table_create(struct table **table, + uint64_t *memory_footprint, + struct rte_swx_table_params *params, + const char *args __rte_unused, + int numa_node) +{ + struct table *t; + uint8_t *memory; + size_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz, + key_stack_sz, bkt_ext_stack_sz, data_sz, total_size; + size_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset, + key_stack_offset, bkt_ext_stack_offset, data_offset; + uint32_t key_size, key_data_size, n_buckets, n_buckets_ext, i; + + /* Check input arguments. */ + CHECK(params, EINVAL); + CHECK(params->match_type == RTE_SWX_TABLE_MATCH_EXACT, EINVAL); + CHECK(params->key_size, EINVAL); + CHECK(params->key_size <= 64, EINVAL); + CHECK(params->n_keys_max, EINVAL); + + /* Memory allocation. */ + key_size = rte_align64pow2(params->key_size); + if (key_size < 8) + key_size = 8; + key_data_size = rte_align64pow2(params->action_data_size + 8); + n_buckets = params->n_keys_max / KEYS_PER_BUCKET; + n_buckets_ext = params->n_keys_max / KEYS_PER_BUCKET; + + table_meta_sz = CL(sizeof(struct table)); + key_mask_sz = CL(key_size); + bucket_sz = CL(n_buckets * sizeof(struct bucket_extension)); + bucket_ext_sz = CL(n_buckets_ext * sizeof(struct bucket_extension)); + key_sz = CL(params->n_keys_max * key_size); + key_stack_sz = CL(params->n_keys_max * sizeof(uint32_t)); + bkt_ext_stack_sz = CL(n_buckets_ext * sizeof(uint32_t)); + data_sz = CL(params->n_keys_max * key_data_size); + total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz + + key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz; + + key_mask_offset = table_meta_sz; + bucket_offset = key_mask_offset + key_mask_sz; + bucket_ext_offset = bucket_offset + bucket_sz; + key_offset = bucket_ext_offset + bucket_ext_sz; + key_stack_offset = key_offset + key_sz; + bkt_ext_stack_offset = key_stack_offset + key_stack_sz; + data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz; + + if (!table) { + if (memory_footprint) + *memory_footprint = total_size; + return 0; + } + + memory = env_malloc(total_size, RTE_CACHE_LINE_SIZE, numa_node); + CHECK(memory, ENOMEM); + memset(memory, 0, total_size); + + /* Initialization. */ + t = (struct table *)memory; + memcpy(&t->params, params, sizeof(*params)); + + t->key_size = key_size; + t->data_size = key_data_size; + t->key_size_shl = __builtin_ctzl(key_size); + t->data_size_shl = __builtin_ctzl(key_data_size); + t->n_buckets = n_buckets; + t->n_buckets_ext = n_buckets_ext; + t->total_size = total_size; + + t->key_mask = &memory[key_mask_offset]; + t->buckets = (struct bucket_extension *)&memory[bucket_offset]; + t->buckets_ext = (struct bucket_extension *)&memory[bucket_ext_offset]; + t->keys = &memory[key_offset]; + t->key_stack = (uint32_t *)&memory[key_stack_offset]; + t->bkt_ext_stack = (uint32_t *)&memory[bkt_ext_stack_offset]; + t->data = &memory[data_offset]; + + t->params.key_mask0 = t->key_mask; + + if (!params->key_mask0) + memset(t->key_mask, 0xFF, params->key_size); + else + memcpy(t->key_mask, params->key_mask0, params->key_size); + + for (i = 0; i < t->params.n_keys_max; i++) + t->key_stack[i] = t->params.n_keys_max - 1 - i; + t->key_stack_tos = t->params.n_keys_max; + + for (i = 0; i < n_buckets_ext; i++) + t->bkt_ext_stack[i] = n_buckets_ext - 1 - i; + t->bkt_ext_stack_tos = n_buckets_ext; + + *table = t; + return 0; +} + +static void +table_free(void *table) +{ + struct table *t = table; + + if (!t) + return; + + env_free(t, t->total_size); +} + +static int +table_add(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + CHECK((!t->params.action_data_size && !entry->action_data) || + (t->params.action_data_size && entry->action_data), EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + bkt_key_data_update(t, bkt, entry, i); + return 0; + } + + /* Key is not present in the bucket. Bucket not full. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_key_is_empty(bkt, i)) { + uint32_t new_bkt_key_id; + + /* Allocate new key & install. */ + CHECK(t->key_stack_tos, ENOSPC); + new_bkt_key_id = + t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, bkt, entry, i, + new_bkt_key_id, input_sig); + return 0; + } + + /* Bucket full: extend bucket. */ + if (t->bkt_ext_stack_tos && t->key_stack_tos) { + struct bucket_extension *new_bkt; + uint32_t new_bkt_id, new_bkt_key_id; + + /* Allocate new bucket extension & install. */ + new_bkt_id = t->bkt_ext_stack[--t->bkt_ext_stack_tos]; + new_bkt = &t->buckets_ext[new_bkt_id]; + memset(new_bkt, 0, sizeof(*new_bkt)); + bkt_prev->next = new_bkt; + + /* Allocate new key & install. */ + new_bkt_key_id = t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, new_bkt, entry, 0, + new_bkt_key_id, input_sig); + return 0; + } + + CHECK(0, ENOSPC); +} + +static int +table_del(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + /* Key free. */ + bkt->sig[i] = 0; + t->key_stack[t->key_stack_tos++] = + bkt->key_id[i]; + + /* Bucket extension free if empty and not the + * 1st in bucket. + */ + if (bkt_prev && bkt_is_empty(bkt)) { + bkt_prev->next = bkt->next; + bkt_id = bkt - t->buckets_ext; + t->bkt_ext_stack[t->bkt_ext_stack_tos++] + = bkt_id; + } + + return 0; + } + + return 0; +} + +static uint64_t +table_mailbox_size_get_unoptimized(void) +{ + return 0; +} + +static int +table_lookup_unoptimized(void *table, + void *mailbox __rte_unused, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt; + uint8_t *input_key; + uint32_t input_sig, bkt_id, i; + + input_key = &(*key)[t->params.key_offset]; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, input_key, i, input_sig)) { + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[i]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = 1; + return 1; + } + + *hit = 0; + return 1; +} + +struct mailbox { + struct bucket_extension *bkt; + uint32_t input_sig; + uint32_t bkt_key_id; + uint32_t sig_match; + uint32_t sig_match_many; + int state; +}; + +static uint64_t +table_mailbox_size_get(void) +{ + return sizeof(struct mailbox); +} + +/* + * mask = match bitmask + * match = at least one match + * match_many = more than one match + * match_pos = position of first match + * + *+------+-------+------------+-----------+ + *| mask | match | match_many | match_pos | + *+------+-------+------------+-----------+ + *| 0000 | 0 | 0 | 00 | + *| 0001 | 1 | 0 | 00 | + *| 0010 | 1 | 0 | 01 | + *| 0011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 0100 | 1 | 0 | 10 | + *| 0101 | 1 | 1 | 00 | + *| 0110 | 1 | 1 | 01 | + *| 0111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1000 | 1 | 0 | 11 | + *| 1001 | 1 | 1 | 00 | + *| 1010 | 1 | 1 | 01 | + *| 1011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1100 | 1 | 1 | 10 | + *| 1101 | 1 | 1 | 00 | + *| 1110 | 1 | 1 | 01 | + *| 1111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + * + * match = 1111_1111_1111_1110 = 0xFFFE + * match_many = 1111_1110_1110_1000 = 0xFEE8 + * match_pos = 0001_0010_0001_0011__0001_0010_0001_0000 = 0x12131210 + * + */ + +#define LUT_MATCH 0xFFFE +#define LUT_MATCH_MANY 0xFEE8 +#define LUT_MATCH_POS 0x12131210 + +static int +table_lookup(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct mailbox *m = mailbox; + + switch (m->state) { + case 0: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt; + uint32_t input_sig, bkt_id; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt = &t->buckets[bkt_id]; + rte_prefetch0(bkt); + + m->bkt = bkt; + m->input_sig = (input_sig >> 16) | 1; + m->state++; + return 0; + } + + case 1: { + struct bucket_extension *bkt = m->bkt; + uint32_t input_sig = m->input_sig; + uint32_t bkt_sig0, bkt_sig1, bkt_sig2, bkt_sig3; + uint32_t mask0 = 0, mask1 = 0, mask2 = 0, mask3 = 0, mask_all; + uint32_t sig_match = LUT_MATCH; + uint32_t sig_match_many = LUT_MATCH_MANY; + uint32_t sig_match_pos = LUT_MATCH_POS; + uint32_t bkt_key_id; + + bkt_sig0 = input_sig ^ bkt->sig[0]; + if (bkt_sig0) + mask0 = 1 << 0; + + bkt_sig1 = input_sig ^ bkt->sig[1]; + if (bkt_sig1) + mask1 = 1 << 1; + + bkt_sig2 = input_sig ^ bkt->sig[2]; + if (bkt_sig2) + mask2 = 1 << 2; + + bkt_sig3 = input_sig ^ bkt->sig[3]; + if (bkt_sig3) + mask3 = 1 << 3; + + mask_all = (mask0 | mask1) | (mask2 | mask3); + sig_match = (sig_match >> mask_all) & 1; + sig_match_many = (sig_match_many >> mask_all) & 1; + sig_match_pos = (sig_match_pos >> (mask_all << 1)) & 3; + + bkt_key_id = bkt->key_id[sig_match_pos]; + rte_prefetch0(table_key(t, bkt_key_id)); + rte_prefetch0(table_key_data(t, bkt_key_id)); + + m->bkt_key_id = bkt_key_id; + m->sig_match = sig_match; + m->sig_match_many = sig_match_many; + m->state++; + return 0; + } + + case 2: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt = m->bkt; + uint32_t bkt_key_id = m->bkt_key_id; + uint8_t *bkt_key = table_key(t, bkt_key_id); + uint64_t *bkt_data = table_key_data(t, bkt_key_id); + uint32_t lkp_hit; + + lkp_hit = keycmp(bkt_key, input_key, t->key_mask, t->key_size); + lkp_hit &= m->sig_match; + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = lkp_hit; + + m->state = 0; + + if (!lkp_hit && (m->sig_match_many || bkt->next)) + return table_lookup_unoptimized(t, + m, + key, + action_id, + action_data, + hit); + + return 1; + } + + default: + return 0; + } +} + +static void * +table_create(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node) +{ + struct table *t; + struct rte_swx_table_entry *entry; + int status; + + /* Table create. */ + status = __table_create(&t, NULL, params, args, numa_node); + if (status) + return NULL; + + /* Table add entries. */ + if (!entries) + return t; + + TAILQ_FOREACH(entry, entries, node) { + int status; + + status = table_add(t, entry); + if (status) { + table_free(t); + return NULL; + } + } + + return t; +} + +static uint64_t +table_footprint(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries __rte_unused, + const char *args) +{ + uint64_t memory_footprint; + int status; + + status = __table_create(NULL, &memory_footprint, params, args, 0); + if (status) + return 0; + + return memory_footprint; +} + +struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get_unoptimized, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup_unoptimized, + .free = table_free, +}; + +struct rte_swx_table_ops rte_swx_table_exact_match_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup, + .free = table_free, +}; diff --git a/lib/librte_table/rte_swx_table_em.h b/lib/librte_table/rte_swx_table_em.h new file mode 100644 index 000000000..909ada483 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_EM_H__ +#define __INCLUDE_RTE_SWX_TABLE_EM_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Exact Match Table + */ + +#include + +#include + +/** Exact match table operations - unoptimized. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops; + +/** Exact match table operations. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_ops; + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map index 568a6c6a8..81c554b63 100644 --- a/lib/librte_table/rte_table_version.map +++ b/lib/librte_table/rte_table_version.map @@ -18,3 +18,10 @@ DPDK_21 { local: *; }; + +EXPERIMENTAL { + global: + + rte_swx_table_exact_match_unoptimized_ops; + rte_swx_table_exact_match_ops; +}; From patchwork Wed Aug 26 15:14:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76030 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 48A2FA04B1; Wed, 26 Aug 2020 17:23:23 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 20DDC1C296; Wed, 26 Aug 2020 17:15:53 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 64D5D1C12A for ; Wed, 26 Aug 2020 17:15:26 +0200 (CEST) IronPort-SDR: BIEBce1FEQOJxpqtEwdPAvsA+6oaX1geuOhA7L0z+YnUbCc31atZzRMJ16+/DLSRZG3Ri0eq3x 8iQyZTw1ex+A== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879623" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879623" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:25 -0700 IronPort-SDR: QTkAagVJq5if1mzl7yqM42HTIy85TBpGhnhWZ7ahqCSbLqdlw+UxMgiVjud//jsMypsXPRHgXz 0QZt1Z2QF6cw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081475" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:24 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:40 +0100 Message-Id: <20200826151445.51500-36-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 35/40] examples/pipeline: add new example application X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add new example application to showcase the new pipeline type API. Signed-off-by: Cristian Dumitrescu --- examples/Makefile | 1 + examples/meson.build | 1 + examples/pipeline/Makefile | 80 +++++ examples/pipeline/main.c | 52 ++++ examples/pipeline/meson.build | 16 + examples/pipeline/obj.c | 470 +++++++++++++++++++++++++++++ examples/pipeline/obj.h | 131 ++++++++ examples/pipeline/thread.c | 549 ++++++++++++++++++++++++++++++++++ examples/pipeline/thread.h | 28 ++ 9 files changed, 1328 insertions(+) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h diff --git a/examples/Makefile b/examples/Makefile index b7e99a2f7..b1ebac681 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -61,6 +61,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += packet_ordering ifeq ($(CONFIG_RTE_ARCH_X86_64),y) DIRS-y += performance-thread endif +DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline DIRS-$(CONFIG_RTE_LIBRTE_IEEE1588) += ptpclient DIRS-$(CONFIG_RTE_LIBRTE_METER) += qos_meter DIRS-$(CONFIG_RTE_LIBRTE_SCHED) += qos_sched diff --git a/examples/meson.build b/examples/meson.build index eb13e8210..245d98575 100644 --- a/examples/meson.build +++ b/examples/meson.build @@ -33,6 +33,7 @@ all_examples = [ 'ntb', 'packet_ordering', 'performance-thread/l3fwd-thread', 'performance-thread/pthread_shim', + 'pipeline', 'ptpclient', 'qos_meter', 'qos_sched', 'rxtx_callbacks', diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile new file mode 100644 index 000000000..3c85e9e40 --- /dev/null +++ b/examples/pipeline/Makefile @@ -0,0 +1,80 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2010-2020 Intel Corporation + +# binary name +APP = pipeline + +# all source are stored in SRCS-y +SRCS-y += main.c +SRCS-y += obj.c +SRCS-y += thread.c + +# Build using pkg-config variables if possible +ifeq ($(shell pkg-config --exists libdpdk && echo 0),0) + +all: shared +.PHONY: shared static +shared: build/$(APP)-shared + ln -sf $(APP)-shared build/$(APP) +static: build/$(APP)-static + ln -sf $(APP)-static build/$(APP) + +PKGCONF ?= pkg-config + +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) +CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) +LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) +LDFLAGS_STATIC = -Wl,-Bstatic $(shell $(PKGCONF) --static --libs libdpdk) + +CFLAGS += -I. + +OBJS := $(patsubst %.c,build/%.o,$(SRCS-y)) + +build/%.o: %.c Makefile $(PC_FILE) | build + $(CC) $(CFLAGS) -c $< -o $@ + +build/$(APP)-shared: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) + +build/$(APP)-static: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) + +build: + @mkdir -p $@ + +.PHONY: clean +clean: + rm -f build/$(APP)* build/*.o + test -d build && rmdir -p build || true + +else + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, detect a build directory, by looking for a path with a .config +RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config))))) + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV_LINUX),y) +$(info This application can only operate in a linux environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +clean: +else + +INC += $(sort $(wildcard *.h)) + +SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := $(SRCS-y) + +CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -I$(SRCDIR) +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif +endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c new file mode 100644 index 000000000..353e62c10 --- /dev/null +++ b/examples/pipeline/main.c @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include +#include +#include +#include +#include + +#include +#include + +#include "obj.h" +#include "thread.h" + +int +main(int argc, char **argv) +{ + struct obj *obj; + int status; + + /* EAL */ + status = rte_eal_init(argc, argv); + if (status < 0) { + printf("Error: EAL initialization failed (%d)\n", status); + return status; + }; + + /* Obj */ + obj = obj_init(); + if (!obj) { + printf("Error: Obj initialization failed (%d)\n", status); + return status; + } + + /* Thread */ + status = thread_init(); + if (status) { + printf("Error: Thread initialization failed (%d)\n", status); + return status; + } + + rte_eal_mp_remote_launch( + thread_main, + NULL, + SKIP_MASTER); + + /* Dispatch loop */ + for ( ; ; ); +} + diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build new file mode 100644 index 000000000..ade485f97 --- /dev/null +++ b/examples/pipeline/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2017-2020 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = cc.has_header('sys/epoll.h') +deps += ['pipeline', 'bus_pci'] +allow_experimental_apis = true +sources = files( + 'main.c', + 'obj.c', + 'thread.c', +) diff --git a/examples/pipeline/obj.c b/examples/pipeline/obj.c new file mode 100644 index 000000000..e37e60540 --- /dev/null +++ b/examples/pipeline/obj.c @@ -0,0 +1,470 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "obj.h" + +/* + * mempool + */ +TAILQ_HEAD(mempool_list, mempool); + +/* + * link + */ +TAILQ_HEAD(link_list, link); + +/* + * pipeline + */ +TAILQ_HEAD(pipeline_list, pipeline); + +/* + * obj + */ +struct obj { + struct mempool_list mempool_list; + struct link_list link_list; + struct pipeline_list pipeline_list; +}; + +/* + * mempool + */ +#define BUFFER_SIZE_MIN (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) + +struct mempool * +mempool_create(struct obj *obj, const char *name, struct mempool_params *params) +{ + struct mempool *mempool; + struct rte_mempool *m; + + /* Check input params */ + if ((name == NULL) || + mempool_find(obj, name) || + (params == NULL) || + (params->buffer_size < BUFFER_SIZE_MIN) || + (params->pool_size == 0)) + return NULL; + + /* Resource create */ + m = rte_pktmbuf_pool_create( + name, + params->pool_size, + params->cache_size, + 0, + params->buffer_size - sizeof(struct rte_mbuf), + params->cpu_id); + + if (m == NULL) + return NULL; + + /* Node allocation */ + mempool = calloc(1, sizeof(struct mempool)); + if (mempool == NULL) { + rte_mempool_free(m); + return NULL; + } + + /* Node fill in */ + strlcpy(mempool->name, name, sizeof(mempool->name)); + mempool->m = m; + mempool->buffer_size = params->buffer_size; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->mempool_list, mempool, node); + + return mempool; +} + +struct mempool * +mempool_find(struct obj *obj, const char *name) +{ + struct mempool *mempool; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(mempool, &obj->mempool_list, node) + if (strcmp(mempool->name, name) == 0) + return mempool; + + return NULL; +} + +/* + * link + */ +static struct rte_eth_conf port_conf_default = { + .link_speeds = 0, + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = 9000, /* Jumbo frame max packet len */ + .split_hdr_size = 0, /* Header split buffer size */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_key_len = 40, + .rss_hf = 0, + }, + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +#define RETA_CONF_SIZE (ETH_RSS_RETA_SIZE_512 / RTE_RETA_GROUP_SIZE) + +static int +rss_setup(uint16_t port_id, + uint16_t reta_size, + struct link_params_rss *rss) +{ + struct rte_eth_rss_reta_entry64 reta_conf[RETA_CONF_SIZE]; + uint32_t i; + int status; + + /* RETA setting */ + memset(reta_conf, 0, sizeof(reta_conf)); + + for (i = 0; i < reta_size; i++) + reta_conf[i / RTE_RETA_GROUP_SIZE].mask = UINT64_MAX; + + for (i = 0; i < reta_size; i++) { + uint32_t reta_id = i / RTE_RETA_GROUP_SIZE; + uint32_t reta_pos = i % RTE_RETA_GROUP_SIZE; + uint32_t rss_qs_pos = i % rss->n_queues; + + reta_conf[reta_id].reta[reta_pos] = + (uint16_t) rss->queue_id[rss_qs_pos]; + } + + /* RETA update */ + status = rte_eth_dev_rss_reta_update(port_id, + reta_conf, + reta_size); + + return status; +} + +struct link * +link_create(struct obj *obj, const char *name, struct link_params *params) +{ + struct rte_eth_dev_info port_info; + struct rte_eth_conf port_conf; + struct link *link; + struct link_params_rss *rss; + struct mempool *mempool; + uint32_t cpu_id, i; + int status; + uint16_t port_id; + + /* Check input params */ + if ((name == NULL) || + link_find(obj, name) || + (params == NULL) || + (params->rx.n_queues == 0) || + (params->rx.queue_size == 0) || + (params->tx.n_queues == 0) || + (params->tx.queue_size == 0)) + return NULL; + + port_id = params->port_id; + if (params->dev_name) { + status = rte_eth_dev_get_port_by_name(params->dev_name, + &port_id); + + if (status) + return NULL; + } else + if (!rte_eth_dev_is_valid_port(port_id)) + return NULL; + + if (rte_eth_dev_info_get(port_id, &port_info) != 0) + return NULL; + + mempool = mempool_find(obj, params->rx.mempool_name); + if (mempool == NULL) + return NULL; + + rss = params->rx.rss; + if (rss) { + if ((port_info.reta_size == 0) || + (port_info.reta_size > ETH_RSS_RETA_SIZE_512)) + return NULL; + + if ((rss->n_queues == 0) || + (rss->n_queues >= LINK_RXQ_RSS_MAX)) + return NULL; + + for (i = 0; i < rss->n_queues; i++) + if (rss->queue_id[i] >= port_info.max_rx_queues) + return NULL; + } + + /** + * Resource create + */ + /* Port */ + memcpy(&port_conf, &port_conf_default, sizeof(port_conf)); + if (rss) { + port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port_conf.rx_adv_conf.rss_conf.rss_hf = + (ETH_RSS_IP | ETH_RSS_TCP | ETH_RSS_UDP) & + port_info.flow_type_rss_offloads; + } + + cpu_id = (uint32_t) rte_eth_dev_socket_id(port_id); + if (cpu_id == (uint32_t) SOCKET_ID_ANY) + cpu_id = 0; + + status = rte_eth_dev_configure( + port_id, + params->rx.n_queues, + params->tx.n_queues, + &port_conf); + + if (status < 0) + return NULL; + + if (params->promiscuous) { + status = rte_eth_promiscuous_enable(port_id); + if (status != 0) + return NULL; + } + + /* Port RX */ + for (i = 0; i < params->rx.n_queues; i++) { + status = rte_eth_rx_queue_setup( + port_id, + i, + params->rx.queue_size, + cpu_id, + NULL, + mempool->m); + + if (status < 0) + return NULL; + } + + /* Port TX */ + for (i = 0; i < params->tx.n_queues; i++) { + status = rte_eth_tx_queue_setup( + port_id, + i, + params->tx.queue_size, + cpu_id, + NULL); + + if (status < 0) + return NULL; + } + + /* Port start */ + status = rte_eth_dev_start(port_id); + if (status < 0) + return NULL; + + if (rss) { + status = rss_setup(port_id, port_info.reta_size, rss); + + if (status) { + rte_eth_dev_stop(port_id); + return NULL; + } + } + + /* Port link up */ + status = rte_eth_dev_set_link_up(port_id); + if ((status < 0) && (status != -ENOTSUP)) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node allocation */ + link = calloc(1, sizeof(struct link)); + if (link == NULL) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node fill in */ + strlcpy(link->name, name, sizeof(link->name)); + link->port_id = port_id; + rte_eth_dev_get_name_by_port(port_id, link->dev_name); + link->n_rxq = params->rx.n_queues; + link->n_txq = params->tx.n_queues; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->link_list, link, node); + + return link; +} + +int +link_is_up(struct obj *obj, const char *name) +{ + struct rte_eth_link link_params; + struct link *link; + + /* Check input params */ + if (!obj || !name) + return 0; + + link = link_find(obj, name); + if (link == NULL) + return 0; + + /* Resource */ + if (rte_eth_link_get(link->port_id, &link_params) < 0) + return 0; + + return (link_params.link_status == ETH_LINK_DOWN) ? 0 : 1; +} + +struct link * +link_find(struct obj *obj, const char *name) +{ + struct link *link; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(link, &obj->link_list, node) + if (strcmp(link->name, name) == 0) + return link; + + return NULL; +} + +struct link * +link_next(struct obj *obj, struct link *link) +{ + return (link == NULL) ? + TAILQ_FIRST(&obj->link_list) : TAILQ_NEXT(link, node); +} + +/* + * pipeline + */ +#ifndef PIPELINE_MSGQ_SIZE +#define PIPELINE_MSGQ_SIZE 64 +#endif + +struct pipeline * +pipeline_create(struct obj *obj, const char *name, int numa_node) +{ + struct pipeline *pipeline; + struct rte_swx_pipeline *p; + int status; + + /* Check input params */ + if ((name == NULL) || + pipeline_find(obj, name)) + return NULL; + + /* Resource create */ + status = rte_swx_pipeline_config(&p, numa_node); + if (status) + return NULL; + + status = rte_swx_pipeline_port_in_type_register(p, + "ethdev", + &rte_swx_port_ethdev_reader_ops); + if (status) { + rte_swx_pipeline_free(p); + return NULL; + } + + status = rte_swx_pipeline_port_out_type_register(p, + "ethdev", + &rte_swx_port_ethdev_writer_ops); + if (status) { + rte_swx_pipeline_free(p); + return NULL; + } + +#ifdef RTE_PORT_PCAP + + status = rte_swx_pipeline_port_in_type_register(p, + "source", + &rte_swx_port_source_ops); + if (status) { + rte_swx_pipeline_free(p); + return NULL; + } + + status = rte_swx_pipeline_port_out_type_register(p, + "sink", + &rte_swx_port_sink_ops); + if (status) { + rte_swx_pipeline_free(p); + return NULL; + } + +#endif + + /* Node allocation */ + pipeline = calloc(1, sizeof(struct pipeline)); + if (pipeline == NULL) { + rte_swx_pipeline_free(p); + return NULL; + } + + /* Node fill in */ + strlcpy(pipeline->name, name, sizeof(pipeline->name)); + pipeline->p = p; + pipeline->timer_period_ms = 10; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->pipeline_list, pipeline, node); + + return pipeline; +} + +struct pipeline * +pipeline_find(struct obj *obj, const char *name) +{ + struct pipeline *pipeline; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(pipeline, &obj->pipeline_list, node) + if (strcmp(name, pipeline->name) == 0) + return pipeline; + + return NULL; +} + +/* + * obj + */ +struct obj * +obj_init(void) +{ + struct obj *obj; + + obj = calloc(1, sizeof(struct obj)); + if (!obj) + return NULL; + + TAILQ_INIT(&obj->mempool_list); + TAILQ_INIT(&obj->link_list); + TAILQ_INIT(&obj->pipeline_list); + + return obj; +} diff --git a/examples/pipeline/obj.h b/examples/pipeline/obj.h new file mode 100644 index 000000000..2f48b790f --- /dev/null +++ b/examples/pipeline/obj.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_OBJ_H_ +#define _INCLUDE_OBJ_H_ + +#include +#include + +#include +#include +#include + +#ifndef NAME_SIZE +#define NAME_SIZE 64 +#endif + +/* + * obj + */ +struct obj; + +struct obj * +obj_init(void); + +/* + * mempool + */ +struct mempool_params { + uint32_t buffer_size; + uint32_t pool_size; + uint32_t cache_size; + uint32_t cpu_id; +}; + +struct mempool { + TAILQ_ENTRY(mempool) node; + char name[NAME_SIZE]; + struct rte_mempool *m; + uint32_t buffer_size; +}; + +struct mempool * +mempool_create(struct obj *obj, + const char *name, + struct mempool_params *params); + +struct mempool * +mempool_find(struct obj *obj, + const char *name); + +/* + * link + */ +#ifndef LINK_RXQ_RSS_MAX +#define LINK_RXQ_RSS_MAX 16 +#endif + +struct link_params_rss { + uint32_t queue_id[LINK_RXQ_RSS_MAX]; + uint32_t n_queues; +}; + +struct link_params { + const char *dev_name; + uint16_t port_id; /**< Valid only when *dev_name* is NULL. */ + + struct { + uint32_t n_queues; + uint32_t queue_size; + const char *mempool_name; + struct link_params_rss *rss; + } rx; + + struct { + uint32_t n_queues; + uint32_t queue_size; + } tx; + + int promiscuous; +}; + +struct link { + TAILQ_ENTRY(link) node; + char name[NAME_SIZE]; + char dev_name[NAME_SIZE]; + uint16_t port_id; + uint32_t n_rxq; + uint32_t n_txq; +}; + +struct link * +link_create(struct obj *obj, + const char *name, + struct link_params *params); + +int +link_is_up(struct obj *obj, const char *name); + +struct link * +link_find(struct obj *obj, const char *name); + +struct link * +link_next(struct obj *obj, struct link *link); + +/* + * pipeline + */ +struct pipeline { + TAILQ_ENTRY(pipeline) node; + char name[NAME_SIZE]; + + struct rte_swx_pipeline *p; + struct rte_swx_ctl_pipeline *ctl; + + uint32_t timer_period_ms; + int enabled; + uint32_t thread_id; + uint32_t cpu_id; +}; + +struct pipeline * +pipeline_create(struct obj *obj, + const char *name, + int numa_node); + +struct pipeline * +pipeline_find(struct obj *obj, const char *name); + +#endif /* _INCLUDE_OBJ_H_ */ diff --git a/examples/pipeline/thread.c b/examples/pipeline/thread.c new file mode 100644 index 000000000..0d1474c55 --- /dev/null +++ b/examples/pipeline/thread.c @@ -0,0 +1,549 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include + +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "obj.h" +#include "thread.h" + +#ifndef THREAD_PIPELINES_MAX +#define THREAD_PIPELINES_MAX 256 +#endif + +#ifndef THREAD_MSGQ_SIZE +#define THREAD_MSGQ_SIZE 64 +#endif + +#ifndef THREAD_TIMER_PERIOD_MS +#define THREAD_TIMER_PERIOD_MS 100 +#endif + +/** + * Master thead: data plane thread context + */ +struct thread { + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + + uint32_t enabled; +}; + +static struct thread thread[RTE_MAX_LCORE]; + +/** + * Data plane threads: context + */ +struct pipeline_data { + struct rte_swx_pipeline *p; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; +}; + +struct thread_data { + struct rte_swx_pipeline *p[THREAD_PIPELINES_MAX]; + uint32_t n_pipelines; + + struct pipeline_data pipeline_data[THREAD_PIPELINES_MAX]; + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; + uint64_t time_next_min; +} __rte_cache_aligned; + +static struct thread_data thread_data[RTE_MAX_LCORE]; + +/** + * Master thread: data plane thread init + */ +static void +thread_free(void) +{ + uint32_t i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct thread *t = &thread[i]; + + if (!rte_lcore_is_enabled(i)) + continue; + + /* MSGQs */ + if (t->msgq_req) + rte_ring_free(t->msgq_req); + + if (t->msgq_rsp) + rte_ring_free(t->msgq_rsp); + } +} + +int +thread_init(void) +{ + uint32_t i; + + RTE_LCORE_FOREACH_SLAVE(i) { + char name[NAME_MAX]; + struct rte_ring *msgq_req, *msgq_rsp; + struct thread *t = &thread[i]; + struct thread_data *t_data = &thread_data[i]; + uint32_t cpu_id = rte_lcore_to_socket_id(i); + + /* MSGQs */ + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-REQ", i); + + msgq_req = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_req == NULL) { + thread_free(); + return -1; + } + + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-RSP", i); + + msgq_rsp = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_rsp == NULL) { + thread_free(); + return -1; + } + + /* Master thread records */ + t->msgq_req = msgq_req; + t->msgq_rsp = msgq_rsp; + t->enabled = 1; + + /* Data plane thread records */ + t_data->n_pipelines = 0; + t_data->msgq_req = msgq_req; + t_data->msgq_rsp = msgq_rsp; + t_data->timer_period = + (rte_get_tsc_hz() * THREAD_TIMER_PERIOD_MS) / 1000; + t_data->time_next = rte_get_tsc_cycles() + t_data->timer_period; + t_data->time_next_min = t_data->time_next; + } + + return 0; +} + +static inline int +thread_is_running(uint32_t thread_id) +{ + enum rte_lcore_state_t thread_state; + + thread_state = rte_eal_get_lcore_state(thread_id); + return (thread_state == RUNNING) ? 1 : 0; +} + +/** + * Master thread & data plane threads: message passing + */ +enum thread_req_type { + THREAD_REQ_PIPELINE_ENABLE = 0, + THREAD_REQ_PIPELINE_DISABLE, + THREAD_REQ_MAX +}; + +struct thread_msg_req { + enum thread_req_type type; + + union { + struct { + struct rte_swx_pipeline *p; + uint32_t timer_period_ms; + } pipeline_enable; + + struct { + struct rte_swx_pipeline *p; + } pipeline_disable; + }; +}; + +struct thread_msg_rsp { + int status; +}; + +/** + * Master thread + */ +static struct thread_msg_req * +thread_msg_alloc(void) +{ + size_t size = RTE_MAX(sizeof(struct thread_msg_req), + sizeof(struct thread_msg_rsp)); + + return calloc(1, size); +} + +static void +thread_msg_free(struct thread_msg_rsp *rsp) +{ + free(rsp); +} + +static struct thread_msg_rsp * +thread_msg_send_recv(uint32_t thread_id, + struct thread_msg_req *req) +{ + struct thread *t = &thread[thread_id]; + struct rte_ring *msgq_req = t->msgq_req; + struct rte_ring *msgq_rsp = t->msgq_rsp; + struct thread_msg_rsp *rsp; + int status; + + /* send */ + do { + status = rte_ring_sp_enqueue(msgq_req, req); + } while (status == -ENOBUFS); + + /* recv */ + do { + status = rte_ring_sc_dequeue(msgq_rsp, (void **) &rsp); + } while (status != 0); + + return rsp; +} + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + struct pipeline_data *tdp = &td->pipeline_data[td->n_pipelines]; + + if (td->n_pipelines >= THREAD_PIPELINES_MAX) + return -1; + + /* Data plane thread */ + td->p[td->n_pipelines] = p->p; + + tdp->p = p->p; + tdp->timer_period = + (rte_get_tsc_hz() * p->timer_period_ms) / 1000; + tdp->time_next = rte_get_tsc_cycles() + tdp->timer_period; + + td->n_pipelines++; + + /* Pipeline */ + p->thread_id = thread_id; + p->enabled = 1; + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_ENABLE; + req->pipeline_enable.p = p->p; + req->pipeline_enable.timer_period_ms = p->timer_period_ms; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->thread_id = thread_id; + p->enabled = 1; + + return 0; +} + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (p->enabled == 0) + return 0; + + if (p->thread_id != thread_id) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + uint32_t i; + + for (i = 0; i < td->n_pipelines; i++) { + struct pipeline_data *tdp = &td->pipeline_data[i]; + + if (tdp->p != p->p) + continue; + + /* Data plane thread */ + if (i < td->n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + td->p[td->n_pipelines - 1]; + struct pipeline_data *tdp_last = + &td->pipeline_data[td->n_pipelines - 1]; + + td->p[i] = pipeline_last; + memcpy(tdp, tdp_last, sizeof(*tdp)); + } + + td->n_pipelines--; + + /* Pipeline */ + p->enabled = 0; + + break; + } + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_DISABLE; + req->pipeline_disable.p = p->p; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->enabled = 0; + + return 0; +} + +/** + * Data plane threads: message handling + */ +static inline struct thread_msg_req * +thread_msg_recv(struct rte_ring *msgq_req) +{ + struct thread_msg_req *req; + + int status = rte_ring_sc_dequeue(msgq_req, (void **) &req); + + if (status != 0) + return NULL; + + return req; +} + +static inline void +thread_msg_send(struct rte_ring *msgq_rsp, + struct thread_msg_rsp *rsp) +{ + int status; + + do { + status = rte_ring_sp_enqueue(msgq_rsp, rsp); + } while (status == -ENOBUFS); +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_enable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + struct pipeline_data *p = &t->pipeline_data[t->n_pipelines]; + + /* Request */ + if (t->n_pipelines >= THREAD_PIPELINES_MAX) { + rsp->status = -1; + return rsp; + } + + t->p[t->n_pipelines] = req->pipeline_enable.p; + + p->p = req->pipeline_enable.p; + p->timer_period = (rte_get_tsc_hz() * + req->pipeline_enable.timer_period_ms) / 1000; + p->time_next = rte_get_tsc_cycles() + p->timer_period; + + t->n_pipelines++; + + /* Response */ + rsp->status = 0; + return rsp; +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_disable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + uint32_t n_pipelines = t->n_pipelines; + struct rte_swx_pipeline *pipeline = req->pipeline_disable.p; + uint32_t i; + + /* find pipeline */ + for (i = 0; i < n_pipelines; i++) { + struct pipeline_data *p = &t->pipeline_data[i]; + + if (p->p != pipeline) + continue; + + if (i < n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + t->p[n_pipelines - 1]; + struct pipeline_data *p_last = + &t->pipeline_data[n_pipelines - 1]; + + t->p[i] = pipeline_last; + memcpy(p, p_last, sizeof(*p)); + } + + t->n_pipelines--; + + rsp->status = 0; + return rsp; + } + + /* should not get here */ + rsp->status = 0; + return rsp; +} + +static void +thread_msg_handle(struct thread_data *t) +{ + for ( ; ; ) { + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + + req = thread_msg_recv(t->msgq_req); + if (req == NULL) + break; + + switch (req->type) { + case THREAD_REQ_PIPELINE_ENABLE: + rsp = thread_msg_handle_pipeline_enable(t, req); + break; + + case THREAD_REQ_PIPELINE_DISABLE: + rsp = thread_msg_handle_pipeline_disable(t, req); + break; + + default: + rsp = (struct thread_msg_rsp *) req; + rsp->status = -1; + } + + thread_msg_send(t->msgq_rsp, rsp); + } +} + +/** + * Data plane threads: main + */ +int +thread_main(void *arg __rte_unused) +{ + struct thread_data *t; + uint32_t thread_id, i; + + thread_id = rte_lcore_id(); + t = &thread_data[thread_id]; + + /* Dispatch loop */ + for (i = 0; ; i++) { + uint32_t j; + + /* Data Plane */ + for (j = 0; j < t->n_pipelines; j++) + rte_swx_pipeline_run(t->p[j], 1000000); + + /* Control Plane */ + if ((i & 0xF) == 0) { + uint64_t time = rte_get_tsc_cycles(); + uint64_t time_next_min = UINT64_MAX; + + if (time < t->time_next_min) + continue; + + /* Thread message queues */ + { + uint64_t time_next = t->time_next; + + if (time_next <= time) { + thread_msg_handle(t); + time_next = time + t->timer_period; + t->time_next = time_next; + } + + if (time_next < time_next_min) + time_next_min = time_next; + } + + t->time_next_min = time_next_min; + } + } + + return 0; +} diff --git a/examples/pipeline/thread.h b/examples/pipeline/thread.h new file mode 100644 index 000000000..829d82cbd --- /dev/null +++ b/examples/pipeline/thread.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_THREAD_H_ +#define _INCLUDE_THREAD_H_ + +#include + +#include "obj.h" + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_init(void); + +int +thread_main(void *arg); + +#endif /* _INCLUDE_THREAD_H_ */ From patchwork Wed Aug 26 15:14:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76031 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A091A04B1; Wed, 26 Aug 2020 17:23:41 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F10B11C29F; Wed, 26 Aug 2020 17:15:53 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 0AE131C129 for ; Wed, 26 Aug 2020 17:15:26 +0200 (CEST) IronPort-SDR: aXG8dFmvaDmlhmYG0YVhe/Dq6q5p2szv01YlfaneAMpUPWN691ON4JLM8gSLIVxCKFZUEZ+Lhk 3xCaOxnE9ARQ== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879631" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879631" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:26 -0700 IronPort-SDR: Kl4QCesP7FhSYXmZ6j/IhWebJGGmYemscFl1CmH0LauaH2XlWUA7CLX90yson5PKkJ2FhOg+SD FJFwW4ETn6Pg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081480" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:25 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:41 +0100 Message-Id: <20200826151445.51500-37-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 36/40] examples/pipeline: add message passing mechanism X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add network-based connectivity mechanism for the application to allow for the exchange of configuration messages through the network as opposed to local CLI only. Signed-off-by: Cristian Dumitrescu --- examples/pipeline/Makefile | 1 + examples/pipeline/conn.c | 331 ++++++++++++++++++++++++++++++++++ examples/pipeline/conn.h | 50 +++++ examples/pipeline/main.c | 136 +++++++++++++- examples/pipeline/meson.build | 1 + 5 files changed, 517 insertions(+), 2 deletions(-) create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 3c85e9e40..eb1085f7c 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c diff --git a/examples/pipeline/conn.c b/examples/pipeline/conn.c new file mode 100644 index 000000000..eed87b8ea --- /dev/null +++ b/examples/pipeline/conn.c @@ -0,0 +1,331 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include +#include +#include +#include +#include + +#include + +#include +#include +#include +#include + +#include "conn.h" + +#define MSG_CMD_TOO_LONG "Command too long." + +struct conn { + char *welcome; + char *prompt; + char *buf; + char *msg_in; + char *msg_out; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + size_t msg_in_len; + int fd_server; + int fd_client_group; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p) +{ + struct sockaddr_in server_address; + struct conn *conn; + int fd_server, fd_client_group, status; + + memset(&server_address, 0, sizeof(server_address)); + + /* Check input arguments */ + if ((p == NULL) || + (p->welcome == NULL) || + (p->prompt == NULL) || + (p->addr == NULL) || + (p->buf_size == 0) || + (p->msg_in_len_max == 0) || + (p->msg_out_len_max == 0) || + (p->msg_handle == NULL)) + return NULL; + + status = inet_aton(p->addr, &server_address.sin_addr); + if (status == 0) + return NULL; + + /* Memory allocation */ + conn = calloc(1, sizeof(struct conn)); + if (conn == NULL) + return NULL; + + conn->welcome = calloc(1, CONN_WELCOME_LEN_MAX + 1); + conn->prompt = calloc(1, CONN_PROMPT_LEN_MAX + 1); + conn->buf = calloc(1, p->buf_size); + conn->msg_in = calloc(1, p->msg_in_len_max + 1); + conn->msg_out = calloc(1, p->msg_out_len_max + 1); + + if ((conn->welcome == NULL) || + (conn->prompt == NULL) || + (conn->buf == NULL) || + (conn->msg_in == NULL) || + (conn->msg_out == NULL)) { + conn_free(conn); + return NULL; + } + + /* Server socket */ + server_address.sin_family = AF_INET; + server_address.sin_port = htons(p->port); + + fd_server = socket(AF_INET, + SOCK_STREAM | SOCK_NONBLOCK, + 0); + if (fd_server == -1) { + conn_free(conn); + return NULL; + } + + status = bind(fd_server, + (struct sockaddr *) &server_address, + sizeof(server_address)); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + status = listen(fd_server, 16); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Client group */ + fd_client_group = epoll_create(1); + if (fd_client_group == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Fill in */ + strncpy(conn->welcome, p->welcome, CONN_WELCOME_LEN_MAX); + strncpy(conn->prompt, p->prompt, CONN_PROMPT_LEN_MAX); + conn->buf_size = p->buf_size; + conn->msg_in_len_max = p->msg_in_len_max; + conn->msg_out_len_max = p->msg_out_len_max; + conn->msg_in_len = 0; + conn->fd_server = fd_server; + conn->fd_client_group = fd_client_group; + conn->msg_handle = p->msg_handle; + conn->msg_handle_arg = p->msg_handle_arg; + + return conn; +} + +void +conn_free(struct conn *conn) +{ + if (conn == NULL) + return; + + if (conn->fd_client_group) + close(conn->fd_client_group); + + if (conn->fd_server) + close(conn->fd_server); + + free(conn->msg_out); + free(conn->msg_in); + free(conn->prompt); + free(conn->welcome); + free(conn); +} + +int +conn_poll_for_conn(struct conn *conn) +{ + struct sockaddr_in client_address; + struct epoll_event event; + socklen_t client_address_length; + int fd_client, status; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Server socket */ + client_address_length = sizeof(client_address); + fd_client = accept4(conn->fd_server, + (struct sockaddr *) &client_address, + &client_address_length, + SOCK_NONBLOCK); + if (fd_client == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + + /* Client group */ + event.events = EPOLLIN | EPOLLRDHUP | EPOLLHUP; + event.data.fd = fd_client; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_ADD, + fd_client, + &event); + if (status == -1) { + close(fd_client); + return -1; + } + + /* Client */ + status = write(fd_client, + conn->welcome, + strlen(conn->welcome)); + if (status == -1) { + close(fd_client); + return -1; + } + + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) { + close(fd_client); + return -1; + } + + return 0; +} + +static int +data_event_handle(struct conn *conn, + int fd_client) +{ + ssize_t len, i, status; + + /* Read input message */ + + len = read(fd_client, + conn->buf, + conn->buf_size); + if (len == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + if (len == 0) + return 0; + + /* Handle input messages */ + for (i = 0; i < len; i++) { + if (conn->buf[i] == '\n') { + size_t n; + + conn->msg_in[conn->msg_in_len] = 0; + conn->msg_out[0] = 0; + + conn->msg_handle(conn->msg_in, + conn->msg_out, + conn->msg_out_len_max, + conn->msg_handle_arg); + + n = strlen(conn->msg_out); + if (n) { + status = write(fd_client, + conn->msg_out, + n); + if (status == -1) + return status; + } + + conn->msg_in_len = 0; + } else if (conn->msg_in_len < conn->msg_in_len_max) { + conn->msg_in[conn->msg_in_len] = conn->buf[i]; + conn->msg_in_len++; + } else { + status = write(fd_client, + MSG_CMD_TOO_LONG, + strlen(MSG_CMD_TOO_LONG)); + if (status == -1) + return status; + + conn->msg_in_len = 0; + } + } + + /* Write prompt */ + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) + return status; + + return 0; +} + +static int +control_event_handle(struct conn *conn, + int fd_client) +{ + int status; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_DEL, + fd_client, + NULL); + if (status == -1) + return -1; + + status = close(fd_client); + if (status == -1) + return -1; + + return 0; +} + +int +conn_poll_for_msg(struct conn *conn) +{ + struct epoll_event event; + int fd_client, status, status_data = 0, status_control = 0; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Client group */ + status = epoll_wait(conn->fd_client_group, + &event, + 1, + 0); + if (status == -1) + return -1; + if (status == 0) + return 0; + + fd_client = event.data.fd; + + /* Data available */ + if (event.events & EPOLLIN) + status_data = data_event_handle(conn, fd_client); + + /* Control events */ + if (event.events & (EPOLLRDHUP | EPOLLERR | EPOLLHUP)) + status_control = control_event_handle(conn, fd_client); + + if (status_data || status_control) + return -1; + + return 0; +} diff --git a/examples/pipeline/conn.h b/examples/pipeline/conn.h new file mode 100644 index 000000000..871a5efd0 --- /dev/null +++ b/examples/pipeline/conn.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CONN_H__ +#define __INCLUDE_CONN_H__ + +#include + +struct conn; + +#ifndef CONN_WELCOME_LEN_MAX +#define CONN_WELCOME_LEN_MAX 1024 +#endif + +#ifndef CONN_PROMPT_LEN_MAX +#define CONN_PROMPT_LEN_MAX 16 +#endif + +typedef void +(*conn_msg_handle_t)(char *msg_in, + char *msg_out, + size_t msg_out_len_max, + void *arg); + +struct conn_params { + const char *welcome; + const char *prompt; + const char *addr; + uint16_t port; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p); + +void +conn_free(struct conn *conn); + +int +conn_poll_for_conn(struct conn *conn); + +int +conn_poll_for_msg(struct conn *conn); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index 353e62c10..1a63c1237 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,15 +11,136 @@ #include #include +#include "conn.h" #include "obj.h" #include "thread.h" +static const char usage[] = + "%s EAL_ARGS -- [-h HOST] [-p PORT] [-s SCRIPT]\n"; + +static struct app_params { + struct conn_params conn; + char *script_name; +} app = { + .conn = { + .welcome = "\nWelcome!\n\n", + .prompt = "pipeline> ", + .addr = "0.0.0.0", + .port = 8086, + .buf_size = 1024 * 1024, + .msg_in_len_max = 1024, + .msg_out_len_max = 1024 * 1024, + .msg_handle = NULL, + .msg_handle_arg = NULL, /* set later. */ + }, + .script_name = NULL, +}; + +static int +parse_args(int argc, char **argv) +{ + char *app_name = argv[0]; + struct option lgopts[] = { + { NULL, 0, 0, 0 } + }; + int opt, option_index; + int h_present, p_present, s_present, n_args, i; + + /* Skip EAL input args */ + n_args = argc; + for (i = 0; i < n_args; i++) + if (strcmp(argv[i], "--") == 0) { + argc -= i; + argv += i; + break; + } + + if (i == n_args) + return 0; + + /* Parse args */ + h_present = 0; + p_present = 0; + s_present = 0; + + while ((opt = getopt_long(argc, argv, "h:p:s:", lgopts, &option_index)) + != EOF) + switch (opt) { + case 'h': + if (h_present) { + printf("Error: Multiple -h arguments\n"); + return -1; + } + h_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -h not provided\n"); + return -1; + } + + app.conn.addr = strdup(optarg); + if (app.conn.addr == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + case 'p': + if (p_present) { + printf("Error: Multiple -p arguments\n"); + return -1; + } + p_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -p not provided\n"); + return -1; + } + + app.conn.port = (uint16_t) atoi(optarg); + break; + + case 's': + if (s_present) { + printf("Error: Multiple -s arguments\n"); + return -1; + } + s_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -s not provided\n"); + return -1; + } + + app.script_name = strdup(optarg); + if (app.script_name == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + default: + printf(usage, app_name); + return -1; + } + + optind = 1; /* reset getopt lib */ + + return 0; +} + int main(int argc, char **argv) { + struct conn *conn; struct obj *obj; int status; + /* Parse application arguments */ + status = parse_args(argc, argv); + if (status < 0) + return status; + /* EAL */ status = rte_eal_init(argc, argv); if (status < 0) { @@ -46,7 +167,18 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Connectivity */ + app.conn.msg_handle_arg = obj; + conn = conn_init(&app.conn); + if (!conn) { + printf("Error: Connectivity initialization failed (%d)\n", + status); + return status; + }; /* Dispatch loop */ - for ( ; ; ); -} + for ( ; ; ) { + conn_poll_for_conn(conn); + conn_poll_for_msg(conn); + } +} diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index ade485f97..a92e84677 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'conn.c', 'main.c', 'obj.c', 'thread.c', From patchwork Wed Aug 26 15:14:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76032 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7ECB7A04B1; Wed, 26 Aug 2020 17:23:55 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0641F1C2A9; Wed, 26 Aug 2020 17:15:55 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 1D1011C132 for ; Wed, 26 Aug 2020 17:15:28 +0200 (CEST) IronPort-SDR: RyiHpLnTaWLZEbX37Yx5at9A8eGjltOsuS1YnwKBOLvS3PcnZ+KLvFoA5b+0TmuMLtQpEmjd5b XvZpm8vQvR1A== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879637" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879637" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:27 -0700 IronPort-SDR: nONWwfJMfS1xOj5koabzHYvrdn2xSRy/zBqQU0EMAffTTXHDve3KMeU2sOu1C9H+/UeJ1OCNmf /vyk1vWTirNg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081491" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:26 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:42 +0100 Message-Id: <20200826151445.51500-38-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 37/40] examples/pipeline: add configuration commands X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add CLI commands for application configuration and query. Signed-off-by: Cristian Dumitrescu --- examples/pipeline/Makefile | 1 + examples/pipeline/cli.c | 1373 +++++++++++++++++++++++++++++++++ examples/pipeline/cli.h | 19 + examples/pipeline/main.c | 11 +- examples/pipeline/meson.build | 1 + 5 files changed, 1404 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index eb1085f7c..dcc0f67bf 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += cli.c SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c new file mode 100644 index 000000000..f68a1a924 --- /dev/null +++ b/examples/pipeline/cli.c @@ -0,0 +1,1373 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#include "cli.h" + +#include "obj.h" +#include "thread.h" + +#ifndef CMD_MAX_TOKENS +#define CMD_MAX_TOKENS 256 +#endif + +#define MSG_OUT_OF_MEMORY "Not enough memory.\n" +#define MSG_CMD_UNKNOWN "Unknown command \"%s\".\n" +#define MSG_CMD_UNIMPLEM "Command \"%s\" not implemented.\n" +#define MSG_ARG_NOT_ENOUGH "Not enough arguments for command \"%s\".\n" +#define MSG_ARG_TOO_MANY "Too many arguments for command \"%s\".\n" +#define MSG_ARG_MISMATCH "Wrong number of arguments for command \"%s\".\n" +#define MSG_ARG_NOT_FOUND "Argument \"%s\" not found.\n" +#define MSG_ARG_INVALID "Invalid value for argument \"%s\".\n" +#define MSG_FILE_ERR "Error in file \"%s\" at line %u.\n" +#define MSG_FILE_NOT_ENOUGH "Not enough rules in file \"%s\".\n" +#define MSG_CMD_FAIL "Command \"%s\" failed.\n" + +#define skip_white_spaces(pos) \ +({ \ + __typeof__(pos) _p = (pos); \ + for ( ; isspace(*_p); _p++) \ + ; \ + _p; \ +}) + +static int +parser_read_uint64(uint64_t *value, const char *p) +{ + char *next; + uint64_t val; + + p = skip_white_spaces(p); + if (!isdigit(*p)) + return -EINVAL; + + val = strtoul(p, &next, 10); + if (p == next) + return -EINVAL; + + p = next; + switch (*p) { + case 'T': + val *= 1024ULL; + /* fall through */ + case 'G': + val *= 1024ULL; + /* fall through */ + case 'M': + val *= 1024ULL; + /* fall through */ + case 'k': + case 'K': + val *= 1024ULL; + p++; + break; + } + + p = skip_white_spaces(p); + if (*p != '\0') + return -EINVAL; + + *value = val; + return 0; +} + +static int +parser_read_uint32(uint32_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT32_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +static int +parser_read_uint16(uint16_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT16_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +#define PARSE_DELIMITER " \f\n\r\t\v" + +static int +parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens) +{ + uint32_t i; + + if ((string == NULL) || + (tokens == NULL) || + (*n_tokens < 1)) + return -EINVAL; + + for (i = 0; i < *n_tokens; i++) { + tokens[i] = strtok_r(string, PARSE_DELIMITER, &string); + if (tokens[i] == NULL) + break; + } + + if ((i == *n_tokens) && strtok_r(string, PARSE_DELIMITER, &string)) + return -E2BIG; + + *n_tokens = i; + return 0; +} + +static int +is_comment(char *in) +{ + if ((strlen(in) && index("!#%;", in[0])) || + (strncmp(in, "//", 2) == 0) || + (strncmp(in, "--", 2) == 0)) + return 1; + + return 0; +} + +static const char cmd_mempool_help[] = +"mempool \n" +" buffer \n" +" pool \n" +" cache \n" +" cpu \n"; + +static void +cmd_mempool(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct mempool_params p; + char *name; + struct mempool *mempool; + + if (n_tokens != 10) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (strcmp(tokens[2], "buffer") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "buffer"); + return; + } + + if (parser_read_uint32(&p.buffer_size, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "buffer_size"); + return; + } + + if (strcmp(tokens[4], "pool") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pool"); + return; + } + + if (parser_read_uint32(&p.pool_size, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "pool_size"); + return; + } + + if (strcmp(tokens[6], "cache") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cache"); + return; + } + + if (parser_read_uint32(&p.cache_size, tokens[7]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cache_size"); + return; + } + + if (strcmp(tokens[8], "cpu") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cpu"); + return; + } + + if (parser_read_uint32(&p.cpu_id, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cpu_id"); + return; + } + + mempool = mempool_create(obj, name, &p); + if (mempool == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +static const char cmd_link_help[] = +"link \n" +" dev | port \n" +" rxq \n" +" txq \n" +" promiscuous on | off\n" +" [rss ... ]\n"; + +static void +cmd_link(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link_params p; + struct link_params_rss rss; + struct link *link; + char *name; + + memset(&p, 0, sizeof(p)); + + if ((n_tokens < 13) || (n_tokens > 14 + LINK_RXQ_RSS_MAX)) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + name = tokens[1]; + + if (strcmp(tokens[2], "dev") == 0) + p.dev_name = tokens[3]; + else if (strcmp(tokens[2], "port") == 0) { + p.dev_name = NULL; + + if (parser_read_uint16(&p.port_id, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + } else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "dev or port"); + return; + } + + if (strcmp(tokens[4], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint32(&p.rx.n_queues, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + if (parser_read_uint32(&p.rx.queue_size, tokens[6]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + p.rx.mempool_name = tokens[7]; + + if (strcmp(tokens[8], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint32(&p.tx.n_queues, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + + if (parser_read_uint32(&p.tx.queue_size, tokens[10]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + if (strcmp(tokens[11], "promiscuous") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "promiscuous"); + return; + } + + if (strcmp(tokens[12], "on") == 0) + p.promiscuous = 1; + else if (strcmp(tokens[12], "off") == 0) + p.promiscuous = 0; + else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "on or off"); + return; + } + + /* RSS */ + p.rx.rss = NULL; + if (n_tokens > 13) { + uint32_t queue_id, i; + + if (strcmp(tokens[13], "rss") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rss"); + return; + } + + p.rx.rss = &rss; + + rss.n_queues = 0; + for (i = 14; i < n_tokens; i++) { + if (parser_read_uint32(&queue_id, tokens[i]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + rss.queue_id[rss.n_queues] = queue_id; + rss.n_queues++; + } + } + + link = link_create(obj, name, &p); + if (link == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +/* Print the link stats and info */ +static void +print_link_info(struct link *link, char *out, size_t out_size) +{ + struct rte_eth_stats stats; + struct rte_ether_addr mac_addr; + struct rte_eth_link eth_link; + uint16_t mtu; + int ret; + + memset(&stats, 0, sizeof(stats)); + rte_eth_stats_get(link->port_id, &stats); + + ret = rte_eth_macaddr_get(link->port_id, &mac_addr); + if (ret != 0) { + snprintf(out, out_size, "\n%s: MAC address get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + ret = rte_eth_link_get(link->port_id, ð_link); + if (ret < 0) { + snprintf(out, out_size, "\n%s: link get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + rte_eth_dev_get_mtu(link->port_id, &mtu); + + snprintf(out, out_size, + "\n" + "%s: flags=<%s> mtu %u\n" + "\tether %02X:%02X:%02X:%02X:%02X:%02X rxqueues %u txqueues %u\n" + "\tport# %u speed %u Mbps\n" + "\tRX packets %" PRIu64" bytes %" PRIu64"\n" + "\tRX errors %" PRIu64" missed %" PRIu64" no-mbuf %" PRIu64"\n" + "\tTX packets %" PRIu64" bytes %" PRIu64"\n" + "\tTX errors %" PRIu64"\n", + link->name, + eth_link.link_status == 0 ? "DOWN" : "UP", + mtu, + mac_addr.addr_bytes[0], mac_addr.addr_bytes[1], + mac_addr.addr_bytes[2], mac_addr.addr_bytes[3], + mac_addr.addr_bytes[4], mac_addr.addr_bytes[5], + link->n_rxq, + link->n_txq, + link->port_id, + eth_link.link_speed, + stats.ipackets, + stats.ibytes, + stats.ierrors, + stats.imissed, + stats.rx_nombuf, + stats.opackets, + stats.obytes, + stats.oerrors); +} + +/* + * link show [] + */ +static void +cmd_link_show(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link *link; + char *link_name; + + if (n_tokens != 2 && n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (n_tokens == 2) { + link = link_next(obj, NULL); + + while (link != NULL) { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + print_link_info(link, out, out_size); + link = link_next(obj, link); + } + } else { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + link_name = tokens[2]; + link = link_find(obj, link_name); + + if (link == NULL) { + snprintf(out, out_size, MSG_ARG_INVALID, + "Link does not exist"); + return; + } + print_link_info(link, out, out_size); + } +} + +static const char cmd_pipeline_create_help[] = +"pipeline create \n"; + +static void +cmd_pipeline_create(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *name; + uint32_t numa_node; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (parser_read_uint32(&numa_node, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "numa_node"); + return; + } + + p = pipeline_create(obj, name, (int)numa_node); + if (!p) { + snprintf(out, out_size, "pipeline create error."); + return; + } +} + +static const char cmd_pipeline_port_in_help[] = +"pipeline port in \n" +" link rxq bsz \n" +" source \n"; + +static void +cmd_pipeline_port_in(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "in") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "in"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_reader_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "source") == 0) { + struct rte_swx_port_source_params params; + struct mempool *mp; + + if (n_tokens < t0 + 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in source"); + return; + } + + mp = mempool_find(obj, tokens[t0 + 1]); + if (!mp) { + snprintf(out, out_size, MSG_ARG_INVALID, + "mempool_name"); + return; + } + params.pool = mp->m; + + params.file_name = tokens[t0 + 2]; + + t0 += 3; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "source", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port in error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_port_out_help[] = +"pipeline port out \n" +" link txq bsz \n" +" | sink | none\n"; + +static void +cmd_pipeline_port_out(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "out") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "out"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_writer_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port out link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "sink") == 0) { + struct rte_swx_port_sink_params params; + + params.file_name = strcmp(tokens[t0 + 1], "none") ? + tokens[t0 + 1] : NULL; + + t0 += 2; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "sink", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port out error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_build_help[] = +"pipeline build \n"; + +static void +cmd_pipeline_build(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *name; + int status; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + p = pipeline_find(obj, name); + if (!p) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + status = rte_swx_pipeline_build(p->p); + if (status) { + snprintf(out, out_size, "Pipeline build failed."); + return; + } + + p->ctl = rte_swx_ctl_pipeline_create(p->p); + if (!p->ctl) { + snprintf(out, out_size, "Pipeline control create failed."); + rte_swx_pipeline_free(p->p); + return; + } +} + +static const char cmd_pipeline_table_update_help[] = +"pipeline table update " +" "; + +static void +cmd_pipeline_table_update(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name, *table_name, *line = NULL; + char *file_name_add, *file_name_delete, *file_name_default; + FILE *file_add = NULL, *file_delete = NULL, *file_default = NULL; + uint32_t line_id; + int status; + + if (n_tokens != 8) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + pipeline_name = tokens[1]; + p = pipeline_find(obj, pipeline_name); + if (!p) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "table") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "table"); + return; + } + + table_name = tokens[3]; + + if (strcmp(tokens[4], "update") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "update"); + return; + } + + file_name_add = tokens[5]; + file_name_delete = tokens[6]; + file_name_default = tokens[7]; + + /* File open. */ + if (strcmp(file_name_add, "none")) { + file_add = fopen(file_name_add, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_add); + goto error; + } + } + + if (strcmp(file_name_delete, "none")) { + file_add = fopen(file_name_delete, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_delete); + goto error; + } + } + + if (strcmp(file_name_default, "none")) { + file_add = fopen(file_name_default, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_default); + goto error; + } + } + + if (!file_add && !file_delete && !file_default) { + snprintf(out, out_size, "Nothing to be done."); + return; + } + + /* Buffer allocation. */ + line = malloc(2048); + if (!line) { + snprintf(out, out_size, MSG_OUT_OF_MEMORY); + goto error; + } + + /* Add. */ + if (file_add) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_add) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_add, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_add, line_id); + goto error; + } + } + + fclose(file_add); + } + + /* Delete. */ + if (file_delete) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_delete) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_delete, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_delete(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_delete, line_id); + goto error; + } + } + + fclose(file_delete); + } + + /* Default. */ + if (file_default) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_default) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_default, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_default_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_default, line_id); + goto error; + } + } + + fclose(file_default); + } + + status = rte_swx_ctl_pipeline_commit(p->ctl, 1); + if (status) { + snprintf(out, out_size, "Commit failed."); + goto error; + } + + free(line); + + rte_swx_ctl_pipeline_table_fprintf(stdout, p->ctl, table_name); + + return; + +error: + rte_swx_ctl_pipeline_abort(p->ctl); + free(line); + if (file_add) + fclose(file_add); + if (file_delete) + fclose(file_delete); + if (file_default) + fclose(file_default); +} + +static const char cmd_pipeline_stats_help[] = +"pipeline stats\n"; + +static void +cmd_pipeline_stats(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct rte_swx_ctl_pipeline_info info; + struct pipeline *p; + uint32_t i; + int status; + + if (n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "stats")) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "stats"); + return; + } + + status = rte_swx_ctl_pipeline_info_get(p->p, &info); + if (status) { + snprintf(out, out_size, "Pipeline info get error."); + return; + } + + snprintf(out, out_size, "Input ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_in; i++) { + struct rte_swx_port_in_stats stats; + + rte_swx_ctl_pipeline_port_in_stats_read(p->p, i, &stats); + + snprintf(out, out_size, + "\tPort %u: packets %lu bytes %lu empty %lu\n", + i, stats.n_pkts, stats.n_bytes, stats.n_empty); + out_size -= strlen(out); + out += strlen(out); + } + + snprintf(out, out_size, "Output ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_out; i++) { + struct rte_swx_port_out_stats stats; + + rte_swx_ctl_pipeline_port_out_stats_read(p->p, i, &stats); + + snprintf(out, out_size, + "\tPort %u: packets %lu bytes %lu\n", + i, stats.n_pkts, stats.n_bytes); + out_size -= strlen(out); + out += strlen(out); + } +} + +static const char cmd_thread_pipeline_enable_help[] = +"thread pipeline enable\n"; + +static void +cmd_thread_pipeline_enable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + char *pipeline_name; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + + if (strcmp(tokens[4], "enable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "enable"); + return; + } + + status = thread_pipeline_enable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, "thread pipeline enable"); + return; + } +} + +static const char cmd_thread_pipeline_disable_help[] = +"thread pipeline disable\n"; + +static void +cmd_thread_pipeline_disable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + char *pipeline_name; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + + if (strcmp(tokens[4], "disable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "disable"); + return; + } + + status = thread_pipeline_disable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, + "thread pipeline disable"); + return; + } +} + +static void +cmd_help(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *arg __rte_unused) +{ + tokens++; + n_tokens--; + + if (n_tokens == 0) { + snprintf(out, out_size, + "Type 'help ' for command details.\n\n"); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_mempool_help); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_link_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens == 1) && (strcmp(tokens[2], "create")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_create_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + (strcmp(tokens[1], "port") == 0)) { + if ((n_tokens == 3) && (strcmp(tokens[2], "in")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_in_help); + return; + } + + if ((n_tokens == 3) && (strcmp(tokens[2], "out")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_out_help); + return; + } + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "build")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_build_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "table")) == 0)) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_table_update_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "stats")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_stats_help); + return; + } + + if ((n_tokens == 3) && + (strcmp(tokens[0], "thread") == 0) && + (strcmp(tokens[1], "pipeline") == 0)) { + if (strcmp(tokens[2], "enable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_enable_help); + return; + } + + if (strcmp(tokens[2], "disable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_disable_help); + return; + } + } + + snprintf(out, out_size, "Invalid command\n"); +} + +void +cli_process(char *in, char *out, size_t out_size, void *obj) +{ + char *tokens[CMD_MAX_TOKENS]; + uint32_t n_tokens = RTE_DIM(tokens); + int status; + + if (is_comment(in)) + return; + + status = parse_tokenize_string(in, tokens, &n_tokens); + if (status) { + snprintf(out, out_size, MSG_ARG_TOO_MANY, ""); + return; + } + + if (n_tokens == 0) + return; + + if (strcmp(tokens[0], "help") == 0) { + cmd_help(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + cmd_mempool(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + if (strcmp(tokens[1], "show") == 0) { + cmd_link_show(tokens, n_tokens, out, out_size, obj); + return; + } + + cmd_link(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "pipeline") == 0) { + if ((n_tokens >= 3) && + (strcmp(tokens[2], "create") == 0)) { + cmd_pipeline_create(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "in") == 0)) { + cmd_pipeline_port_in(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "out") == 0)) { + cmd_pipeline_port_out(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "build") == 0)) { + cmd_pipeline_build(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "table") == 0)) { + cmd_pipeline_table_update(tokens, n_tokens, out, + out_size, obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "stats") == 0)) { + cmd_pipeline_stats(tokens, n_tokens, out, out_size, + obj); + return; + } + } + + if (strcmp(tokens[0], "thread") == 0) { + if ((n_tokens >= 5) && + (strcmp(tokens[4], "enable") == 0)) { + cmd_thread_pipeline_enable(tokens, n_tokens, + out, out_size, obj); + return; + } + + if ((n_tokens >= 5) && + (strcmp(tokens[4], "disable") == 0)) { + cmd_thread_pipeline_disable(tokens, n_tokens, + out, out_size, obj); + return; + } + } + + snprintf(out, out_size, MSG_CMD_UNKNOWN, tokens[0]); +} + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *obj) +{ + char *msg_in = NULL, *msg_out = NULL; + FILE *f = NULL; + + /* Check input arguments */ + if ((file_name == NULL) || + (strlen(file_name) == 0) || + (msg_in_len_max == 0) || + (msg_out_len_max == 0)) + return -EINVAL; + + msg_in = malloc(msg_in_len_max + 1); + msg_out = malloc(msg_out_len_max + 1); + if ((msg_in == NULL) || + (msg_out == NULL)) { + free(msg_out); + free(msg_in); + return -ENOMEM; + } + + /* Open input file */ + f = fopen(file_name, "r"); + if (f == NULL) { + free(msg_out); + free(msg_in); + return -EIO; + } + + /* Read file */ + for ( ; ; ) { + if (fgets(msg_in, msg_in_len_max + 1, f) == NULL) + break; + + printf("%s", msg_in); + msg_out[0] = 0; + + cli_process(msg_in, + msg_out, + msg_out_len_max, + obj); + + if (strlen(msg_out)) + printf("%s", msg_out); + } + + /* Close file */ + fclose(f); + free(msg_out); + free(msg_in); + return 0; +} diff --git a/examples/pipeline/cli.h b/examples/pipeline/cli.h new file mode 100644 index 000000000..dad7233fe --- /dev/null +++ b/examples/pipeline/cli.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CLI_H__ +#define __INCLUDE_CLI_H__ + +#include + +void +cli_process(char *in, char *out, size_t out_size, void *arg); + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *arg); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index 1a63c1237..97bb66288 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,6 +11,7 @@ #include #include +#include "cli.h" #include "conn.h" #include "obj.h" #include "thread.h" @@ -30,7 +31,7 @@ static struct app_params { .buf_size = 1024 * 1024, .msg_in_len_max = 1024, .msg_out_len_max = 1024 * 1024, - .msg_handle = NULL, + .msg_handle = cli_process, .msg_handle_arg = NULL, /* set later. */ }, .script_name = NULL, @@ -167,6 +168,13 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Script */ + if (app.script_name) + cli_script_process(app.script_name, + app.conn.msg_in_len_max, + app.conn.msg_out_len_max, + obj); + /* Connectivity */ app.conn.msg_handle_arg = obj; conn = conn_init(&app.conn); @@ -175,6 +183,7 @@ main(int argc, char **argv) status); return status; }; + /* Dispatch loop */ for ( ; ; ) { conn_poll_for_conn(conn); diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index a92e84677..4f47dec3e 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'cli.c', 'conn.c', 'main.c', 'obj.c', From patchwork Wed Aug 26 15:14:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76033 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D4029A04B1; Wed, 26 Aug 2020 17:24:10 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 244551C2B2; Wed, 26 Aug 2020 17:15:56 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 386441C191 for ; Wed, 26 Aug 2020 17:15:30 +0200 (CEST) IronPort-SDR: nOOZLQ56eOcTzWr9vMbQ+QNNqHynKJ3xwQlMP50iVpimKBKjrBhas+QfmmA9sIb+wIuL/dyLjI 6oFI5gkssbBg== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879640" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879640" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:29 -0700 IronPort-SDR: VT3tSjYx3humTXs0RVO9shwgvTi89Nmr9rk4NsnpyBQX9tbsA/wsEkSW/TR1JcAxUzeFBUP+ba uuIg8KNT1O5g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081493" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:28 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:43 +0100 Message-Id: <20200826151445.51500-39-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 38/40] examples/pipeline: add l2fwd example X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add L2 Forwarding example to the pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd.cli Signed-off-by: Cristian Dumitrescu --- examples/pipeline/Makefile | 1 + examples/pipeline/cli.c | 16 ++- examples/pipeline/example_l2fwd.c | 125 ++++++++++++++++++++++ examples/pipeline/examples/l2fwd.cli | 25 +++++ examples/pipeline/examples/l2fwd_pcap.cli | 20 ++++ examples/pipeline/examples/packet.txt | 102 ++++++++++++++++++ examples/pipeline/meson.build | 1 + 7 files changed, 289 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/example_l2fwd.c create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index dcc0f67bf..405978ced 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -10,6 +10,7 @@ SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c +SRCS-y += example_l2fwd.c # Build using pkg-config variables if possible ifeq ($(shell pkg-config --exists libdpdk && echo 0),0) diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c index f68a1a924..a54cd79ff 100644 --- a/examples/pipeline/cli.c +++ b/examples/pipeline/cli.c @@ -730,6 +730,7 @@ cmd_pipeline_port_out(char **tokens, static const char cmd_pipeline_build_help[] = "pipeline build \n"; +int pipeline_setup_l2fwd(struct rte_swx_pipeline *p); static void cmd_pipeline_build(char **tokens, uint32_t n_tokens, @@ -738,7 +739,7 @@ cmd_pipeline_build(char **tokens, void *obj) { struct pipeline *p; - char *name; + char *name, *app; int status; if (n_tokens != 4) { @@ -753,6 +754,19 @@ cmd_pipeline_build(char **tokens, return; } + app = tokens[3]; + if (!strcmp(app, "l2fwd")) + status = pipeline_setup_l2fwd(p->p); + else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "Pipeline build: app setup error."); + return; + } + status = rte_swx_pipeline_build(p->p); if (status) { snprintf(out, out_size, "Pipeline build failed."); diff --git a/examples/pipeline/example_l2fwd.c b/examples/pipeline/example_l2fwd.c new file mode 100644 index 000000000..a0b627709 --- /dev/null +++ b/examples/pipeline/example_l2fwd.c @@ -0,0 +1,125 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include +#include +#include + +#include + +#include "rte_swx_pipeline.h" +#include "rte_swx_port_ethdev.h" +#include "rte_swx_port_source_sink.h" +#include "rte_swx_table_em.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) { \ + printf("Error in function %s at line %d\n", \ + __FUNCTION__, __LINE__); \ + return -1; \ + } \ +} while (0) + +/* + * Packet headers. + */ + +/* + * Packet meta-data. + */ +static struct rte_swx_field_params metadata_type[] = { + {"port_in", 32}, + {"port_out", 32}, +}; + +/* + * Actions. + */ +static const char *action_passthrough_instructions[] = { + "return", +}; + +/* + * Tables. + */ +static const char *table_stub_actions[] = {"passthrough"}; + +static struct rte_swx_pipeline_table_params table_stub_params = { + /* Match. */ + .fields = NULL, + .n_fields = 0, + + /* Action. */ + .action_names = table_stub_actions, + .n_actions = RTE_DIM(table_stub_actions), + .default_action_name = "passthrough", + .default_action_data = NULL, + .default_action_is_const = 0, +}; + +/* + * Pipeline. + */ +static const char *pipeline_instructions[] = { + "rx m.port_in", + "table stub", + "tx m.port_in", +}; + +int +pipeline_setup_l2fwd(struct rte_swx_pipeline *p); + +int +pipeline_setup_l2fwd(struct rte_swx_pipeline *p) +{ + int err; + + /* + * Packet headers. + */ + + /* + * Packet meta-data. + */ + err = rte_swx_pipeline_struct_type_register(p, + "metadata_type", + metadata_type, + RTE_DIM(metadata_type)); + CHECK(!err); + + err = rte_swx_pipeline_packet_metadata_register(p, + "metadata_type"); + CHECK(!err); + + /* + * Actions. + */ + err = rte_swx_pipeline_action_config(p, + "passthrough", + NULL, + action_passthrough_instructions, + RTE_DIM(action_passthrough_instructions)); + CHECK(!err); + + /* + * Tables. + */ + err = rte_swx_pipeline_table_config(p, + "stub", + &table_stub_params, + NULL, + NULL, + 0); + CHECK(!err); + + /* + * Pipeline. + */ + err = rte_swx_pipeline_instructions_config(p, + pipeline_instructions, + RTE_DIM(pipeline_instructions)); + CHECK(!err); + + return 0; +} diff --git a/examples/pipeline/examples/l2fwd.cli b/examples/pipeline/examples/l2fwd.cli new file mode 100644 index 000000000..15f8b1782 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build l2fwd + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd_pcap.cli b/examples/pipeline/examples/l2fwd_pcap.cli new file mode 100644 index 000000000..f9c37ab7e --- /dev/null +++ b/examples/pipeline/examples/l2fwd_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build l2fwd + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/packet.txt b/examples/pipeline/examples/packet.txt new file mode 100644 index 000000000..d1c79b7e7 --- /dev/null +++ b/examples/pipeline/examples/packet.txt @@ -0,0 +1,102 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +#Text to PCAP: text2pcap packet.txt packet.pcap +#PCAP to text: tcpdump -r packet.pcap -xx + +#Packet 0 +000000 aa bb cc dd 00 00 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 1 +000000 aa bb cc dd 00 01 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 2 +000000 aa bb cc dd 00 02 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 3 +000000 aa bb cc dd 00 03 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 4 +000000 aa bb cc dd 00 04 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 5 +000000 aa bb cc dd 00 05 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 6 +000000 aa bb cc dd 00 06 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 7 +000000 aa bb cc dd 00 07 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 8 +000000 aa bb cc dd 00 08 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 9 +000000 aa bb cc dd 00 09 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 10 +000000 aa bb cc dd 00 0a b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 11 +000000 aa bb cc dd 00 0b b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 12 +000000 aa bb cc dd 00 0c b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 13 +000000 aa bb cc dd 00 0d b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 14 +000000 aa bb cc dd 00 0e b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 15 +000000 aa bb cc dd 00 0f b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index 4f47dec3e..b16de3714 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -15,4 +15,5 @@ sources = files( 'main.c', 'obj.c', 'thread.c', + 'example_l2fwd.c', ) From patchwork Wed Aug 26 15:14:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76034 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 63963A04B1; Wed, 26 Aug 2020 17:24:25 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 338C91C2DD; Wed, 26 Aug 2020 17:15:57 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 303031C195 for ; Wed, 26 Aug 2020 17:15:31 +0200 (CEST) IronPort-SDR: eKB/TzKBCBZNqujk7ghOJu2Ijjm8ML8gnbKgxcddPDbsPcnGJVL+Vzj9S8R+LuyR1aTTjvOzoq IrX1oAOY3L8Q== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879643" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879643" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:29 -0700 IronPort-SDR: 2hlpZI+zgsDdwHOL6ia8YMatEnlitT9W/9Ldy/Y+khfg7W/9ApTwWs65/FJJ0SF7HhmTbfEjTA RTtEF0qPjuuw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081505" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:29 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:44 +0100 Message-Id: <20200826151445.51500-40-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 39/40] examples/pipeline: add l2fwd with MAC swap example X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add L2 Forwarding example with MAC destination and source address swap to the pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd_macswp.cli Signed-off-by: Cristian Dumitrescu --- examples/pipeline/Makefile | 2 + examples/pipeline/cli.c | 4 + examples/pipeline/example_l2fwd_macswp.c | 146 ++++++++++++++++++ examples/pipeline/examples/l2fwd_macswp.cli | 25 +++ .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 +++ examples/pipeline/meson.build | 1 + 6 files changed, 198 insertions(+) create mode 100644 examples/pipeline/example_l2fwd_macswp.c create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 405978ced..df5176296 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -11,6 +11,8 @@ SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c SRCS-y += example_l2fwd.c +SRCS-y += example_l2fwd_macswp.c + # Build using pkg-config variables if possible ifeq ($(shell pkg-config --exists libdpdk && echo 0),0) diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c index a54cd79ff..9547e1b4c 100644 --- a/examples/pipeline/cli.c +++ b/examples/pipeline/cli.c @@ -731,6 +731,8 @@ static const char cmd_pipeline_build_help[] = "pipeline build \n"; int pipeline_setup_l2fwd(struct rte_swx_pipeline *p); +int pipeline_setup_l2fwd_macswp(struct rte_swx_pipeline *p); + static void cmd_pipeline_build(char **tokens, uint32_t n_tokens, @@ -757,6 +759,8 @@ cmd_pipeline_build(char **tokens, app = tokens[3]; if (!strcmp(app, "l2fwd")) status = pipeline_setup_l2fwd(p->p); + else if (!strcmp(app, "l2fwd_macswp")) + status = pipeline_setup_l2fwd_macswp(p->p); else { snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); return; diff --git a/examples/pipeline/example_l2fwd_macswp.c b/examples/pipeline/example_l2fwd_macswp.c new file mode 100644 index 000000000..7922a75aa --- /dev/null +++ b/examples/pipeline/example_l2fwd_macswp.c @@ -0,0 +1,146 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include +#include +#include + +#include + +#include "rte_swx_pipeline.h" +#include "rte_swx_port_ethdev.h" +#include "rte_swx_port_source_sink.h" +#include "rte_swx_table_em.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) { \ + printf("Error in function %s at line %d\n", \ + __FUNCTION__, __LINE__); \ + return -1; \ + } \ +} while (0) + +/* + * Packet headers. + */ +static struct rte_swx_field_params ethernet_type[] = { + {"dst_addr", 48}, + {"src_addr", 48}, + {"ether_type", 16}, +}; + +/* + * Packet meta-data. + */ +static struct rte_swx_field_params metadata_type[] = { + {"port", 32}, + {"addr", 48}, +}; + +/* + * Actions. + */ +static const char *action_macswp_instructions[] = { + "mov m.addr h.ethernet.dst_addr", + "mov h.ethernet.dst_addr h.ethernet.src_addr", + "mov h.ethernet.src_addr m.addr", + "return", +}; + +/* + * Tables. + */ +static const char *table_stub_actions[] = {"macswp"}; + +static struct rte_swx_pipeline_table_params table_stub_params = { + /* Match. */ + .fields = NULL, + .n_fields = 0, + + /* Action. */ + .action_names = table_stub_actions, + .n_actions = RTE_DIM(table_stub_actions), + .default_action_name = "macswp", + .default_action_data = NULL, + .default_action_is_const = 0, +}; + +/* + * Pipeline. + */ +static const char *pipeline_instructions[] = { + "rx m.port", + "extract h.ethernet", + "table stub", + "xor m.port 1", + "emit h.ethernet", + "tx m.port", +}; + +int +pipeline_setup_l2fwd_macswp(struct rte_swx_pipeline *p); + +int +pipeline_setup_l2fwd_macswp(struct rte_swx_pipeline *p) +{ + int err; + + /* + * Packet headers. + */ + err = rte_swx_pipeline_struct_type_register(p, + "ethernet_type", + ethernet_type, + RTE_DIM(ethernet_type)); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "ethernet", + "ethernet_type"); + CHECK(!err); + + /* + * Packet meta-data. + */ + err = rte_swx_pipeline_struct_type_register(p, + "metadata_type", + metadata_type, + RTE_DIM(metadata_type)); + CHECK(!err); + + err = rte_swx_pipeline_packet_metadata_register(p, + "metadata_type"); + CHECK(!err); + + /* + * Actions. + */ + err = rte_swx_pipeline_action_config(p, + "macswp", + NULL, + action_macswp_instructions, + RTE_DIM(action_macswp_instructions)); + CHECK(!err); + + /* + * Tables. + */ + err = rte_swx_pipeline_table_config(p, + "stub", + &table_stub_params, + NULL, + NULL, + 0); + CHECK(!err); + + /* + * Pipeline. + */ + err = rte_swx_pipeline_instructions_config(p, + pipeline_instructions, + RTE_DIM(pipeline_instructions)); + CHECK(!err); + + return 0; +} diff --git a/examples/pipeline/examples/l2fwd_macswp.cli b/examples/pipeline/examples/l2fwd_macswp.cli new file mode 100644 index 000000000..c5eb23d18 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build l2fwd_macswp + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd_macswp_pcap.cli b/examples/pipeline/examples/l2fwd_macswp_pcap.cli new file mode 100644 index 000000000..76c3071e3 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build l2fwd_macswp + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index b16de3714..b13f04e01 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -16,4 +16,5 @@ sources = files( 'obj.c', 'thread.c', 'example_l2fwd.c', + 'example_l2fwd_macswp.c', ) From patchwork Wed Aug 26 15:14:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 76035 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31EB8A04B1; Wed, 26 Aug 2020 17:24:40 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 34C171C2ED; Wed, 26 Aug 2020 17:15:58 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 011FB1C196 for ; Wed, 26 Aug 2020 17:15:31 +0200 (CEST) IronPort-SDR: NcI23TO2zSo266ftIhrLPZR434WSYWc7Trm25diDrqeEzKLHay7+L1oc/u4vZf8WVGAI1CInxO UGLpPYqA+EMg== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879650" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879650" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:15:31 -0700 IronPort-SDR: YQDyyjP5aCFLxLrv9l9l9IW/7ULEXUrjKn14ZJVEHRXzXk7SyCR6slYsY+Lx5XA5204Fmx1W2J 482XEHaE95+w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081508" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:15:30 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:45 +0100 Message-Id: <20200826151445.51500-41-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826151445.51500-1-cristian.dumitrescu@intel.com> References: <20200826151445.51500-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 40/40] examples/pipeline: add VXLAN encap example X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add VXLAN encapsulation example to the pipeline application. The VXLAN tunnels can be generated with the vxlan.py script. Example command line: ./build/pipeline -l0-1 -- -s ./examples/vxlan.cli Signed-off-by: Cristian Dumitrescu --- examples/pipeline/Makefile | 2 +- examples/pipeline/cli.c | 3 + examples/pipeline/example_vxlan.c | 318 ++++++++++++++++++++++ examples/pipeline/examples/vxlan.cli | 27 ++ examples/pipeline/examples/vxlan.py | 71 +++++ examples/pipeline/examples/vxlan.txt | 16 ++ examples/pipeline/examples/vxlan_pcap.cli | 22 ++ examples/pipeline/meson.build | 1 + 8 files changed, 459 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/example_vxlan.c create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.py create mode 100644 examples/pipeline/examples/vxlan.txt create mode 100644 examples/pipeline/examples/vxlan_pcap.cli diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index df5176296..9e8088ec4 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -12,7 +12,7 @@ SRCS-y += obj.c SRCS-y += thread.c SRCS-y += example_l2fwd.c SRCS-y += example_l2fwd_macswp.c - +SRCS-y += example_vxlan.c # Build using pkg-config variables if possible ifeq ($(shell pkg-config --exists libdpdk && echo 0),0) diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c index 9547e1b4c..d98400c78 100644 --- a/examples/pipeline/cli.c +++ b/examples/pipeline/cli.c @@ -732,6 +732,7 @@ static const char cmd_pipeline_build_help[] = int pipeline_setup_l2fwd(struct rte_swx_pipeline *p); int pipeline_setup_l2fwd_macswp(struct rte_swx_pipeline *p); +int pipeline_setup_vxlan(struct rte_swx_pipeline *p); static void cmd_pipeline_build(char **tokens, @@ -761,6 +762,8 @@ cmd_pipeline_build(char **tokens, status = pipeline_setup_l2fwd(p->p); else if (!strcmp(app, "l2fwd_macswp")) status = pipeline_setup_l2fwd_macswp(p->p); + else if (!strcmp(app, "vxlan")) + status = pipeline_setup_vxlan(p->p); else { snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); return; diff --git a/examples/pipeline/example_vxlan.c b/examples/pipeline/example_vxlan.c new file mode 100644 index 000000000..b4701e20f --- /dev/null +++ b/examples/pipeline/example_vxlan.c @@ -0,0 +1,318 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include +#include +#include + +#include + +#include "rte_swx_pipeline.h" +#include "rte_swx_table_em.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) { \ + printf("Error in function %s at line %d\n", \ + __FUNCTION__, __LINE__); \ + return -1; \ + } \ +} while (0) + +/* + * Packet headers. + */ +static struct rte_swx_field_params ethernet_h[] = { + {"dst_addr", 48}, + {"src_addr", 48}, + {"ether_type", 16}, +}; + +static struct rte_swx_field_params ipv4_h[] = { + {"ver_ihl", 8}, + {"diffserv", 8}, + {"total_len", 16}, + {"identification", 16}, + {"flags_offset", 16}, + {"ttl", 8}, + {"protocol", 8}, + {"hdr_checksum", 16}, + {"src_addr", 32}, + {"dst_addr", 32}, +}; + +static struct rte_swx_field_params udp_h[] = { + {"src_port", 16}, + {"dst_port", 16}, + {"length", 16}, + {"checksum", 16}, +}; + +static struct rte_swx_field_params vxlan_h[] = { + {"flags", 8}, + {"reserved", 24}, + {"vni", 24}, + {"reserved2", 8}, +}; + +/* + * Packet meta-data. + */ +static struct rte_swx_field_params metadata_t[] = { + {"port_in", 32}, + {"port_out", 32}, +}; + +/* + * Actions. + */ +static const char *drop_instructions[] = { + "mov m.port_out 4", + "tx m.port_out", +}; + +static struct rte_swx_field_params vxlan_encap_args_t[] = { + {"ethernet_dst_addr", 48}, + {"ethernet_src_addr", 48}, + {"ethernet_ether_type", 16}, + {"ipv4_ver_ihl", 8}, + {"ipv4_diffserv", 8}, + {"ipv4_total_len", 16}, + {"ipv4_identification", 16}, + {"ipv4_flags_offset", 16}, + {"ipv4_ttl", 8}, + {"ipv4_protocol", 8}, + {"ipv4_hdr_checksum", 16}, + {"ipv4_src_addr", 32}, + {"ipv4_dst_addr", 32}, + {"udp_src_port", 16}, + {"udp_dst_port", 16}, + {"udp_length", 16}, + {"udp_checksum", 16}, + {"vxlan_flags", 8}, + {"vxlan_reserved", 24}, + {"vxlan_vni", 24}, + {"vxlan_reserved2", 8}, + {"port_out", 32}, +}; + +/* Input frame: + * Ethernet (14) | IPv4 (total_len) + * + * Output frame: + * Ethernet (14) | IPv4 (20) | UDP (8) | VXLAN (8) | Input frame | FCS (4) + * + * Note: The input frame has its FCS removed before encapsulation in the output + * frame. + * + * Assumption: When read from the table, the outer IPv4 and UDP headers contain + * the following fields: + * - t.ipv4_total_len: Set to 50, which covers the length of: + * - The outer IPv4 header (20 bytes); + * - The outer UDP header (8 bytes); + * - The outer VXLAN header (8 bytes); + * - The inner Ethernet header (14 bytes); + * - t.ipv4_hdr_checksum: Includes the above total length. + * - t.udp_length: Set to 30, which covers the length of: + * - The outer UDP header (8 bytes); + * - The outer VXLAN header (8 bytes); + * - The inner Ethernet header (14 bytes); + * - t.udp_checksum: Set to 0. + * + * Once the total length of the inner IPv4 packet (h.ipv4.total_len) is known, + * the outer IPv4 and UDP headers are updated as follows: + * - h.outer_ipv4.total_len = t.ipv4_total_len + h.ipv4.total_len + * - h.outer_ipv4.hdr_checksum = t.ipv4_hdr_checksum + h.ipv4.total_len + * - h.outer_udp.length = t.udp_length + h.ipv4.total_len + * - h.outer_udp.checksum: No change. + */ +static const char *vxlan_encap_instructions[] = { + /* Copy from table entry to haders and metadata. */ + "dma h.outer_ethernet t.ethernet_dst_addr", + "dma h.outer_ipv4 t.ipv4_ver_ihl", + "dma h.outer_udp t.udp_src_port", + "dma h.outer_vxlan t.vxlan_flags", + "mov m.port_out t.port_out", + + /* Update h.outer_ipv4.total_len field. */ + "add h.outer_ipv4.total_len h.ipv4.total_len", + + /* Update h.outer_ipv4.hdr_checksum field. */ + "ckadd h.outer_ipv4.hdr_checksum h.ipv4.total_len", + + /* Update h.outer_udp.length field. */ + "add h.outer_udp.length h.ipv4.total_len", + + "return" +}; + +/* + * Tables. + */ +static struct rte_swx_match_field_params table_match_fields[] = { + [0] = { + .name = "h.ethernet.dst_addr", + .match_type = RTE_SWX_TABLE_MATCH_EXACT, + }, +}; + +static const char *table_actions[] = {"drop", "vxlan_encap"}; + +static struct rte_swx_pipeline_table_params table_params = { + /* Match. */ + .fields = table_match_fields, + .n_fields = RTE_DIM(table_match_fields), + + /* Action. */ + .action_names = table_actions, + .n_actions = RTE_DIM(table_actions), + .default_action_name = "drop", + .default_action_data = NULL, + .default_action_is_const = 0, +}; + +/* + * Pipeline. + */ +static const char *pipeline_instructions[] = { + "rx m.port_in", + "extract h.ethernet", + "extract h.ipv4", + "table vxlan", + "emit h.outer_ethernet", + "emit h.outer_ipv4", + "emit h.outer_udp", + "emit h.outer_vxlan", + "emit h.ethernet", + "emit h.ipv4", + "tx m.port_out", +}; + +int +pipeline_setup_vxlan(struct rte_swx_pipeline *p); + +int +pipeline_setup_vxlan(struct rte_swx_pipeline *p) +{ + int err; + + /* + * Packet headers. + */ + err = rte_swx_pipeline_struct_type_register(p, + "ethernet_h", + ethernet_h, + RTE_DIM(ethernet_h)); + CHECK(!err); + + err = rte_swx_pipeline_struct_type_register(p, + "ipv4_h", + ipv4_h, + RTE_DIM(ipv4_h)); + CHECK(!err); + + err = rte_swx_pipeline_struct_type_register(p, + "udp_h", + udp_h, + RTE_DIM(udp_h)); + CHECK(!err); + + err = rte_swx_pipeline_struct_type_register(p, + "vxlan_h", + vxlan_h, + RTE_DIM(vxlan_h)); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "outer_ethernet", + "ethernet_h"); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "outer_ipv4", + "ipv4_h"); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "outer_udp", + "udp_h"); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "outer_vxlan", + "vxlan_h"); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "ethernet", + "ethernet_h"); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "ipv4", + "ipv4_h"); + CHECK(!err); + + /* + * Packet meta-data. + */ + err = rte_swx_pipeline_struct_type_register(p, + "metadata_t", + metadata_t, + RTE_DIM(metadata_t)); + CHECK(!err); + + err = rte_swx_pipeline_packet_metadata_register(p, + "metadata_t"); + CHECK(!err); + + /* + * Actions. + */ + err = rte_swx_pipeline_action_config(p, + "drop", + NULL, + drop_instructions, + RTE_DIM(drop_instructions)); + CHECK(!err); + + err = rte_swx_pipeline_struct_type_register(p, + "vxlan_encap_args_t", + vxlan_encap_args_t, + RTE_DIM(vxlan_encap_args_t)); + CHECK(!err); + + err = rte_swx_pipeline_action_config(p, + "vxlan_encap", + "vxlan_encap_args_t", + vxlan_encap_instructions, + RTE_DIM(vxlan_encap_instructions)); + CHECK(!err); + + /* + * Tables. + */ + err = rte_swx_pipeline_table_type_register(p, + "exact", + RTE_SWX_TABLE_MATCH_EXACT, + &rte_swx_table_exact_match_ops); + CHECK(!err); + + err = rte_swx_pipeline_table_config(p, + "vxlan", + &table_params, + NULL, + NULL, + 1 * 1024 * 1024); + CHECK(!err); + + /* + * Pipeline. + */ + err = rte_swx_pipeline_instructions_config(p, + pipeline_instructions, + RTE_DIM(pipeline_instructions)); + CHECK(!err); + + return 0; +} diff --git a/examples/pipeline/examples/vxlan.cli b/examples/pipeline/examples/vxlan.cli new file mode 100644 index 000000000..53a8b2a0a --- /dev/null +++ b/examples/pipeline/examples/vxlan.cli @@ -0,0 +1,27 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build vxlan +pipeline PIPELINE0 table vxlan update ./examples/vxlan.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan.py b/examples/pipeline/examples/vxlan.py new file mode 100644 index 000000000..179d31b53 --- /dev/null +++ b/examples/pipeline/examples/vxlan.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python2 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +from __future__ import print_function +import argparse +import re +import os + +DESCRIPTION = 'Table Generator' + +KEY = '0xaabbccdd{0:04x}' +ACTION = 'vxlan_encap' +ETHERNET_HEADER = 'ethernet_dst_addr N(0xa0a1a2a3{0:04x}) ' \ + 'ethernet_src_addr N(0xb0b1b2b3{0:04x}) ' \ + 'ethernet_ether_type N(0x0800)' +IPV4_HEADER = 'ipv4_ver_ihl N(0x45) ' \ + 'ipv4_diffserv N(0) ' \ + 'ipv4_total_len N(50) ' \ + 'ipv4_identification N(0) ' \ + 'ipv4_flags_offset N(0) ' \ + 'ipv4_ttl N(64) ' \ + 'ipv4_protocol N(17) ' \ + 'ipv4_hdr_checksum N(0x{1:04x}) ' \ + 'ipv4_src_addr N(0xc0c1{0:04x}) ' \ + 'ipv4_dst_addr N(0xd0d1{0:04x})' +UDP_HEADER = 'udp_src_port N(0xe0{0:02x}) ' \ + 'udp_dst_port N(4789) ' \ + 'udp_length N(30) ' \ + 'udp_checksum N(0)' +VXLAN_HEADER = 'vxlan_flags N(0) ' \ + 'vxlan_reserved N(0) ' \ + 'vxlan_vni N({0:d}) ' \ + 'vxlan_reserved2 N(0)' +PORT_OUT = 'port_out H({0:d})' + +def ipv4_header_checksum(i): + cksum = (0x4500 + 0x0032) + (0x0000 + 0x0000) + (0x4011 + 0x0000) + (0xc0c1 + i) + (0xd0d1 + i) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = ~cksum & 0xFFFF + return cksum + +def table_generate(n, p): + for i in range(0, n): + print("match %s action %s %s %s %s %s %s" % (KEY.format(i), + ACTION, + ETHERNET_HEADER.format(i), + IPV4_HEADER.format(i, ipv4_header_checksum(i)), + UDP_HEADER.format(i % 256), + VXLAN_HEADER.format(i), + PORT_OUT.format(i % p))) + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=DESCRIPTION) + + parser.add_argument( + '-n', + help='number of table entries (default: 65536)', + required=False, + default=65536) + + parser.add_argument( + '-p', + help='number of network ports (default: 4)', + required=False, + default=4) + + args = parser.parse_args() + table_generate(int(args.n), int(args.p)) diff --git a/examples/pipeline/examples/vxlan.txt b/examples/pipeline/examples/vxlan.txt new file mode 100644 index 000000000..acac80a38 --- /dev/null +++ b/examples/pipeline/examples/vxlan.txt @@ -0,0 +1,16 @@ +match 0xaabbccdd0000 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30000) ethernet_src_addr N(0xb0b1b2b30000) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe928) ipv4_src_addr N(0xc0c10000) ipv4_dst_addr N(0xd0d10000) udp_src_port N(0xe000) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(0) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0001 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30001) ethernet_src_addr N(0xb0b1b2b30001) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe926) ipv4_src_addr N(0xc0c10001) ipv4_dst_addr N(0xd0d10001) udp_src_port N(0xe001) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(1) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0002 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30002) ethernet_src_addr N(0xb0b1b2b30002) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe924) ipv4_src_addr N(0xc0c10002) ipv4_dst_addr N(0xd0d10002) udp_src_port N(0xe002) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(2) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0003 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30003) ethernet_src_addr N(0xb0b1b2b30003) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe922) ipv4_src_addr N(0xc0c10003) ipv4_dst_addr N(0xd0d10003) udp_src_port N(0xe003) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(3) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0004 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30004) ethernet_src_addr N(0xb0b1b2b30004) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe920) ipv4_src_addr N(0xc0c10004) ipv4_dst_addr N(0xd0d10004) udp_src_port N(0xe004) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(4) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0005 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30005) ethernet_src_addr N(0xb0b1b2b30005) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91e) ipv4_src_addr N(0xc0c10005) ipv4_dst_addr N(0xd0d10005) udp_src_port N(0xe005) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(5) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0006 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30006) ethernet_src_addr N(0xb0b1b2b30006) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91c) ipv4_src_addr N(0xc0c10006) ipv4_dst_addr N(0xd0d10006) udp_src_port N(0xe006) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(6) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0007 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30007) ethernet_src_addr N(0xb0b1b2b30007) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91a) ipv4_src_addr N(0xc0c10007) ipv4_dst_addr N(0xd0d10007) udp_src_port N(0xe007) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(7) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0008 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30008) ethernet_src_addr N(0xb0b1b2b30008) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe918) ipv4_src_addr N(0xc0c10008) ipv4_dst_addr N(0xd0d10008) udp_src_port N(0xe008) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(8) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0009 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30009) ethernet_src_addr N(0xb0b1b2b30009) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe916) ipv4_src_addr N(0xc0c10009) ipv4_dst_addr N(0xd0d10009) udp_src_port N(0xe009) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(9) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000a action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000a) ethernet_src_addr N(0xb0b1b2b3000a) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe914) ipv4_src_addr N(0xc0c1000a) ipv4_dst_addr N(0xd0d1000a) udp_src_port N(0xe00a) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(10) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000b action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000b) ethernet_src_addr N(0xb0b1b2b3000b) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe912) ipv4_src_addr N(0xc0c1000b) ipv4_dst_addr N(0xd0d1000b) udp_src_port N(0xe00b) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(11) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd000c action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000c) ethernet_src_addr N(0xb0b1b2b3000c) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe910) ipv4_src_addr N(0xc0c1000c) ipv4_dst_addr N(0xd0d1000c) udp_src_port N(0xe00c) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(12) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd000d action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000d) ethernet_src_addr N(0xb0b1b2b3000d) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90e) ipv4_src_addr N(0xc0c1000d) ipv4_dst_addr N(0xd0d1000d) udp_src_port N(0xe00d) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(13) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000e action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000e) ethernet_src_addr N(0xb0b1b2b3000e) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90c) ipv4_src_addr N(0xc0c1000e) ipv4_dst_addr N(0xd0d1000e) udp_src_port N(0xe00e) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(14) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000f action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000f) ethernet_src_addr N(0xb0b1b2b3000f) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90a) ipv4_src_addr N(0xc0c1000f) ipv4_dst_addr N(0xd0d1000f) udp_src_port N(0xe00f) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(15) vxlan_reserved2 N(0) port_out H(3) diff --git a/examples/pipeline/examples/vxlan_pcap.cli b/examples/pipeline/examples/vxlan_pcap.cli new file mode 100644 index 000000000..c406e27d3 --- /dev/null +++ b/examples/pipeline/examples/vxlan_pcap.cli @@ -0,0 +1,22 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build vxlan +pipeline PIPELINE0 table vxlan update ./examples/vxlan.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index b13f04e01..7f6f5218b 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -17,4 +17,5 @@ sources = files( 'thread.c', 'example_l2fwd.c', 'example_l2fwd_macswp.c', + 'example_vxlan.c', )