From patchwork Wed Aug 26 15:14:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cristian Dumitrescu X-Patchwork-Id: 75995 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 615B6A04B4; Wed, 26 Aug 2020 17:15:04 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B310C58C4; Wed, 26 Aug 2020 17:15:03 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id E5D84B62 for ; Wed, 26 Aug 2020 17:15:01 +0200 (CEST) IronPort-SDR: W7gDkcdv+6XPxxbd1JmGJgchdjENPCVC5MON+/ENUYhBPcFlffj+9nnNv6TlgZyFqe7+83COl1 J5a2NChKJw0A== X-IronPort-AV: E=McAfee;i="6000,8403,9725"; a="153879474" X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="153879474" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2020 08:14:47 -0700 IronPort-SDR: LRZnt+Sw/cGA/UmeDVkBOaDDyi7YG0NynL8GRFei3ndsi6ATB+CYYKTfuY5AfN3G+232+/JlIw 1CXaM4imgD4A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,356,1592895600"; d="scan'208";a="444081247" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga004.jf.intel.com with ESMTP; 26 Aug 2020 08:14:46 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Wed, 26 Aug 2020 16:14:05 +0100 Message-Id: <20200826151445.51500-1-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 Subject: [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch set introduces a new pipeline type that combines the DPDK performance with the flexibility of the P4-16 language[1]. The new API can be used either by itself to code a complete software switch (SWX) or data plane app, or in combination with the open-source P4 compiler P4C [2], potentially acting as a P4C back-end, thus allowing the P4 programs to be translated to the DPDK API and run on multi-core CPUs. Main new features: * Nothing is hard-wired, everything is dynamically defined: The packet headers (i.e. protocols), the packet meta-data, the actions, the tables and the pipeline itself are dynamically defined instead of having to be selected from a pre-defined set. * Instructions: The actions and the life of the packet through the pipeline are defined with instructions that manipulate the pipeline objects mentioned above. The pipeline is the main function of the packet program, with actions as subroutines triggered by the tables. * Call external plugins: Extern objects and functions can be defined to call functionality that cannot be efficiently implemented with the existing pipeline-oriented instruction set, such as: special error detecting/correcting codes, crypto, meters, stats arrays, heuristics, etc. * Better control plane interaction: Transaction-oriented table update mechanism that supports multi-table atomic updates. Multiple tables can be updated in a single step with only the before and after table sets visible to the packets. Alignment with P4Runtime [3]. * Performance: Multiple packets are in-flight within the pipeline at any moment. Each packet is owned by a different time-sharing thread in run-to-completion, with the thread pausing before memory access operations such as packet I/O and table lookup to allow the memory prefetch to complete. The instructions are verified and translated at initialization time with no run-time impact. The instructions are also optimized to detect and "fuse" frequently used patterns into vector-like instructions transparently to the user. [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf [2] P4-16 compiler: https://github.com/p4lang/p4c [3] P4Runtime specification: https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf Cristian Dumitrescu (40): pipeline: add pipeline pipeline: add input port pipeline: add output port pipeline: add headers and meta-data pipeline: add extern objects and functions pipeline: add action pipeline: add tables pipeline: add pipeline instructions pipeline: add rx and extract instructions pipeline: add tx and emit instructions pipeline: add header validate and invalidate instructions pipeline: add mov instruction pipeline: add dma instruction pipeline: introduce add instruction pipeline: introduce sub instruction pipeline: introduce ckadd instruction pipeline: introduce cksub instruction pipeline: introduce and instruction pipeline: introduce or instruction pipeline: introduce xor instruction pipeline: introduce shl instruction pipeline: introduce shr instruction pipeline: introduce table instruction pipeline: introduce extern instruction pipeline: introduce jmp and return instructions pipeline: add instruction verifier pipeline: add instruction optimizer pipeline: add pipeline query API pipeline: add pipeline flush pipeline: add instruction description pipeline: add table update high level API port: add ethernet device port port: add source and sink ports table: add exact match table examples/pipeline: add new example application examples/pipeline: add message passing mechanism examples/pipeline: add configuration commands examples/pipeline: add l2fwd example examples/pipeline: add l2fwd with MAC swap example examples/pipeline: add VXLAN encap example examples/Makefile | 1 + examples/meson.build | 1 + examples/pipeline/Makefile | 85 + examples/pipeline/cli.c | 1394 ++++ examples/pipeline/cli.h | 19 + examples/pipeline/conn.c | 331 + examples/pipeline/conn.h | 50 + examples/pipeline/example_l2fwd.c | 125 + examples/pipeline/example_l2fwd_macswp.c | 146 + examples/pipeline/example_vxlan.c | 318 + examples/pipeline/examples/l2fwd.cli | 25 + examples/pipeline/examples/l2fwd_macswp.cli | 25 + .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 + examples/pipeline/examples/l2fwd_pcap.cli | 20 + examples/pipeline/examples/packet.txt | 102 + examples/pipeline/examples/vxlan.cli | 27 + examples/pipeline/examples/vxlan.py | 71 + examples/pipeline/examples/vxlan.txt | 16 + examples/pipeline/examples/vxlan_pcap.cli | 22 + examples/pipeline/main.c | 193 + examples/pipeline/meson.build | 21 + examples/pipeline/obj.c | 470 ++ examples/pipeline/obj.h | 131 + examples/pipeline/thread.c | 549 ++ examples/pipeline/thread.h | 28 + lib/librte_pipeline/Makefile | 5 + lib/librte_pipeline/meson.build | 13 +- lib/librte_pipeline/rte_pipeline_version.map | 43 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++ lib/librte_pipeline/rte_swx_ctl.h | 568 ++ lib/librte_pipeline/rte_swx_extern.h | 98 + lib/librte_pipeline/rte_swx_pipeline.c | 7197 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 685 ++ lib/librte_port/Makefile | 5 + lib/librte_port/meson.build | 9 +- lib/librte_port/rte_port_version.map | 5 +- lib/librte_port/rte_swx_port.h | 202 + lib/librte_port/rte_swx_port_ethdev.c | 313 + lib/librte_port/rte_swx_port_ethdev.h | 54 + lib/librte_port/rte_swx_port_source_sink.c | 335 + lib/librte_port/rte_swx_port_source_sink.h | 57 + lib/librte_table/Makefile | 3 + lib/librte_table/meson.build | 7 +- lib/librte_table/rte_swx_table.h | 295 + lib/librte_table/rte_swx_table_em.c | 851 ++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 47 files changed, 16516 insertions(+), 8 deletions(-) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h create mode 100644 examples/pipeline/example_l2fwd.c create mode 100644 examples/pipeline/example_l2fwd_macswp.c create mode 100644 examples/pipeline/example_vxlan.c create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.py create mode 100644 examples/pipeline/examples/vxlan.txt create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h create mode 100644 lib/librte_pipeline/rte_swx_ctl.c create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_pipeline/rte_swx_extern.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h create mode 100644 lib/librte_port/rte_swx_port.h create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h create mode 100644 lib/librte_table/rte_swx_table.h create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h