@@ -1,5 +1,6 @@
.. SPDX-License-Identifier: BSD-3-Clause
Copyright(c) 2016-2017 Intel Corporation.
+ Copyright (C) 2020 Marvell International Ltd.
IPsec Security Gateway Sample Application
=========================================
@@ -61,6 +62,44 @@ The Path for the IPsec Outbound traffic is:
* Routing.
* Write packet to port.
+The application supports two modes of operation: poll mode and event mode.
+
+* In the poll mode a core receives packets from statically configured list
+ of eth ports and eth ports' queues.
+
+* In the event mode a core receives packets as events. After packet processing
+ is done core submits them back as events to an event device. This enables
+ multicore scaling and HW assisted scheduling by making use of the event device
+ capabilities. The event mode configuration is predefined. All packets reaching
+ given eth port will arrive at the same event queue. All event queues are mapped
+ to all event ports. This allows all cores to receive traffic from all ports.
+ Since the underlying event device might have varying capabilities, the worker
+ threads can be drafted differently to maximize performance. For example, if an
+ event device - eth device pair has Tx internal port, then application can call
+ rte_event_eth_tx_adapter_enqueue() instead of regular rte_event_enqueue_burst().
+ So a thread which assumes that the device pair has internal port will not be the
+ right solution for another pair. The infrastructure added for the event mode aims
+ to help application to have multiple worker threads by maximizing performance from
+ every type of event device without affecting existing paths/use cases. The worker
+ to be used will be determined by the operating conditions and the underlying device
+ capabilities. **Currently the application provides non-burst, internal port worker
+ threads and supports inline protocol only.** It also provides infrastructure for
+ non-internal port however does not define any worker threads.
+
+Additionally the event mode introduces two submodes of processing packets:
+
+* Driver submode: This submode has bare minimum changes in the application to support
+ IPsec. There are no lookups, no routing done in the application. And for inline
+ protocol use case, the worker thread resembles l2fwd worker thread as the IPsec
+ processing is done entirely in HW. This mode can be used to benchmark the raw
+ performance of the HW. The driver submode is selected with --single-sa option
+ (used also by poll mode). When --single-sa option is used in conjution with event
+ mode then index passed to --single-sa is ignored.
+
+* App submode: This submode has all the features currently implemented with the
+ application (non librte_ipsec path). All the lookups, routing follows existing
+ methods and report numbers that can be compared against regular poll mode
+ benchmark numbers.
Constraints
-----------
@@ -94,13 +133,18 @@ The application has a number of command line options::
-p PORTMASK -P -u PORTMASK -j FRAMESIZE
-l -w REPLAY_WINOW_SIZE -e -a
-c SAD_CACHE_SIZE
+ -s NUMBER_OF_MBUFS_IN_PACKET_POOL
+ -f CONFIG_FILE_PATH
--config (port,queue,lcore)[,(port,queue,lcore]
--single-sa SAIDX
+ --cryptodev_mask MASK
+ --transfer-mode MODE
+ --event-schedule-type TYPE
--rxoffload MASK
--txoffload MASK
- --mtu MTU
--reassemble NUM
- -f CONFIG_FILE_PATH
+ --mtu MTU
+ --frag-ttl FRAG_TTL_NS
Where:
@@ -138,12 +182,38 @@ Where:
Zero value disables cache.
Default value: 128.
-* ``--config (port,queue,lcore)[,(port,queue,lcore)]``: determines which queues
- from which ports are mapped to which cores.
+* ``-s``: sets number of mbufs in packet pool, if not provided number of mbufs
+ will be calculated based on number of cores, eth ports and crypto queues.
+
+* ``-f CONFIG_FILE_PATH``: the full path of text-based file containing all
+ configuration items for running the application (See Configuration file
+ syntax section below). ``-f CONFIG_FILE_PATH`` **must** be specified.
+ **ONLY** the UNIX format configuration file is accepted.
+
+* ``--config (port,queue,lcore)[,(port,queue,lcore)]``: in poll mode determines
+ which queues from which ports are mapped to which cores. In event mode this
+ is required for eth ports initialization only. Afterwards packets are dynamically
+ scheduled to cores by HW.
+
+* ``--single-sa SAIDX``: in poll mode use a single SA for outbound traffic,
+ bypassing the SP on both Inbound and Outbound. This option is meant for
+ debugging/performance purposes. In event mode selects driver submode, SA index
+ value is ignored.
-* ``--single-sa SAIDX``: use a single SA for outbound traffic, bypassing the SP
- on both Inbound and Outbound. This option is meant for debugging/performance
- purposes.
+* ``--cryptodev_mask MASK``: hexadecimal bitmask of the crypto devices
+ to configure.
+
+* ``--transfer-mode MODE``: sets operating mode of the application
+ "poll" : packet transfer via polling (default)
+ "event" : Packet transfer via event device
+
+* ``--event-schedule-type TYPE``: queue schedule type, applies only when
+ --transfer-mode is set to event.
+ "ordered" : Ordered (default)
+ "atomic" : Atomic
+ "parallel" : Parallel
+ When --event-schedule-type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event
+ device will ensure the ordering. Ordering will be lost when tried in PARALLEL.
* ``--rxoffload MASK``: RX HW offload capabilities to enable/use on this port
(bitmask of DEV_RX_OFFLOAD_* values). It is an optional parameter and
@@ -155,6 +225,10 @@ Where:
allows user to disable some of the TX HW offload capabilities.
By default all HW TX offloads are enabled.
+* ``--reassemble NUM``: max number of entries in reassemble fragment table.
+ Zero value disables reassembly functionality.
+ Default value: 0.
+
* ``--mtu MTU``: MTU value (in bytes) on all attached ethernet ports.
Outgoing packets with length bigger then MTU will be fragmented.
Incoming packets with length bigger then MTU will be discarded.
@@ -167,26 +241,17 @@ Where:
Should be lower for low number of reassembly buckets.
Valid values: from 1 ns to 10 s. Default value: 10000000 (10 s).
-* ``--reassemble NUM``: max number of entries in reassemble fragment table.
- Zero value disables reassembly functionality.
- Default value: 0.
-
-* ``-f CONFIG_FILE_PATH``: the full path of text-based file containing all
- configuration items for running the application (See Configuration file
- syntax section below). ``-f CONFIG_FILE_PATH`` **must** be specified.
- **ONLY** the UNIX format configuration file is accepted.
-
The mapping of lcores to port/queues is similar to other l3fwd applications.
-For example, given the following command line::
+For example, given the following command line to run application in poll mode::
./build/ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048 \
- --vdev "crypto_null" -- -p 0xf -P -u 0x3 \
+ --vdev "crypto_null" -- -p 0xf -P -u 0x3 \
--config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)" \
- -f /path/to/config_file \
+ -f /path/to/config_file --transfer-mode poll \
-where each options means:
+where each option means:
* The ``-l`` option enables cores 20 and 21.
@@ -200,7 +265,7 @@ where each options means:
* The ``-P`` option enables promiscuous mode.
-* The ``-u`` option sets ports 1 and 2 as unprotected, leaving 2 and 3 as protected.
+* The ``-u`` option sets ports 0 and 1 as unprotected, leaving 2 and 3 as protected.
* The ``--config`` option enables one queue per port with the following mapping:
@@ -228,6 +293,37 @@ where each options means:
**note** the parser only accepts UNIX format text file. Other formats
such as DOS/MAC format will cause a parse error.
+* The ``--transfer-mode`` option selects poll mode for processing packets.
+
+Similarly for example, given the following command line to run application in
+event app mode::
+
+ ./build/ipsec-secgw -c 0x3 -- -P -p 0x3 -u 0x1 \
+ --config "(1,0,0),(0,0,1)" \
+ -f /path/to/config_file --transfer-mode event \
+ --event-schedule-type parallel \
+
+where each option means:
+
+* The ``-c`` option selects cores 0 and 1 to run on.
+
+* The ``-P`` option enables promiscuous mode.
+
+* The ``-p`` option enables ports (detected) 0 and 1.
+
+* The ``-u`` option sets ports 0 as unprotected, leaving 1 as protected.
+
+* The ``--config`` option provides configuration for eth ports initialization
+ only. Afterwards packets are dynamically scheduled to cores by HW.
+
+* The ``-f /path/to/config_file`` option has the same behavior as in poll
+ mode example.
+
+* The ``--transfer-mode`` option selects event mode for processing packets.
+
+* The ``--event-schedule-type`` option selects parallel ordering of event queues.
+
+
Refer to the *DPDK Getting Started Guide* for general information on running
applications and the Environment Abstraction Layer (EAL) options.