[RFC,00/13] add eventmode to ipsec-secgw

  This series introduces event-mode additions to ipsec-secgw. This effort
is based on the proposed changes for l2fwd-event and the additions in
l3fwd for event support.

With this series, ipsec-secgw would be able to run in eventmode. The
worker thread (executing loop) would be receiving events and would be
submitting it back to the eventdev after the processing. This way,
multicore scaling and h/w assisted scheduling is achieved by making use
of the eventdev capabilities.

Since the underlying event device would be having varying capabilities,
the worker thread could be drafted differently to maximize performance.
This series introduces usage of multiple worker threads, among which the
one to be used will be determined by the operating conditions and the
underlying device capabilities.

For example, if an event device - eth device pair has Tx internal port,
then application can do tx_adapter_enqueue() instead of regular
event_enqueue(). So a thread making an assumption that the device pair
has internal port will not be the right solution for another pair. The
infrastructure added with these patches aims to help application to have
multiple worker threads, there by extracting maximum performance from
every device without affecting existing paths/use cases.

The eventmode configuration is predefined. All packets reaching one eth
port will hit one event queue. All event queues will be mapped to all
event ports. So all cores will be able to receive traffic from all ports.
When schedule_type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event device
will ensure the ordering. Ordering would be lost when tried in PARALLEL.

Following command line options are introduced,

--transfer-mode: to choose between poll mode & event mode
--schedule-type: to specify the scheduling type
                 (RTE_SCHED_TYPE_ORDERED/
                  RTE_SCHED_TYPE_ATOMIC/
                  RTE_SCHED_TYPE_PARALLEL)
--process-dir: outbound/inbound
--process-mode: app mode /driver mode

The two s/w config options added to ipsec-secgw can be used in
benchmarking h/w performance,

1. process-dir : states whether the direction is outbound/inbound.
This option aims to avoid an unnecessary check of determining whether
inbound/outbound processing need to be done on the packet. For each
option a different light weight worker thread would be executed.

2. process-mode: states whether the application has to run in driver
                 mode or app mode.

Driver-mode: This mode will have bare minimum changes in the application
             to support ipsec. There woudn't be any lookup etc done in
             the application. And for inline-protocol use case, the
             thread would resemble l2fwd as the ipsec processing would be
             done entirely in the h/w. This mode can be used to benchmark
             the raw performance of the h/w. All the application side
             steps (like lookup) can be redone based on the requirement
             of the end user. Hence the need for a mode which would
             report the raw performance.

App-mode: This mode will have all the features currently implemented with
          ipsec-secgw (non librte_ipsec mode). All the lookups etc
          would follow the existing methods and would report numbers
          that can be compared against regular ipsec-secgw benchmark
          numbers.

Example commands to execute ipsec-secgw in various modes on OCTEONTX2 platform,

#Inbound driver mode
./ipsec-secgw -w 0002:02:00.0,nb_ipsec_in_sa=128 -w 0002:03:00.0,nb_ipsec_in_sa=128 -w 0002:04:00.0,nb_ipsec_in_sa=128 -w 0002:07:00.0,nb_ipsec_in_sa=128 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x7 – -P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f dpdk_internal/100g_4.3.cfg --transfer-mode 1 --schedule-type 2 --process-mode 1 --process-dir 1

#Inbound app mode
./ipsec-secgw -w 0002:02:00.0,nb_ipsec_in_sa=128 -w 0002:03:00.0,nb_ipsec_in_sa=128 -w 0002:04:00.0,nb_ipsec_in_sa=128 -w 0002:07:00.0,nb_ipsec_in_sa=128 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x3f – -P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f dpdk_internal/100g_4.3.cfg --transfer-mode 1 --schedule-type 2 --process-mode 0 --process-dir 1

#Outbound driver mode
./ipsec-secgw -w 0002:02:00.0 -w 0002:03:00.0 -w 0002:04:00.0 -w 0002:07:00.0 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x1f – -P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f a-aes-gcm-new.cfg --transfer-mode 1 --schedule-type 2 --process-mode 1 --process-dir 0

#Outbound app mode
./ipsec-secgw -w 0002:02:00.0 -w 0002:03:00.0 -w 0002:04:00.0 -w 0002:07:00.0 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x7f – -P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f a-aes-gcm-new.cfg --transfer-mode 1 --schedule-type 2 --process-mode 0 --process-dir 0

This series is targeted for next release (20.02). This series doesn't introduce
any library change. And the decision to add eventmode additions in ipsec-secgw
was approved by the Tech Board.

Following are missing in the RFC. Will add it when sending patches.
1. Documentation.
2. More cleanup is needed. There are options that are added so that future
   expansion is not hindered. Need inputs from the community if there is use
   case for them.

Following are planned features,
1. Add burst mode workers.
2. Add non tx internal port worker.
3. Verify support for Rx core (the support is added but lack of h/w to verify).
4. Add lookaside protocol support.

Following are features that Marvell won't be attempting.
1. Inline crypto support.
2. Lookaside crypto support.

For the features that Marvell won't be attempting, new workers can be
introduced by the respective stake holders.

Anoob Joseph (13):
  examples/ipsec-secgw: add framework for eventmode helper
  examples/ipsec-secgw: add eventdev port-lcore link
  examples/ipsec-secgw: add Rx adapter support
  examples/ipsec-secgw: add Tx adapter support
  examples/ipsec-secgw: add routines to display config
  examples/ipsec-secgw: add routines to launch workers
  examples/ipsec-secgw: add support for internal ports
  examples/ipsec-secgw: add eventmode to ipsec-secgw
  examples/ipsec-secgw: add app inbound worker
  examples/ipsec-secgw: add app processing code
  examples/ipsec-secgw: add driver outbound worker
  examples/ipsec-secgw: add app outbound worker
  examples/ipsec-secgw: add cmd line option for bufs

 examples/ipsec-secgw/Makefile       |    2 +
 examples/ipsec-secgw/event_helper.c | 1757 +++++++++++++++++++++++++++++++++++
 examples/ipsec-secgw/event_helper.h |  334 +++++++
 examples/ipsec-secgw/ipsec-secgw.c  |  436 +++++++--
 examples/ipsec-secgw/ipsec-secgw.h  |   81 ++
 examples/ipsec-secgw/ipsec.c        |    4 +
 examples/ipsec-secgw/ipsec.h        |   30 +-
 examples/ipsec-secgw/ipsec_worker.c |  766 +++++++++++++++
 examples/ipsec-secgw/ipsec_worker.h |   39 +
 examples/ipsec-secgw/meson.build    |    4 +-
 examples/ipsec-secgw/sa.c           |   11 -
 11 files changed, 3360 insertions(+), 104 deletions(-)
 create mode 100644 examples/ipsec-secgw/event_helper.c
 create mode 100644 examples/ipsec-secgw/event_helper.h
 create mode 100644 examples/ipsec-secgw/ipsec-secgw.h
 create mode 100644 examples/ipsec-secgw/ipsec_worker.c
 create mode 100644 examples/ipsec-secgw/ipsec_worker.h
  

> This series introduces event-mode additions to ipsec-secgw. This effort
> is based on the proposed changes for l2fwd-event and the additions in
> l3fwd for event support.
> 
> With this series, ipsec-secgw would be able to run in eventmode. The
> worker thread (executing loop) would be receiving events and would be
> submitting it back to the eventdev after the processing. This way,
> multicore scaling and h/w assisted scheduling is achieved by making use
> of the eventdev capabilities.
> 
> Since the underlying event device would be having varying capabilities,
> the worker thread could be drafted differently to maximize performance.
> This series introduces usage of multiple worker threads, among which the
> one to be used will be determined by the operating conditions and the
> underlying device capabilities.
> 
> For example, if an event device - eth device pair has Tx internal port,
> then application can do tx_adapter_enqueue() instead of regular
> event_enqueue(). So a thread making an assumption that the device pair
> has internal port will not be the right solution for another pair. The
> infrastructure added with these patches aims to help application to have
> multiple worker threads, there by extracting maximum performance from
> every device without affecting existing paths/use cases.
> 
> The eventmode configuration is predefined. All packets reaching one eth
> port will hit one event queue. All event queues will be mapped to all
> event ports. So all cores will be able to receive traffic from all ports.
> When schedule_type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event device
> will ensure the ordering. Ordering would be lost when tried in PARALLEL.
> 
> Following command line options are introduced,
> 
> --transfer-mode: to choose between poll mode & event mode
> --schedule-type: to specify the scheduling type
>                  (RTE_SCHED_TYPE_ORDERED/
>                   RTE_SCHED_TYPE_ATOMIC/
>                   RTE_SCHED_TYPE_PARALLEL)
> --process-dir: outbound/inbound
> --process-mode: app mode /driver mode
> 
> The two s/w config options added to ipsec-secgw can be used in
> benchmarking h/w performance,
> 

I didn't look at the actual code (yet), just cover letter,
few quick questions below. 

> 1. process-dir : states whether the direction is outbound/inbound.
> This option aims to avoid an unnecessary check of determining whether
> inbound/outbound processing need to be done on the packet. For each
> option a different light weight worker thread would be executed.

Bur right now app can do both inbound and outbound simultaneously on the same core.
I presume the default behavior will be preserved?

> 
> 2. process-mode: states whether the application has to run in driver
>                  mode or app mode.
> 
> Driver-mode: This mode will have bare minimum changes in the application
>              to support ipsec. There woudn't be any lookup etc done in
>              the application. And for inline-protocol use case, the
>              thread would resemble l2fwd as the ipsec processing would be
>              done entirely in the h/w. This mode can be used to benchmark
>              the raw performance of the h/w. All the application side
>              steps (like lookup) can be redone based on the requirement
>              of the end user. Hence the need for a mode which would
>              report the raw performance.
> 
> App-mode: This mode will have all the features currently implemented with
>           ipsec-secgw (non librte_ipsec mode). All the lookups etc
>           would follow the existing methods and would report numbers
>           that can be compared against regular ipsec-secgw benchmark
>           numbers.
> 
> Example commands to execute ipsec-secgw in various modes on OCTEONTX2 platform,
> 
> #Inbound driver mode
> ./ipsec-secgw -w 0002:02:00.0,nb_ipsec_in_sa=128 -w 0002:03:00.0,nb_ipsec_in_sa=128 -w 0002:04:00.0,nb_ipsec_in_sa=128 -w
> 0002:07:00.0,nb_ipsec_in_sa=128 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x7 – -P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f
> dpdk_internal/100g_4.3.cfg --transfer-mode 1 --schedule-type 2 --process-mode 1 --process-dir 1

For all these parameters, I think better to use names (app/driver, etc.) instead of raw numbers.

> 
> #Inbound app mode
> ./ipsec-secgw -w 0002:02:00.0,nb_ipsec_in_sa=128 -w 0002:03:00.0,nb_ipsec_in_sa=128 -w 0002:04:00.0,nb_ipsec_in_sa=128 -w
> 0002:07:00.0,nb_ipsec_in_sa=128 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x3f – -P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -
> f dpdk_internal/100g_4.3.cfg --transfer-mode 1 --schedule-type 2 --process-mode 0 --process-dir 1
> 
> #Outbound driver mode
> ./ipsec-secgw -w 0002:02:00.0 -w 0002:03:00.0 -w 0002:04:00.0 -w 0002:07:00.0 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x1f – -
> P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f a-aes-gcm-new.cfg --transfer-mode 1 --schedule-type 2 --process-mode 1 --process-dir 0
> 
> #Outbound app mode
> ./ipsec-secgw -w 0002:02:00.0 -w 0002:03:00.0 -w 0002:04:00.0 -w 0002:07:00.0 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x7f – -
> P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f a-aes-gcm-new.cfg --transfer-mode 1 --schedule-type 2 --process-mode 0 --process-dir 0
> 
> This series is targeted for next release (20.02). This series doesn't introduce
> any library change. 

By 'library change' you mean that this new event-mode will be supported
only by legacy code-path or ...?

>And the decision to add eventmode additions in ipsec-secgw
> was approved by the Tech Board.
> 
> Following are missing in the RFC. Will add it when sending patches.
> 1. Documentation.
> 2. More cleanup is needed. There are options that are added so that future
>    expansion is not hindered. Need inputs from the community if there is use
>    case for them.
> 
> Following are planned features,
> 1. Add burst mode workers.
> 2. Add non tx internal port worker.
> 3. Verify support for Rx core (the support is added but lack of h/w to verify).
> 4. Add lookaside protocol support.
> 
> Following are features that Marvell won't be attempting.
> 1. Inline crypto support.
> 2. Lookaside crypto support.

Ok so what mode is supported right now with this RFC?

> 
> For the features that Marvell won't be attempting, new workers can be
> introduced by the respective stake holders.
> 
> Anoob Joseph (13):
>   examples/ipsec-secgw: add framework for eventmode helper
>   examples/ipsec-secgw: add eventdev port-lcore link
>   examples/ipsec-secgw: add Rx adapter support
>   examples/ipsec-secgw: add Tx adapter support
>   examples/ipsec-secgw: add routines to display config
>   examples/ipsec-secgw: add routines to launch workers
>   examples/ipsec-secgw: add support for internal ports
>   examples/ipsec-secgw: add eventmode to ipsec-secgw
>   examples/ipsec-secgw: add app inbound worker
>   examples/ipsec-secgw: add app processing code
>   examples/ipsec-secgw: add driver outbound worker
>   examples/ipsec-secgw: add app outbound worker
>   examples/ipsec-secgw: add cmd line option for bufs
> 
>  examples/ipsec-secgw/Makefile       |    2 +
>  examples/ipsec-secgw/event_helper.c | 1757 +++++++++++++++++++++++++++++++++++
>  examples/ipsec-secgw/event_helper.h |  334 +++++++
>  examples/ipsec-secgw/ipsec-secgw.c  |  436 +++++++--
>  examples/ipsec-secgw/ipsec-secgw.h  |   81 ++
>  examples/ipsec-secgw/ipsec.c        |    4 +
>  examples/ipsec-secgw/ipsec.h        |   30 +-
>  examples/ipsec-secgw/ipsec_worker.c |  766 +++++++++++++++
>  examples/ipsec-secgw/ipsec_worker.h |   39 +
>  examples/ipsec-secgw/meson.build    |    4 +-
>  examples/ipsec-secgw/sa.c           |   11 -
>  11 files changed, 3360 insertions(+), 104 deletions(-)
>  create mode 100644 examples/ipsec-secgw/event_helper.c
>  create mode 100644 examples/ipsec-secgw/event_helper.h
>  create mode 100644 examples/ipsec-secgw/ipsec-secgw.h
>  create mode 100644 examples/ipsec-secgw/ipsec_worker.c
>  create mode 100644 examples/ipsec-secgw/ipsec_worker.h
> 
> --
> 2.7.4
  

Hi Konstantin,

Thanks for the review. Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Wednesday, October 16, 2019 6:33 PM
> To: Anoob Joseph <anoobj@marvell.com>; Akhil Goyal
> <akhil.goyal@nxp.com>; Nicolau, Radu <radu.nicolau@intel.com>
> Cc: Thomas Monjalon <thomas@monjalon.net>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; Lukas Bartosik <lbartosik@marvell.com>;
> dev@dpdk.org
> Subject: [EXT] RE: [dpdk-dev] [RFC PATCH 00/13] add eventmode to ipsec-
> secgw
> 
> External Email
> 
> ----------------------------------------------------------------------
> 
> 
> > This series introduces event-mode additions to ipsec-secgw. This
> > effort is based on the proposed changes for l2fwd-event and the
> > additions in l3fwd for event support.
> >
> > With this series, ipsec-secgw would be able to run in eventmode. The
> > worker thread (executing loop) would be receiving events and would be
> > submitting it back to the eventdev after the processing. This way,
> > multicore scaling and h/w assisted scheduling is achieved by making
> > use of the eventdev capabilities.
> >
> > Since the underlying event device would be having varying
> > capabilities, the worker thread could be drafted differently to maximize
> performance.
> > This series introduces usage of multiple worker threads, among which
> > the one to be used will be determined by the operating conditions and
> > the underlying device capabilities.
> >
> > For example, if an event device - eth device pair has Tx internal
> > port, then application can do tx_adapter_enqueue() instead of regular
> > event_enqueue(). So a thread making an assumption that the device pair
> > has internal port will not be the right solution for another pair. The
> > infrastructure added with these patches aims to help application to
> > have multiple worker threads, there by extracting maximum performance
> > from every device without affecting existing paths/use cases.
> >
> > The eventmode configuration is predefined. All packets reaching one
> > eth port will hit one event queue. All event queues will be mapped to
> > all event ports. So all cores will be able to receive traffic from all ports.
> > When schedule_type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event
> > device will ensure the ordering. Ordering would be lost when tried in
> PARALLEL.
> >
> > Following command line options are introduced,
> >
> > --transfer-mode: to choose between poll mode & event mode
> > --schedule-type: to specify the scheduling type
> >                  (RTE_SCHED_TYPE_ORDERED/
> >                   RTE_SCHED_TYPE_ATOMIC/
> >                   RTE_SCHED_TYPE_PARALLEL)
> > --process-dir: outbound/inbound
> > --process-mode: app mode /driver mode
> >
> > The two s/w config options added to ipsec-secgw can be used in
> > benchmarking h/w performance,
> >
> 
> I didn't look at the actual code (yet), just cover letter, few quick questions
> below.
> 
> > 1. process-dir : states whether the direction is outbound/inbound.
> > This option aims to avoid an unnecessary check of determining whether
> > inbound/outbound processing need to be done on the packet. For each
> > option a different light weight worker thread would be executed.
> 
> Bur right now app can do both inbound and outbound simultaneously on the
> same core.
> I presume the default behavior will be preserved?

[Anoob] The existing behavior with poll mode thread is not touched. The current changes are only applicable when launched in event-mode.

> 
> >
> > 2. process-mode: states whether the application has to run in driver
> >                  mode or app mode.
> >
> > Driver-mode: This mode will have bare minimum changes in the application
> >              to support ipsec. There woudn't be any lookup etc done in
> >              the application. And for inline-protocol use case, the
> >              thread would resemble l2fwd as the ipsec processing would be
> >              done entirely in the h/w. This mode can be used to benchmark
> >              the raw performance of the h/w. All the application side
> >              steps (like lookup) can be redone based on the requirement
> >              of the end user. Hence the need for a mode which would
> >              report the raw performance.
> >
> > App-mode: This mode will have all the features currently implemented
> with
> >           ipsec-secgw (non librte_ipsec mode). All the lookups etc
> >           would follow the existing methods and would report numbers
> >           that can be compared against regular ipsec-secgw benchmark
> >           numbers.
> >
> > Example commands to execute ipsec-secgw in various modes on
> OCTEONTX2
> > platform,
> >
> > #Inbound driver mode
> > ./ipsec-secgw -w 0002:02:00.0,nb_ipsec_in_sa=128 -w
> > 0002:03:00.0,nb_ipsec_in_sa=128 -w 0002:04:00.0,nb_ipsec_in_sa=128 -w
> > 0002:07:00.0,nb_ipsec_in_sa=128 -w 0002:0e:00.0 -w 0002:10:00.1
> > --log-level=8 -c 0x7 – -P -p 0xf --config
> > "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f dpdk_internal/100g_4.3.cfg
> > --transfer-mode 1 --schedule-type 2 --process-mode 1 --process-dir 1
> 
> For all these parameters, I think better to use names (app/driver, etc.)
> instead of raw numbers.

[Anoob] Agreed. Will make this change when we submit the actual patches.

> 
> >
> > #Inbound app mode
> > ./ipsec-secgw -w 0002:02:00.0,nb_ipsec_in_sa=128 -w
> > 0002:03:00.0,nb_ipsec_in_sa=128 -w 0002:04:00.0,nb_ipsec_in_sa=128 -w
> > 0002:07:00.0,nb_ipsec_in_sa=128 -w 0002:0e:00.0 -w 0002:10:00.1
> > --log-level=8 -c 0x3f – -P -p 0xf --config
> > "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" - f dpdk_internal/100g_4.3.cfg
> > --transfer-mode 1 --schedule-type 2 --process-mode 0 --process-dir 1
> >
> > #Outbound driver mode
> > ./ipsec-secgw -w 0002:02:00.0 -w 0002:03:00.0 -w 0002:04:00.0 -w
> > 0002:07:00.0 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x1f – -
> > P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f
> > a-aes-gcm-new.cfg --transfer-mode 1 --schedule-type 2 --process-mode 1
> > --process-dir 0
> >
> > #Outbound app mode
> > ./ipsec-secgw -w 0002:02:00.0 -w 0002:03:00.0 -w 0002:04:00.0 -w
> > 0002:07:00.0 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x7f – -
> > P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f
> > a-aes-gcm-new.cfg --transfer-mode 1 --schedule-type 2 --process-mode 0
> > --process-dir 0
> >
> > This series is targeted for next release (20.02). This series doesn't
> > introduce any library change.
> 
> By 'library change' you mean that this new event-mode will be supported
> only by legacy code-path or ...?

[Anoob] All the changes are confined to 'examples/ipsec-secgw' directory. Right now, the worker threads make use of the existing routines in non-librte_ipsec mode. 

> 
> >And the decision to add eventmode additions in ipsec-secgw  was
> >approved by the Tech Board.
> >
> > Following are missing in the RFC. Will add it when sending patches.
> > 1. Documentation.
> > 2. More cleanup is needed. There are options that are added so that future
> >    expansion is not hindered. Need inputs from the community if there is
> use
> >    case for them.
> >
> > Following are planned features,
> > 1. Add burst mode workers.
> > 2. Add non tx internal port worker.
> > 3. Verify support for Rx core (the support is added but lack of h/w to
> verify).
> > 4. Add lookaside protocol support.
> >
> > Following are features that Marvell won't be attempting.
> > 1. Inline crypto support.
> > 2. Lookaside crypto support.
> 
> Ok so what mode is supported right now with this RFC?

[Anoob] Inline protocol support is added with the RFC.

> 
> >
> > For the features that Marvell won't be attempting, new workers can be
> > introduced by the respective stake holders.
> >
> > Anoob Joseph (13):
> >   examples/ipsec-secgw: add framework for eventmode helper
> >   examples/ipsec-secgw: add eventdev port-lcore link
> >   examples/ipsec-secgw: add Rx adapter support
> >   examples/ipsec-secgw: add Tx adapter support
> >   examples/ipsec-secgw: add routines to display config
> >   examples/ipsec-secgw: add routines to launch workers
> >   examples/ipsec-secgw: add support for internal ports
> >   examples/ipsec-secgw: add eventmode to ipsec-secgw
> >   examples/ipsec-secgw: add app inbound worker
> >   examples/ipsec-secgw: add app processing code
> >   examples/ipsec-secgw: add driver outbound worker
> >   examples/ipsec-secgw: add app outbound worker
> >   examples/ipsec-secgw: add cmd line option for bufs
> >
> >  examples/ipsec-secgw/Makefile       |    2 +
> >  examples/ipsec-secgw/event_helper.c | 1757
> > +++++++++++++++++++++++++++++++++++
> >  examples/ipsec-secgw/event_helper.h |  334 +++++++
> > examples/ipsec-secgw/ipsec-secgw.c  |  436 +++++++--
> >  examples/ipsec-secgw/ipsec-secgw.h  |   81 ++
> >  examples/ipsec-secgw/ipsec.c        |    4 +
> >  examples/ipsec-secgw/ipsec.h        |   30 +-
> >  examples/ipsec-secgw/ipsec_worker.c |  766 +++++++++++++++
> >  examples/ipsec-secgw/ipsec_worker.h |   39 +
> >  examples/ipsec-secgw/meson.build    |    4 +-
> >  examples/ipsec-secgw/sa.c           |   11 -
> >  11 files changed, 3360 insertions(+), 104 deletions(-)  create mode
> > 100644 examples/ipsec-secgw/event_helper.c
> >  create mode 100644 examples/ipsec-secgw/event_helper.h
> >  create mode 100644 examples/ipsec-secgw/ipsec-secgw.h
> >  create mode 100644 examples/ipsec-secgw/ipsec_worker.c
> >  create mode 100644 examples/ipsec-secgw/ipsec_worker.h
> >
> > --
> > 2.7.4
  

<snip>

> > >
> > > This series is targeted for next release (20.02). This series doesn't
> > > introduce any library change.
> >
> > By 'library change' you mean that this new event-mode will be supported
> > only by legacy code-path or ...?
> 
> [Anoob] All the changes are confined to 'examples/ipsec-secgw' directory. Right now, the worker threads make use of the existing
> routines in non-librte_ipsec mode.

And as I understand, you don't plan to use/support library mode?
I suppose I need to look to the actual code to understand more here.

> 
> >
> > >And the decision to add eventmode additions in ipsec-secgw  was
> > >approved by the Tech Board.
> > >
> > > Following are missing in the RFC. Will add it when sending patches.
> > > 1. Documentation.
> > > 2. More cleanup is needed. There are options that are added so that future
> > >    expansion is not hindered. Need inputs from the community if there is
> > use
> > >    case for them.
> > >
> > > Following are planned features,
> > > 1. Add burst mode workers.
> > > 2. Add non tx internal port worker.
> > > 3. Verify support for Rx core (the support is added but lack of h/w to
> > verify).
> > > 4. Add lookaside protocol support.
> > >
> > > Following are features that Marvell won't be attempting.
> > > 1. Inline crypto support.
> > > 2. Lookaside crypto support.
> >
> > Ok so what mode is supported right now with this RFC?
> 
> [Anoob] Inline protocol support is added with the RFC.

Ok..., but  do we have within DPDK any PMD that implements inline-proto?
AFAIK we don't, but might be I am missing something?

> 
> >
> > >
> > > For the features that Marvell won't be attempting, new workers can be
> > > introduced by the respective stake holders.
> > >
> > > Anoob Joseph (13):
> > >   examples/ipsec-secgw: add framework for eventmode helper
> > >   examples/ipsec-secgw: add eventdev port-lcore link
> > >   examples/ipsec-secgw: add Rx adapter support
> > >   examples/ipsec-secgw: add Tx adapter support
> > >   examples/ipsec-secgw: add routines to display config
> > >   examples/ipsec-secgw: add routines to launch workers
> > >   examples/ipsec-secgw: add support for internal ports
> > >   examples/ipsec-secgw: add eventmode to ipsec-secgw
> > >   examples/ipsec-secgw: add app inbound worker
> > >   examples/ipsec-secgw: add app processing code
> > >   examples/ipsec-secgw: add driver outbound worker
> > >   examples/ipsec-secgw: add app outbound worker
> > >   examples/ipsec-secgw: add cmd line option for bufs
> > >
> > >  examples/ipsec-secgw/Makefile       |    2 +
> > >  examples/ipsec-secgw/event_helper.c | 1757
> > > +++++++++++++++++++++++++++++++++++
> > >  examples/ipsec-secgw/event_helper.h |  334 +++++++
> > > examples/ipsec-secgw/ipsec-secgw.c  |  436 +++++++--
> > >  examples/ipsec-secgw/ipsec-secgw.h  |   81 ++
> > >  examples/ipsec-secgw/ipsec.c        |    4 +
> > >  examples/ipsec-secgw/ipsec.h        |   30 +-
> > >  examples/ipsec-secgw/ipsec_worker.c |  766 +++++++++++++++
> > >  examples/ipsec-secgw/ipsec_worker.h |   39 +
> > >  examples/ipsec-secgw/meson.build    |    4 +-
> > >  examples/ipsec-secgw/sa.c           |   11 -
> > >  11 files changed, 3360 insertions(+), 104 deletions(-)  create mode
> > > 100644 examples/ipsec-secgw/event_helper.c
> > >  create mode 100644 examples/ipsec-secgw/event_helper.h
> > >  create mode 100644 examples/ipsec-secgw/ipsec-secgw.h
> > >  create mode 100644 examples/ipsec-secgw/ipsec_worker.c
> > >  create mode 100644 examples/ipsec-secgw/ipsec_worker.h
> > >
> > > --
> > > 2.7.4
  

Hi Konstantin,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Friday, October 25, 2019 3:10 PM
> To: Anoob Joseph <anoobj@marvell.com>; Akhil Goyal
> <akhil.goyal@nxp.com>; Nicolau, Radu <radu.nicolau@intel.com>
> Cc: Thomas Monjalon <thomas@monjalon.net>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; Lukas Bartosik <lbartosik@marvell.com>;
> dev@dpdk.org
> Subject: [EXT] RE: [dpdk-dev] [RFC PATCH 00/13] add eventmode to ipsec-
> secgw
> 
> External Email
> 
> ----------------------------------------------------------------------
> 
> <snip>
> 
> > > >
> > > > This series is targeted for next release (20.02). This series
> > > > doesn't introduce any library change.
> > >
> > > By 'library change' you mean that this new event-mode will be
> > > supported only by legacy code-path or ...?
> >
> > [Anoob] All the changes are confined to 'examples/ipsec-secgw'
> > directory. Right now, the worker threads make use of the existing routines
> in non-librte_ipsec mode.
> 
> And as I understand, you don't plan to use/support library mode?
> I suppose I need to look to the actual code to understand more here.

[Anoob] Supporting library mode is also in the pipeline. But initial target would be support with non-librte_ipsec mode.
 
> 
> >
> > >
> > > >And the decision to add eventmode additions in ipsec-secgw  was
> > > >approved by the Tech Board.
> > > >
> > > > Following are missing in the RFC. Will add it when sending patches.
> > > > 1. Documentation.
> > > > 2. More cleanup is needed. There are options that are added so that
> future
> > > >    expansion is not hindered. Need inputs from the community if
> > > > there is
> > > use
> > > >    case for them.
> > > >
> > > > Following are planned features,
> > > > 1. Add burst mode workers.
> > > > 2. Add non tx internal port worker.
> > > > 3. Verify support for Rx core (the support is added but lack of
> > > > h/w to
> > > verify).
> > > > 4. Add lookaside protocol support.
> > > >
> > > > Following are features that Marvell won't be attempting.
> > > > 1. Inline crypto support.
> > > > 2. Lookaside crypto support.
> > >
> > > Ok so what mode is supported right now with this RFC?
> >
> > [Anoob] Inline protocol support is added with the RFC.
> 
> Ok..., but  do we have within DPDK any PMD that implements inline-proto?
> AFAIK we don't, but might be I am missing something?

[Anoob] Inline protocol support would be added to OCTEON TX2 eth PMD in the next cycle. This is an RFC to introduce the idea of using events in ipsec-segcw, which is required for OCTEON TX2 platform to perform inline ipsec processing.

> 
> >
> > >
> > > >
> > > > For the features that Marvell won't be attempting, new workers can
> > > > be introduced by the respective stake holders.
> > > >
> > > > Anoob Joseph (13):
> > > >   examples/ipsec-secgw: add framework for eventmode helper
> > > >   examples/ipsec-secgw: add eventdev port-lcore link
> > > >   examples/ipsec-secgw: add Rx adapter support
> > > >   examples/ipsec-secgw: add Tx adapter support
> > > >   examples/ipsec-secgw: add routines to display config
> > > >   examples/ipsec-secgw: add routines to launch workers
> > > >   examples/ipsec-secgw: add support for internal ports
> > > >   examples/ipsec-secgw: add eventmode to ipsec-secgw
> > > >   examples/ipsec-secgw: add app inbound worker
> > > >   examples/ipsec-secgw: add app processing code
> > > >   examples/ipsec-secgw: add driver outbound worker
> > > >   examples/ipsec-secgw: add app outbound worker
> > > >   examples/ipsec-secgw: add cmd line option for bufs
> > > >
> > > >  examples/ipsec-secgw/Makefile       |    2 +
> > > >  examples/ipsec-secgw/event_helper.c | 1757
> > > > +++++++++++++++++++++++++++++++++++
> > > >  examples/ipsec-secgw/event_helper.h |  334 +++++++
> > > > examples/ipsec-secgw/ipsec-secgw.c  |  436 +++++++--
> > > >  examples/ipsec-secgw/ipsec-secgw.h  |   81 ++
> > > >  examples/ipsec-secgw/ipsec.c        |    4 +
> > > >  examples/ipsec-secgw/ipsec.h        |   30 +-
> > > >  examples/ipsec-secgw/ipsec_worker.c |  766 +++++++++++++++
> > > >  examples/ipsec-secgw/ipsec_worker.h |   39 +
> > > >  examples/ipsec-secgw/meson.build    |    4 +-
> > > >  examples/ipsec-secgw/sa.c           |   11 -
> > > >  11 files changed, 3360 insertions(+), 104 deletions(-)  create
> > > > mode
> > > > 100644 examples/ipsec-secgw/event_helper.c
> > > >  create mode 100644 examples/ipsec-secgw/event_helper.h
> > > >  create mode 100644 examples/ipsec-secgw/ipsec-secgw.h
> > > >  create mode 100644 examples/ipsec-secgw/ipsec_worker.c
> > > >  create mode 100644 examples/ipsec-secgw/ipsec_worker.h
> > > >
> > > > --
> > > > 2.7.4