[v10,0/4] cryptodev: add raw data-path APIs

78756 mbox series

The Crypto Raw data-path APIs are a set of APIs are designed to enable
externel libraries/applications which want to leverage the cryptographic
processing provided by DPDK crypto PMDs through the cryptodev API but in a
manner that is not dependent on native DPDK data structures (eg. rte_mbuf,
rte_crypto_op, ... etc) in their data-path implementation.

The raw data-path APIs have the following advantages:
- External data structure friendly design. The new APIs uses the operation
descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and
IOVA addresses as input. Moreover, the APIs does not require the user to
allocate the descriptor from mempool, nor requiring mbufs to describe input
data's virtual and IOVA addresses. All these features made the translation
from user's own data structure into the descriptor easier and more efficient.
- Flexible enqueue and dequeue operation. The raw data-path APIs gives the
user more control to the enqueue and dequeue operations, including the
capability of precious enqueue/dequeue count, abandoning enqueue or dequeue
at any time, and operation status translation and set on the fly.

V10:
- Changed rte_crypto_sym_vec structure to support both sync cpu_crypto and
  async raw data-path API.
- Changed documentation.
- Changed API names.
- Changed the way data-path context is initialized.
- Added new API to attach session or xform to existing context.
- Changed QAT PMD accordingly with new APIs.
- Changed unit test to use the device feature flag for the raw API tests.

v9:
- Changed return types of submit_done() and dequeue_done() APIs.
- Added release note update. 

v8:
- Updated following by comments.
- Fixed a few bugs.
- Fixed ARM build error.
- Updated the unit test covering all tests.

v7:
- Fixed a few typos.
- Fixed length calculation bugs.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Fan Zhang (4):
  cryptodev: change crypto symmetric vector structure
  cryptodev: add raw crypto data-path APIs
  crypto/qat: add raw crypto data-path API support
  test/crypto: add unit-test for cryptodev raw API test

 app/test/test_cryptodev.c                     | 784 ++++++++++++++-
 app/test/test_cryptodev.h                     |  12 +
 app/test/test_cryptodev_blockcipher.c         |  58 +-
 doc/guides/prog_guide/cryptodev_lib.rst       |  96 +-
 doc/guides/rel_notes/release_20_11.rst        |  10 +
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c      |  18 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    |   9 +-
 drivers/crypto/qat/meson.build                |   1 +
 drivers/crypto/qat/qat_sym.h                  |  11 +
 drivers/crypto/qat/qat_sym_hw_dp.c            | 951 ++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c              |   9 +-
 lib/librte_cryptodev/rte_crypto_sym.h         |  40 +-
 lib/librte_cryptodev/rte_cryptodev.c          | 104 ++
 lib/librte_cryptodev/rte_cryptodev.h          | 354 ++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 +-
 .../rte_cryptodev_version.map                 |  11 +
 lib/librte_ipsec/esp_inb.c                    |  12 +-
 lib/librte_ipsec/esp_outb.c                   |  12 +-
 lib/librte_ipsec/misc.h                       |   6 +-
 19 files changed, 2437 insertions(+), 108 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c

Hi Fan,

> The Crypto Raw data-path APIs are a set of APIs are designed to enable
> externel libraries/applications which want to leverage the cryptographic
> processing provided by DPDK crypto PMDs through the cryptodev API but in a
> manner that is not dependent on native DPDK data structures (eg. rte_mbuf,
> rte_crypto_op, ... etc) in their data-path implementation.
> 
> The raw data-path APIs have the following advantages:
> - External data structure friendly design. The new APIs uses the operation
> descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and
> IOVA addresses as input. Moreover, the APIs does not require the user to
> allocate the descriptor from mempool, nor requiring mbufs to describe input
> data's virtual and IOVA addresses. All these features made the translation
> from user's own data structure into the descriptor easier and more efficient.
> - Flexible enqueue and dequeue operation. The raw data-path APIs gives the
> user more control to the enqueue and dequeue operations, including the
> capability of precious enqueue/dequeue count, abandoning enqueue or dequeue
> at any time, and operation status translation and set on the fly.
> 
> V10:
> - Changed rte_crypto_sym_vec structure to support both sync cpu_crypto and
>   async raw data-path API.
> - Changed documentation.
> - Changed API names.
> - Changed the way data-path context is initialized.
> - Added new API to attach session or xform to existing context.
> - Changed QAT PMD accordingly with new APIs.
> - Changed unit test to use the device feature flag for the raw API tests.
> 
I have some comment on the V10, please revert asap so that these can be merged in RC1.

@konstantin.ananyev@intel.com: Could you please have a look at the 2/4 patch as well?

Regards,
Akhil

Ok. Will do ASAP.

> -----Original Message-----
> From: Akhil Goyal <akhil.goyal@nxp.com>
> Sent: Thursday, October 8, 2020 4:04 PM
> To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; dev@dpdk.org; Ananyev,
> Konstantin <konstantin.ananyev@intel.com>
> Cc: Trahe, Fiona <fiona.trahe@intel.com>; Kusztal, ArkadiuszX
> <arkadiuszx.kusztal@intel.com>; Dybkowski, AdamX
> <adamx.dybkowski@intel.com>; anoobj@marvell.com
> Subject: RE: [dpdk-dev v10 0/4] cryptodev: add raw data-path APIs
> 
> Hi Fan,
> 
> > The Crypto Raw data-path APIs are a set of APIs are designed to enable
> > externel libraries/applications which want to leverage the cryptographic
> > processing provided by DPDK crypto PMDs through the cryptodev API but
> in a
> > manner that is not dependent on native DPDK data structures (eg.
> rte_mbuf,
> > rte_crypto_op, ... etc) in their data-path implementation.
> >
> > The raw data-path APIs have the following advantages:
> > - External data structure friendly design. The new APIs uses the operation
> > descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer
> and
> > IOVA addresses as input. Moreover, the APIs does not require the user to
> > allocate the descriptor from mempool, nor requiring mbufs to describe
> input
> > data's virtual and IOVA addresses. All these features made the translation
> > from user's own data structure into the descriptor easier and more
> efficient.
> > - Flexible enqueue and dequeue operation. The raw data-path APIs gives
> the
> > user more control to the enqueue and dequeue operations, including the
> > capability of precious enqueue/dequeue count, abandoning enqueue or
> dequeue
> > at any time, and operation status translation and set on the fly.
> >
> > V10:
> > - Changed rte_crypto_sym_vec structure to support both sync cpu_crypto
> and
> >   async raw data-path API.
> > - Changed documentation.
> > - Changed API names.
> > - Changed the way data-path context is initialized.
> > - Added new API to attach session or xform to existing context.
> > - Changed QAT PMD accordingly with new APIs.
> > - Changed unit test to use the device feature flag for the raw API tests.
> >
> I have some comment on the V10, please revert asap so that these can be
> merged in RC1.
> 
> @konstantin.ananyev@intel.com: Could you please have a look at the 2/4
> patch as well?
> 
> Regards,
> Akhil