From patchwork Mon Aug 8 08:05:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114691 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B0D11A034C; Mon, 8 Aug 2022 10:06:18 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3FDAE42B79; Mon, 8 Aug 2022 10:06:13 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 2245342B77 for ; Mon, 8 Aug 2022 10:06:12 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277MwkKn002933 for ; Mon, 8 Aug 2022 01:06:11 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=OnIKocMc22pkSG+IkxP5G4WyVzBqGbZQm5JJM091grk=; b=GlDWUbht7vB5nElVW1SHjzcCLxRwsi15c2Wd3KBPmsj0VqrXn1b9cWMVdNKiPJFbOvMY PgmANNnPiv2hiDoRdkg292BPZ1PIqx0wGWc7NbRMxmqAOL97KBKStJ8msy3KSgG3M6jq sF378BE5syA/YtVxO1KPB+j3QmU8OsM2RVAKRSo4VfC+5qYBkfrnFC+h+uumv8tOTFp0 pQH+qzBiqAY6Ihe9elpsDxUThhukk6lGIEDKU35dygvmvl4MwntPy/KvqoyFgilyR/pv g1u4t8kHZLyBY9PebbwvpM74Iki+puH6qgbuMaC+MruE7VqJiU9S3qU8pN7kqEU/Wl6C iA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxdq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:11 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Aug 2022 01:06:09 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Mon, 8 Aug 2022 01:06:09 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id 2E1C03F7043; Mon, 8 Aug 2022 01:06:06 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 01/18] crypto/cnxk: add AES-CCM support Date: Mon, 8 Aug 2022 13:35:49 +0530 Message-ID: <20220808080606.220-2-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: fxYBbm1aNGEWbobCbCLrb68A6q4P3ycM X-Proofpoint-ORIG-GUID: fxYBbm1aNGEWbobCbCLrb68A6q4P3ycM X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Archana Muniganti Add lookaside IPsec AES-CCM support in CN9K & CN10K PMDs. Signed-off-by: Archana Muniganti --- doc/guides/rel_notes/release_22_11.rst | 4 ++ drivers/common/cnxk/cnxk_security.c | 38 ++++++++++++-- drivers/common/cnxk/roc_cpt.h | 13 ++--- drivers/crypto/cnxk/cn10k_ipsec_la_ops.h | 1 + drivers/crypto/cnxk/cnxk_cryptodev.h | 2 +- .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 49 ++++++++++++++++--- drivers/crypto/cnxk/cnxk_ipsec.h | 3 +- 7 files changed, 93 insertions(+), 17 deletions(-) diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index 8c021cf050..333f66bef3 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -55,6 +55,10 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Updated Marvell cnxk crypto driver.** + + * Added AES-CCM support in lookaside protocol (IPsec) for CN9K & CN10K. + Removed Items ------------- diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c index dca8742be3..8a0e4dea4c 100644 --- a/drivers/common/cnxk/cnxk_security.c +++ b/drivers/common/cnxk/cnxk_security.c @@ -58,6 +58,7 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, { struct rte_crypto_sym_xform *auth_xfrm, *cipher_xfrm; const uint8_t *key = NULL; + uint8_t ccm_flag = 0; uint32_t *tmp_salt; uint64_t *tmp_key; int i, length = 0; @@ -113,6 +114,15 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, tmp_salt = (uint32_t *)salt_key; *tmp_salt = rte_be_to_cpu_32(*tmp_salt); break; + case RTE_CRYPTO_AEAD_AES_CCM: + w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CCM; + w2->s.auth_type = ROC_IE_OT_SA_AUTH_NULL; + ccm_flag = 0x07 & ~ROC_CPT_AES_CCM_CTR_LEN; + *salt_key = ccm_flag; + memcpy(PLT_PTR_ADD(salt_key, 1), &ipsec_xfrm->salt, 3); + tmp_salt = (uint32_t *)salt_key; + *tmp_salt = rte_be_to_cpu_32(*tmp_salt); + break; default: return -ENOTSUP; } @@ -204,6 +214,7 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CCM || w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CTR || w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_GCM || + w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CCM || w2->s.auth_type == ROC_IE_OT_SA_AUTH_AES_GMAC) { switch (length) { case ROC_CPT_AES128_KEY_LEN: @@ -612,6 +623,7 @@ onf_ipsec_sa_common_param_fill(struct roc_ie_onf_sa_ctl *ctl, uint8_t *salt, struct rte_crypto_sym_xform *auth_xfrm, *cipher_xfrm; int rc, length, auth_key_len; const uint8_t *key = NULL; + uint8_t ccm_flag = 0; /* Set direction */ switch (ipsec_xfrm->direction) { @@ -663,6 +675,14 @@ onf_ipsec_sa_common_param_fill(struct roc_ie_onf_sa_ctl *ctl, uint8_t *salt, memcpy(salt, &ipsec_xfrm->salt, 4); key = crypto_xfrm->aead.key.data; break; + case RTE_CRYPTO_AEAD_AES_CCM: + ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CCM; + ctl->auth_type = ROC_IE_ON_SA_AUTH_NULL; + ccm_flag = 0x07 & ~ROC_CPT_AES_CCM_CTR_LEN; + *salt = ccm_flag; + memcpy(PLT_PTR_ADD(salt, 1), &ipsec_xfrm->salt, 3); + key = crypto_xfrm->aead.key.data; + break; default: return -ENOTSUP; } @@ -810,7 +830,7 @@ cnxk_ipsec_ivlen_get(enum rte_crypto_cipher_algorithm c_algo, { uint8_t ivlen = 0; - if (aead_algo == RTE_CRYPTO_AEAD_AES_GCM) + if ((aead_algo == RTE_CRYPTO_AEAD_AES_GCM) || (aead_algo == RTE_CRYPTO_AEAD_AES_CCM)) ivlen = 8; switch (c_algo) { @@ -873,6 +893,7 @@ cnxk_ipsec_icvlen_get(enum rte_crypto_cipher_algorithm c_algo, switch (aead_algo) { case RTE_CRYPTO_AEAD_AES_GCM: + case RTE_CRYPTO_AEAD_AES_CCM: icv = 16; break; default: @@ -888,7 +909,7 @@ cnxk_ipsec_outb_roundup_byte(enum rte_crypto_cipher_algorithm c_algo, { uint8_t roundup_byte = 4; - if (aead_algo == RTE_CRYPTO_AEAD_AES_GCM) + if ((aead_algo == RTE_CRYPTO_AEAD_AES_GCM) || (aead_algo == RTE_CRYPTO_AEAD_AES_CCM)) return roundup_byte; switch (c_algo) { @@ -1023,6 +1044,10 @@ on_ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec, ctl->enc_type = ROC_IE_ON_SA_ENC_AES_GCM; aes_key_len = crypto_xform->aead.key.length; break; + case RTE_CRYPTO_AEAD_AES_CCM: + ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CCM; + aes_key_len = crypto_xform->aead.key.length; + break; default: plt_err("Unsupported AEAD algorithm"); return -ENOTSUP; @@ -1087,6 +1112,7 @@ on_ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec, ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CCM || ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CTR || ctl->enc_type == ROC_IE_ON_SA_ENC_AES_GCM || + ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CCM || ctl->auth_type == ROC_IE_ON_SA_AUTH_AES_GMAC) { switch (aes_key_len) { case 16: @@ -1129,6 +1155,7 @@ on_fill_ipsec_common_sa(struct rte_security_ipsec_xform *ipsec, struct rte_crypto_sym_xform *cipher_xform, *auth_xform; const uint8_t *cipher_key; int cipher_key_len = 0; + uint8_t ccm_flag = 0; int ret; ret = on_ipsec_sa_ctl_set(ipsec, crypto_xform, &common_sa->ctl); @@ -1146,6 +1173,11 @@ on_fill_ipsec_common_sa(struct rte_security_ipsec_xform *ipsec, if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) { if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) memcpy(common_sa->iv.gcm.nonce, &ipsec->salt, 4); + else if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_CCM) { + ccm_flag = 0x07 & ~ROC_CPT_AES_CCM_CTR_LEN; + *common_sa->iv.gcm.nonce = ccm_flag; + memcpy(PLT_PTR_ADD(common_sa->iv.gcm.nonce, 1), &ipsec->salt, 3); + } cipher_key = crypto_xform->aead.key.data; cipher_key_len = crypto_xform->aead.key.length; } else { @@ -1194,7 +1226,7 @@ cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipsec, return ret; if (ctl->enc_type == ROC_IE_ON_SA_ENC_AES_GCM || - ctl->auth_type == ROC_IE_ON_SA_AUTH_NULL || + ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CCM || ctl->auth_type == ROC_IE_ON_SA_AUTH_NULL || ctl->auth_type == ROC_IE_ON_SA_AUTH_AES_GMAC) { template = &out_sa->aes_gcm.template; ctx_len = offsetof(struct roc_ie_on_outb_sa, aes_gcm.template); diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h index a3a65f1e94..0cebc05c74 100644 --- a/drivers/common/cnxk/roc_cpt.h +++ b/drivers/common/cnxk/roc_cpt.h @@ -43,12 +43,13 @@ ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 14)) /* CPT helper macros */ -#define ROC_CPT_AH_HDR_LEN 12 -#define ROC_CPT_AES_GCM_IV_LEN 8 -#define ROC_CPT_AES_GCM_MAC_LEN 16 -#define ROC_CPT_AES_CBC_IV_LEN 16 -#define ROC_CPT_SHA1_HMAC_LEN 12 -#define ROC_CPT_SHA2_HMAC_LEN 16 +#define ROC_CPT_AH_HDR_LEN 12 +#define ROC_CPT_AES_GCM_IV_LEN 8 +#define ROC_CPT_AES_GCM_MAC_LEN 16 +#define ROC_CPT_AES_CCM_CTR_LEN 4 +#define ROC_CPT_AES_CBC_IV_LEN 16 +#define ROC_CPT_SHA1_HMAC_LEN 12 +#define ROC_CPT_SHA2_HMAC_LEN 16 #define ROC_CPT_DES3_KEY_LEN 24 #define ROC_CPT_AES128_KEY_LEN 16 diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h index 66cfe6ca98..e220863799 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h +++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h @@ -66,6 +66,7 @@ process_outb_sa(struct roc_cpt_lf *lf, struct rte_crypto_op *cop, #ifdef LA_IPSEC_DEBUG if (sess->out_sa.w2.s.iv_src == ROC_IE_OT_SA_IV_SRC_FROM_SA) { if (sess->out_sa.w2.s.enc_type == ROC_IE_OT_SA_ENC_AES_GCM || + sess->out_sa.w2.s.enc_type == ROC_IE_OT_SA_ENC_AES_CCM || sess->out_sa.w2.s.auth_type == ROC_IE_OT_SA_AUTH_AES_GMAC) ipsec_po_sa_aes_gcm_iv_set(sess, cop); else diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index 8870021725..a3dcfbfa6d 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -11,7 +11,7 @@ #include "roc_cpt.h" #define CNXK_CPT_MAX_CAPS 35 -#define CNXK_SEC_CRYPTO_MAX_CAPS 13 +#define CNXK_SEC_CRYPTO_MAX_CAPS 14 #define CNXK_SEC_MAX_CAPS 9 #define CNXK_AE_EC_ID_MAX 8 /** diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index 705d67e91f..fdc646a6fc 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -775,6 +775,36 @@ static const struct rte_cryptodev_capabilities sec_caps_aes[] = { }, } }, } }, + { /* AES CCM */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD, + {.aead = { + .algo = RTE_CRYPTO_AEAD_AES_CCM, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .digest_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .aad_size = { + .min = 8, + .max = 12, + .increment = 4 + }, + .iv_size = { + .min = 12, + .max = 12, + .increment = 0 + } + }, } + }, } + }, { /* AES CTR */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, {.sym = { @@ -1155,14 +1185,23 @@ cnxk_crypto_capabilities_get(struct cnxk_cpt_vf *vf) return vf->crypto_caps; } +static bool +sec_caps_limit_check(int *cur_pos, int nb_caps) +{ + if (*cur_pos + nb_caps > CNXK_SEC_CRYPTO_MAX_CAPS) { + rte_panic("Could not add sec crypto caps"); + return true; + } + + return false; +} + static void sec_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos, const struct rte_cryptodev_capabilities *caps, int nb_caps) { - if (*cur_pos + nb_caps > CNXK_SEC_CRYPTO_MAX_CAPS) { - rte_panic("Could not add sec crypto caps"); + if (sec_caps_limit_check(cur_pos, nb_caps)) return; - } memcpy(&cnxk_caps[*cur_pos], caps, nb_caps * sizeof(caps[0])); *cur_pos += nb_caps; @@ -1175,10 +1214,8 @@ cn10k_sec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[], const struct rte_cryptodev_capabilities *cap; unsigned int i; - if ((CNXK_SEC_CRYPTO_MAX_CAPS - *cur_pos) < 1) { - rte_panic("Could not add sec crypto caps"); + if (sec_caps_limit_check(cur_pos, 1)) return; - } /* NULL auth */ for (i = 0; i < RTE_DIM(caps_null); i++) { diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h index 07ab2cf4ee..00873ca6ac 100644 --- a/drivers/crypto/cnxk/cnxk_ipsec.h +++ b/drivers/crypto/cnxk/cnxk_ipsec.h @@ -87,7 +87,8 @@ ipsec_xform_aead_verify(struct rte_security_ipsec_xform *ipsec_xform, crypto_xform->aead.op != RTE_CRYPTO_AEAD_OP_DECRYPT) return -EINVAL; - if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) { + if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM || + crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_CCM) { switch (crypto_xform->aead.key.length) { case 16: case 24: From patchwork Mon Aug 8 08:05:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114692 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D6288A034C; Mon, 8 Aug 2022 10:06:25 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7041042B83; Mon, 8 Aug 2022 10:06:15 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 7365C42B83 for ; Mon, 8 Aug 2022 10:06:14 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277N0NGU005827 for ; Mon, 8 Aug 2022 01:06:13 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=2sEp2bU0Gn60Tb6nO1uddLqqiCn1mbPS5Z3qmxYEUY0=; b=Iqlrv2lhE6FqWkIJ82LWsLwqsrjeBxAtBBa3MBM2fsIVrWE1QMn7zn8aWEQ3uDfQZ4ZX 1+XN1Ia/fpTM5gOnS4SaW2Lp7tLqtYiawdR04aQFGPB0mnKaIlM72veRNAqj2XxjA7Qt +PdtyEWCuj1JRtvGi3IAgQEdFa2hvBge1IE1yvUNW0R8DxQublAZ2wTjTwZ+P3h54S9Z t1pwmoYqAUF3/XK9rUOM3XtP3KPm21VxjLEzyrK5B0pi2hEKj+WBGkQsYMHfWNwf1wJ7 pgORtxWu6zwsBOeHVT15fP/UqDSpyOllnQ9OOrPAqFiZMGPst9M978Fg8Bk7IV94fEyv yw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxdv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:13 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 8 Aug 2022 01:06:11 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Mon, 8 Aug 2022 01:06:11 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id 9CAD63F7059; Mon, 8 Aug 2022 01:06:09 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Volodymyr Fialko , Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 02/18] crypto/cnxk: add burst enqueue for event crypto Date: Mon, 8 Aug 2022 13:35:50 +0530 Message-ID: <20220808080606.220-3-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: AKGo1LjXxyvSANlSB574oaj5jXFASd7N X-Proofpoint-ORIG-GUID: AKGo1LjXxyvSANlSB574oaj5jXFASd7N X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko Added support for burst enqueue for cn10k event crypto adapter. Instruction will be grouped based on the queue pair and sent in a burst. Signed-off-by: Volodymyr Fialko --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 141 +++++++++++++++------- drivers/crypto/cnxk/cn10k_cryptodev_ops.h | 7 +- drivers/crypto/cnxk/meson.build | 2 +- drivers/event/cnxk/cn10k_eventdev.c | 2 +- drivers/event/cnxk/cn10k_worker.c | 10 -- drivers/event/cnxk/cn10k_worker.h | 2 - 6 files changed, 105 insertions(+), 59 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index f761ba36e2..bfa6374005 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -9,11 +9,12 @@ #include "cn10k_cryptodev.h" #include "cn10k_cryptodev_ops.h" -#include "cn10k_ipsec_la_ops.h" #include "cn10k_ipsec.h" +#include "cn10k_ipsec_la_ops.h" #include "cnxk_ae.h" #include "cnxk_cryptodev.h" #include "cnxk_cryptodev_ops.h" +#include "cnxk_eventdev.h" #include "cnxk_se.h" #include "roc_api.h" @@ -391,79 +392,135 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, return 0; } -uint16_t -cn10k_cpt_crypto_adapter_enqueue(uintptr_t base, struct rte_crypto_op *op) +static inline uint16_t +ca_lmtst_burst_submit(struct cn10k_sso_hws *ws, uint64_t w2[], struct cnxk_cpt_qp *qp, + struct rte_crypto_op *op[], uint16_t nb_ops) { + struct cpt_inflight_req *infl_reqs[PKTS_PER_LOOP]; + uint64_t lmt_base, lmt_arg, io_addr; + struct cpt_inst_s *inst, *inst_base; struct cpt_inflight_req *infl_req; - uint64_t lmt_base, lmt_arg, w2; - struct cpt_inst_s *inst; union cpt_fc_write_s fc; - struct cnxk_cpt_qp *qp; uint64_t *fc_addr; uint16_t lmt_id; - int ret; + int ret, i; - ret = cn10k_ca_meta_info_extract(op, &qp, &w2); - if (unlikely(ret)) { - rte_errno = EINVAL; - return 0; - } + lmt_base = qp->lmtline.lmt_base; + io_addr = qp->lmtline.io_addr; + fc_addr = qp->lmtline.fc_addr; + + const uint32_t fc_thresh = qp->lmtline.fc_thresh; + + ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); + inst_base = (struct cpt_inst_s *)lmt_base; if (unlikely(!qp->ca.enabled)) { rte_errno = EINVAL; return 0; } - if (unlikely(rte_mempool_get(qp->ca.req_mp, (void **)&infl_req))) { + if (unlikely(rte_mempool_get_bulk(qp->ca.req_mp, (void **)infl_reqs, nb_ops))) { rte_errno = ENOMEM; return 0; } - infl_req->op_flags = 0; - - lmt_base = qp->lmtline.lmt_base; - fc_addr = qp->lmtline.fc_addr; - const uint32_t fc_thresh = qp->lmtline.fc_thresh; + for (i = 0; i < nb_ops; i++) { + inst = &inst_base[2 * i]; + infl_req = infl_reqs[i]; + infl_req->op_flags = 0; - ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); - inst = (struct cpt_inst_s *)lmt_base; + ret = cn10k_cpt_fill_inst(qp, &op[i], inst, infl_req); + if (unlikely(ret != 1)) { + plt_dp_err("Could not process op: %p", op[i]); + if (i != 0) + goto submit; + else + goto put; + } - ret = cn10k_cpt_fill_inst(qp, &op, inst, infl_req); - if (unlikely(ret != 1)) { - plt_dp_err("Could not process op: %p", op); - rte_mempool_put(qp->ca.req_mp, infl_req); - return 0; + infl_req->res.cn10k.compcode = CPT_COMP_NOT_DONE; + infl_req->qp = qp; + inst->w0.u64 = 0; + inst->res_addr = (uint64_t)&infl_req->res; + inst->w2.u64 = w2[i]; + inst->w3.u64 = CNXK_CPT_INST_W3(1, infl_req); } - infl_req->cop = op; - infl_req->res.cn10k.compcode = CPT_COMP_NOT_DONE; - infl_req->qp = qp; - inst->w0.u64 = 0; - inst->res_addr = (uint64_t)&infl_req->res; - inst->w2.u64 = w2; - inst->w3.u64 = CNXK_CPT_INST_W3(1, infl_req); - fc.u64[0] = __atomic_load_n(fc_addr, __ATOMIC_RELAXED); if (unlikely(fc.s.qsize > fc_thresh)) { - rte_mempool_put(qp->ca.req_mp, infl_req); rte_errno = EAGAIN; - return 0; + i = 0; + goto put; } - if (inst->w2.s.tt == RTE_SCHED_TYPE_ORDERED) - roc_sso_hws_head_wait(base); +submit: + if (CNXK_TT_FROM_TAG(ws->gw_rdata) == SSO_TT_ORDERED) + roc_sso_hws_head_wait(ws->base); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, qp->lmtline.io_addr); + if (i > PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (PKTS_PER_STEORL - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, io_addr); + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + PKTS_PER_STEORL); + roc_lmt_submit_steorl(lmt_arg, io_addr); + } else { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, io_addr); + } rte_io_wmb(); - return 1; +put: + if (unlikely(i != nb_ops)) + rte_mempool_put_bulk(qp->ca.req_mp, (void *)&infl_reqs[i], nb_ops - i); + + return i; +} + +uint16_t __rte_hot +cn10k_cpt_crypto_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_events) +{ + struct rte_crypto_op *ops[PKTS_PER_LOOP], *op; + struct cnxk_cpt_qp *qp, *curr_qp = NULL; + uint64_t w2s[PKTS_PER_LOOP], w2; + uint16_t submitted, count = 0; + int ret, i, ops_len = 0; + + for (i = 0; i < nb_events; i++) { + op = ev[i].event_ptr; + ret = cn10k_ca_meta_info_extract(op, &qp, &w2); + if (unlikely(ret)) { + rte_errno = EINVAL; + return count; + } + + if (qp != curr_qp) { + if (ops_len) { + submitted = ca_lmtst_burst_submit(ws, w2s, curr_qp, ops, ops_len); + count += submitted; + if (unlikely(submitted != ops_len)) + return count; + ops_len = 0; + } + curr_qp = qp; + } + w2s[ops_len] = w2; + ops[ops_len] = op; + if (++ops_len == PKTS_PER_LOOP) { + submitted = ca_lmtst_burst_submit(ws, w2s, curr_qp, ops, ops_len); + count += submitted; + if (unlikely(submitted != ops_len)) + return count; + ops_len = 0; + } + } + if (ops_len) + count += ca_lmtst_burst_submit(ws, w2s, curr_qp, ops, ops_len); + return count; } static inline void -cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, - struct cpt_cn10k_res_s *res) +cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *res) { struct rte_mbuf *mbuf = cop->sym->m_src; const uint16_t m_len = res->rlen; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h index 1ad4c16873..628d6a567c 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h @@ -5,16 +5,17 @@ #ifndef _CN10K_CRYPTODEV_OPS_H_ #define _CN10K_CRYPTODEV_OPS_H_ -#include #include +#include +#include extern struct rte_cryptodev_ops cn10k_cpt_ops; void cn10k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev); __rte_internal -uint16_t cn10k_cpt_crypto_adapter_enqueue(uintptr_t base, - struct rte_crypto_op *op); +uint16_t __rte_hot cn10k_cpt_crypto_adapter_enqueue(void *ws, struct rte_event ev[], + uint16_t nb_events); __rte_internal uintptr_t cn10k_cpt_crypto_adapter_dequeue(uintptr_t get_work1); diff --git a/drivers/crypto/cnxk/meson.build b/drivers/crypto/cnxk/meson.build index 23a1cc3aac..952554ac12 100644 --- a/drivers/crypto/cnxk/meson.build +++ b/drivers/crypto/cnxk/meson.build @@ -24,7 +24,7 @@ sources = files( deps += ['bus_pci', 'common_cnxk', 'security', 'eventdev'] -includes += include_directories('../../../lib/net') +includes += include_directories('../../../lib/net', '../../event/cnxk') if get_option('buildtype').contains('debug') cflags += [ '-DLA_IPSEC_DEBUG' ] diff --git a/drivers/event/cnxk/cn10k_eventdev.c b/drivers/event/cnxk/cn10k_eventdev.c index 5a0cab40a9..25c85902d6 100644 --- a/drivers/event/cnxk/cn10k_eventdev.c +++ b/drivers/event/cnxk/cn10k_eventdev.c @@ -454,7 +454,7 @@ cn10k_sso_fp_fns_set(struct rte_eventdev *event_dev) sso_hws_deq_tmo_ca_burst); } } - event_dev->ca_enqueue = cn10k_sso_hws_ca_enq; + event_dev->ca_enqueue = cn10k_cpt_crypto_adapter_enqueue; if (dev->tx_offloads & NIX_TX_MULTI_SEG_F) CN10K_SET_EVDEV_ENQ_OP(dev, event_dev->txa_enqueue, diff --git a/drivers/event/cnxk/cn10k_worker.c b/drivers/event/cnxk/cn10k_worker.c index f953e19dd0..4581c41233 100644 --- a/drivers/event/cnxk/cn10k_worker.c +++ b/drivers/event/cnxk/cn10k_worker.c @@ -64,13 +64,3 @@ cn10k_sso_hws_enq_fwd_burst(void *port, const struct rte_event ev[], return 1; } - -uint16_t __rte_hot -cn10k_sso_hws_ca_enq(void *port, struct rte_event ev[], uint16_t nb_events) -{ - struct cn10k_sso_hws *ws = port; - - RTE_SET_USED(nb_events); - - return cn10k_cpt_crypto_adapter_enqueue(ws->base, ev->event_ptr); -} diff --git a/drivers/event/cnxk/cn10k_worker.h b/drivers/event/cnxk/cn10k_worker.h index 0915f404e0..65bb08c0a1 100644 --- a/drivers/event/cnxk/cn10k_worker.h +++ b/drivers/event/cnxk/cn10k_worker.h @@ -353,8 +353,6 @@ uint16_t __rte_hot cn10k_sso_hws_enq_new_burst(void *port, uint16_t __rte_hot cn10k_sso_hws_enq_fwd_burst(void *port, const struct rte_event ev[], uint16_t nb_events); -uint16_t __rte_hot cn10k_sso_hws_ca_enq(void *port, struct rte_event ev[], - uint16_t nb_events); #define R(name, flags) \ uint16_t __rte_hot cn10k_sso_hws_deq_##name( \ From patchwork Mon Aug 8 08:05:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114693 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B7D82A034C; Mon, 8 Aug 2022 10:06:32 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 63CB942B8D; Mon, 8 Aug 2022 10:06:18 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id DB67742B71 for ; Mon, 8 Aug 2022 10:06:16 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277MmMps011992 for ; Mon, 8 Aug 2022 01:06:16 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=pAKCv13jG+RVF+gHIWW5q0DCH42DvAMXpP1xMC5zYZU=; b=cEjdX+0djO9HQrM+/ZlaVDr50RBtJl1NqyJifL7X5jZ8sdZ63YYS4uN6HevOwYoth0Ya StqVBoRyrKna6dj3yZ7q4jItvopgzREDq1bDZsKu1+hCsRMxmUgb0O6ddsXvknNDSt+S SrmURdWdlxEu0j6bjX5QjNMmZofSDrFff13r1MZJCoPw5vpu61iq/50ZdkjWEGWX7eV0 fPG7ucj1oFM/Nzo99IjiL8YDHGnmbNj1xjSs+ZDmttNO+YYTQ60VRC1+RZIB1pCz0qWG lo00/erNJPPeh23KZtPtXADK4NFDTqCD9OCZzKCSzWAEQXFNPXMhhg/lRhbdY9zJaO4G Ww== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxdy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:16 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 8 Aug 2022 01:06:14 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Mon, 8 Aug 2022 01:06:14 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id 652663F7043; Mon, 8 Aug 2022 01:06:12 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 03/18] crypto/cnxk: remove zero IV Date: Mon, 8 Aug 2022 13:35:51 +0530 Message-ID: <20220808080606.220-4-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: QyuDlkueXKaxvqTtWmj_ZmuKhDVfFBZY X-Proofpoint-ORIG-GUID: QyuDlkueXKaxvqTtWmj_ZmuKhDVfFBZY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Zeroising AES-CMAC IV would be done in microcode. Extra clearing in DPDK is not required. Signed-off-by: Anoob Joseph Signed-off-by: Tejasree Kondoj --- drivers/crypto/cnxk/cnxk_se.h | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index a75003f2c6..5574dbfa04 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -91,10 +91,6 @@ pdcp_iv_copy(uint8_t *iv_d, uint8_t *iv_s, const uint8_t pdcp_alg_type, memcpy(iv_d + 6, iv_s + 8, 17); } else memcpy(iv_d, iv_s, 16); - } else { - /* AES-CMAC EIA2, microcode expects 16B zeroized IV */ - for (j = 0; j < 16; j++) - iv_d[j] = 0; } } From patchwork Mon Aug 8 08:05:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114694 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CDF2BA034C; Mon, 8 Aug 2022 10:06:41 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C7B5042B8B; Mon, 8 Aug 2022 10:06:23 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id B624242B8B for ; Mon, 8 Aug 2022 10:06:21 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277MwkKo002933 for ; Mon, 8 Aug 2022 01:06:21 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=YgDaChHrDnDLfna2QxKdzmLTlajZWmcnGxvxA4+PZ4k=; b=OFIBw+DnFZP2ydWcDuNF2h7Fv+r4Wr4+Ml5hkYJFqEL5CJOpvRSCF4GQgdk5PdNyMNM8 7bKBhqWxxGDanXY/SyF97gcLLypxaPLkjwRlEespspBjVkbRrVw05QJBWs32bAWzeKtZ 8GoUkvSK5KAwAZowiwYjauBYwLuK9srCin/v+uT0AuG+5mzY99ckzfLA2Z5WSLLPDvts oTNAtMdpXW+o0dVBEbYS5WlaNdHPfyoSQ8yY6YYwVo1/SpyQNysDla4bWWga9GqjUd+p e6JZTcURTR+fUlM95AwBGqUxIWldOPOnzLIwanZu5Nz3Nvn0XnxQNWc687DUeFZlf/go oQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxe5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:21 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 8 Aug 2022 01:06:18 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Mon, 8 Aug 2022 01:06:18 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id F180F3F7068; Mon, 8 Aug 2022 01:06:14 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , , Volodymyr Fialko Subject: [PATCH 04/18] crypto/cnxk: limit the meta buf cache to 128 Date: Mon, 8 Aug 2022 13:35:52 +0530 Message-ID: <20220808080606.220-5-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: _zdVv-CaF1EZ1KiET9UgJLl8gdgZSX83 X-Proofpoint-ORIG-GUID: _zdVv-CaF1EZ1KiET9UgJLl8gdgZSX83 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Limit meta buf pool cache size as 128. Having 512 as the cache size would cause more time for refill. Signed-off-by: Anoob Joseph Signed-off-by: Volodymyr Fialko --- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index 80071872f1..a73c156d01 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -10,12 +10,13 @@ #include "cnxk_ae.h" #include "cnxk_cryptodev.h" -#include "cnxk_cryptodev_ops.h" #include "cnxk_cryptodev_capabilities.h" +#include "cnxk_cryptodev_ops.h" #include "cnxk_se.h" -#define CNXK_CPT_MAX_ASYM_OP_NUM_PARAMS 5 -#define CNXK_CPT_MAX_ASYM_OP_MOD_LEN 1024 +#define CNXK_CPT_MAX_ASYM_OP_NUM_PARAMS 5 +#define CNXK_CPT_MAX_ASYM_OP_MOD_LEN 1024 +#define CNXK_CPT_META_BUF_MAX_CACHE_SIZE 128 static int cnxk_cpt_get_mlen(void) @@ -200,7 +201,7 @@ cnxk_cpt_metabuf_mempool_create(const struct rte_cryptodev *dev, } mb_pool_sz = nb_elements; - cache_sz = RTE_MIN(RTE_MEMPOOL_CACHE_MAX_SIZE, nb_elements / 1.5); + cache_sz = RTE_MIN(CNXK_CPT_META_BUF_MAX_CACHE_SIZE, nb_elements / 1.5); /* For poll mode, core that enqueues and core that dequeues can be * different. For event mode, all cores are allowed to use same crypto From patchwork Mon Aug 8 08:05:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114695 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4E5CDA034C; Mon, 8 Aug 2022 10:06:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B2C0F42C08; Mon, 8 Aug 2022 10:06:24 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 4689A42B6D for ; Mon, 8 Aug 2022 10:06:22 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277MwkKp002933 for ; Mon, 8 Aug 2022 01:06:21 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=YWHJB1ZVudPgdZvVpaCO2oOiaSLMLL+wXZHU7dKoG/Q=; b=KscBDBgE7ulrux2V2AgO0ueUxVSh4GAe3e1Zqoa/w/eIkYpTqkzHDDRtiLXvO/meZrFE UyXIAINl3AGhIGBc0TRym2+kdfTnAs6S5dOdTuJyqWEIQ03MBCqv0cV6yquVPrOTeLjE PDuCOewwRnh850rkKQ0ohVZyF9iHHN0SUUfXIalKDKXqH4P5qJcAoOZxd3BHKiDo/xax uQgxE08dhTrAVE4GVWlGTcrvwRGQqlM703D1VvidDkyzhPB0IIhEanX4uAWBfnuYAUw0 V+mgOu3kGIoISuUDdvxyWjgi6ZPQoYUaZSKXffs8HvUFuQ7iQW+jD/aBeA5dgzvebln5 VQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxe5-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:21 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 8 Aug 2022 01:06:19 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 8 Aug 2022 01:06:19 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id EDFDC3F7090; Mon, 8 Aug 2022 01:06:17 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 05/18] crypto/cnxk: add separate path for pdcp chain opcode Date: Mon, 8 Aug 2022 13:35:53 +0530 Message-ID: <20220808080606.220-6-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: i8qCno3GvTCCv3kq-b-KZmpcl4IXMAOv X-Proofpoint-ORIG-GUID: i8qCno3GvTCCv3kq-b-KZmpcl4IXMAOv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add separate datapath for PDCP chain opcode. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 7 +- drivers/crypto/cnxk/cnxk_se.h | 229 +++++++++++++++++++---- drivers/crypto/cnxk/meson.build | 4 +- 3 files changed, 196 insertions(+), 44 deletions(-) diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index 3c4cc41bdb..2182c1bd2f 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -28,11 +28,12 @@ cn9k_cpt_sym_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, cpt_op = sess->cpt_op; - if (cpt_op & ROC_SE_OP_CIPHER_MASK) + if (sess->roc_se_ctx.fc_type == ROC_SE_PDCP_CHAIN) + ret = fill_pdcp_chain_params(op, sess, &qp->meta_info, infl_req, inst); + else if (cpt_op & ROC_SE_OP_CIPHER_MASK) ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst); else - ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, - inst); + ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, inst); return ret; } diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 5574dbfa04..2b477284c0 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -94,6 +94,47 @@ pdcp_iv_copy(uint8_t *iv_d, uint8_t *iv_s, const uint8_t pdcp_alg_type, } } +/* + * Digest immediately at the end of the data is the best case. Switch to SG if + * that cannot be ensured. + */ +static inline void +cpt_digest_buf_lb_check(const struct cnxk_se_sess *sess, struct rte_mbuf *m, + struct roc_se_fc_params *fc_params, uint32_t *flags, + struct rte_crypto_sym_op *sym_op, bool *inplace, uint32_t a_data_off, + uint32_t a_data_len, uint32_t c_data_off, uint32_t c_data_len, + const bool is_pdcp_chain) +{ + const uint32_t auth_end = a_data_off + a_data_len; + uint32_t mc_hash_off; + + /* PDCP_CHAIN only supports auth_first */ + + if (is_pdcp_chain || sess->auth_first) + mc_hash_off = auth_end; + else + mc_hash_off = RTE_MAX(c_data_off + c_data_len, auth_end); + + /* Digest immediately following data is best case */ + + if (unlikely(rte_pktmbuf_mtod_offset(m, uint8_t *, mc_hash_off) != + sym_op->auth.digest.data)) { + *flags |= ROC_SE_VALID_MAC_BUF; + fc_params->mac_buf.size = sess->mac_len; + fc_params->mac_buf.vaddr = sym_op->auth.digest.data; + *inplace = false; + } +} + +static inline struct rte_mbuf * +cpt_m_dst_get(uint8_t cpt_op, struct rte_mbuf *m_src, struct rte_mbuf *m_dst) +{ + if (m_dst != NULL && (cpt_op & ROC_SE_OP_ENCODE)) + return m_dst; + else + return m_src; +} + static __rte_always_inline int cpt_mac_len_verify(struct rte_crypto_auth_xform *auth) { @@ -1828,9 +1869,6 @@ cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst); } else if (fc_type == ROC_SE_KASUMI) { ret = cpt_kasumi_dec_prep(d_offs, d_lens, fc_params, inst); - } else if (fc_type == ROC_SE_PDCP_CHAIN) { - ret = cpt_pdcp_chain_alg_prep(flags, d_offs, d_lens, fc_params, - inst); } /* @@ -1858,13 +1896,9 @@ cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, } else if (fc_type == ROC_SE_PDCP) { ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst); } else if (fc_type == ROC_SE_KASUMI) { - ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, fc_params, - inst); + ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, fc_params, inst); } else if (fc_type == ROC_SE_HASH_HMAC) { ret = cpt_digest_gen_prep(flags, d_lens, fc_params, inst); - } else if (fc_type == ROC_SE_PDCP_CHAIN) { - ret = cpt_pdcp_chain_alg_prep(flags, d_offs, d_lens, fc_params, - inst); } return ret; @@ -2384,15 +2418,11 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, uint8_t inplace = 1; #endif struct roc_se_fc_params fc_params; - bool chain = sess->chained_op; char src[SRC_IOV_SIZE]; char dst[SRC_IOV_SIZE]; uint32_t iv_buf[4]; - bool pdcp_chain; int ret; - pdcp_chain = chain && (sess->zs_auth || sess->zs_cipher); - fc_params.cipher_iv_len = sess->iv_length; fc_params.auth_iv_len = sess->auth_iv_length; @@ -2410,11 +2440,11 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } - if (sess->zsk_flag || sess->zs_auth) { + if (sess->zsk_flag) { if (sess->auth_iv_length) - fc_params.auth_iv_buf = rte_crypto_op_ctod_offset( - cop, uint8_t *, sess->auth_iv_offset); - if ((!chain) && (sess->zsk_flag != ROC_SE_ZS_EA)) + fc_params.auth_iv_buf = + rte_crypto_op_ctod_offset(cop, uint8_t *, sess->auth_iv_offset); + if (sess->zsk_flag != ROC_SE_ZS_EA) inplace = 0; } m_src = sym_op->m_src; @@ -2453,11 +2483,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } fc_params.iv_buf = salt + 4; if (likely(sess->mac_len)) { - struct rte_mbuf *m = - (cpt_op & ROC_SE_OP_ENCODE) ? m_dst : m_src; - - if (!m) - m = m_src; + struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); /* hmac immediately following data is best case */ if (unlikely(rte_pktmbuf_mtod(m, uint8_t *) + @@ -2476,17 +2502,6 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, uint32_t a_data_length = sym_op->auth.data.length; uint32_t a_data_offset = sym_op->auth.data.offset; - if (pdcp_chain) { - if (sess->zs_cipher) { - ci_data_length /= 8; - ci_data_offset /= 8; - } - if (sess->zs_auth) { - a_data_length /= 8; - a_data_offset /= 8; - } - } - d_offs = ci_data_offset; d_offs = (d_offs << 16) | a_data_offset; @@ -2512,11 +2527,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.iv_buf = salt + 4; } if (likely(sess->mac_len)) { - struct rte_mbuf *m; - - m = (cpt_op & ROC_SE_OP_ENCODE) ? m_dst : m_src; - if (!m) - m = m_src; + struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); /* hmac immediately following data is best case */ if (!(op_minor & ROC_SE_FC_MINOR_OP_HMAC_FIRST) && @@ -2533,8 +2544,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; - if (!(sess->auth_first) && (!pdcp_chain) && - unlikely(sess->is_null || sess->cpt_op == ROC_SE_OP_DECODE)) + if (!(sess->auth_first) && unlikely(sess->is_null || sess->cpt_op == ROC_SE_OP_DECODE)) inplace = 0; if (likely(!m_dst && inplace)) { @@ -2622,6 +2632,147 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, return ret; } +static __rte_always_inline int +fill_pdcp_chain_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, + struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, + struct cpt_inst_s *inst) +{ + uint32_t ci_data_length, ci_data_offset, a_data_length, a_data_offset; + struct rte_crypto_sym_op *sym_op = cop->sym; + struct roc_se_fc_params fc_params; + struct rte_mbuf *m_src, *m_dst; + uint8_t cpt_op = sess->cpt_op; + uint64_t d_offs, d_lens; + char src[SRC_IOV_SIZE]; + char dst[SRC_IOV_SIZE]; + bool inplace = true; + uint32_t flags = 0; + void *mdata; + int ret; + + fc_params.cipher_iv_len = sess->iv_length; + fc_params.auth_iv_len = sess->auth_iv_length; + fc_params.iv_buf = NULL; + fc_params.auth_iv_buf = NULL; + + m_src = sym_op->m_src; + m_dst = sym_op->m_dst; + + if (likely(sess->iv_length)) + fc_params.iv_buf = rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset); + + ci_data_length = sym_op->cipher.data.length; + ci_data_offset = sym_op->cipher.data.offset; + a_data_length = sym_op->auth.data.length; + a_data_offset = sym_op->auth.data.offset; + + /* + * For ZUC & SNOW, length & offset is provided in bits. Convert to + * bytes. + */ + + if (sess->zs_cipher) { + ci_data_length /= 8; + ci_data_offset /= 8; + } + + if (sess->zs_auth) { + a_data_length /= 8; + a_data_offset /= 8; + /* + * ZUC & SNOW would have valid iv_buf. AES-CMAC doesn't require + * IV from application. + */ + fc_params.auth_iv_buf = + rte_crypto_op_ctod_offset(cop, uint8_t *, sess->auth_iv_offset); +#ifdef CNXK_CRYPTODEV_DEBUG + if (sess->auth_iv_length == 0) + plt_err("Invalid auth IV length"); +#endif + } + + d_offs = ci_data_offset; + d_offs = (d_offs << 16) | a_data_offset; + d_lens = ci_data_length; + d_lens = (d_lens << 32) | a_data_length; + + if (likely(sess->mac_len)) { + struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); + + cpt_digest_buf_lb_check(sess, m, &fc_params, &flags, sym_op, &inplace, + a_data_offset, a_data_length, ci_data_offset, + ci_data_length, true); + } + + fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; + + if (likely((m_dst == NULL || m_dst == m_src)) && inplace) { + fc_params.dst_iov = fc_params.src_iov = (void *)src; + prepare_iov_from_pkt_inplace(m_src, &fc_params, &flags); + } else { + /* Out of place processing */ + fc_params.src_iov = (void *)src; + fc_params.dst_iov = (void *)dst; + + /* Store SG I/O in the api for reuse */ + if (unlikely(prepare_iov_from_pkt(m_src, fc_params.src_iov, 0))) { + plt_dp_err("Could not prepare src iov"); + ret = -EINVAL; + goto err_exit; + } + + if (unlikely(m_dst != NULL)) { + uint32_t pkt_len; + + /* Try to make room as much as src has */ + pkt_len = rte_pktmbuf_pkt_len(m_dst); + + if (unlikely(pkt_len < rte_pktmbuf_pkt_len(m_src))) { + pkt_len = rte_pktmbuf_pkt_len(m_src) - pkt_len; + if (!rte_pktmbuf_append(m_dst, pkt_len)) { + plt_dp_err("Not enough space in m_dst " + "%p, need %u more", + m_dst, pkt_len); + ret = -EINVAL; + goto err_exit; + } + } + + if (unlikely(prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0))) { + plt_dp_err("Could not prepare m_dst iov %p", m_dst); + ret = -EINVAL; + goto err_exit; + } + } else { + fc_params.dst_iov = (void *)src; + } + } + + if (unlikely(!((flags & ROC_SE_SINGLE_BUF_INPLACE) && + (flags & ROC_SE_SINGLE_BUF_HEADROOM)))) { + mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, m_info->pool, infl_req); + if (unlikely(mdata == NULL)) { + plt_dp_err("Could not allocate meta buffer for request"); + return -ENOMEM; + } + } + + /* Finally prepare the instruction */ + ret = cpt_pdcp_chain_alg_prep(flags, d_offs, d_lens, &fc_params, inst); + if (unlikely(ret)) { + plt_dp_err("Could not prepare instruction"); + goto free_mdata_and_exit; + } + + return 0; + +free_mdata_and_exit: + if (infl_req->op_flags & CPT_OP_FLAGS_METABUF) + rte_mempool_put(m_info->pool, infl_req->mdata); +err_exit: + return ret; +} + static __rte_always_inline void compl_auth_verify(struct rte_crypto_op *op, uint8_t *gen_mac, uint64_t mac_len) { diff --git a/drivers/crypto/cnxk/meson.build b/drivers/crypto/cnxk/meson.build index 952554ac12..8db861f908 100644 --- a/drivers/crypto/cnxk/meson.build +++ b/drivers/crypto/cnxk/meson.build @@ -27,7 +27,7 @@ deps += ['bus_pci', 'common_cnxk', 'security', 'eventdev'] includes += include_directories('../../../lib/net', '../../event/cnxk') if get_option('buildtype').contains('debug') - cflags += [ '-DLA_IPSEC_DEBUG' ] + cflags += [ '-DLA_IPSEC_DEBUG','-DCNXK_CRYPTODEV_DEBUG' ] else - cflags += [ '-ULA_IPSEC_DEBUG' ] + cflags += [ '-ULA_IPSEC_DEBUG','-UCNXK_CRYPTODEV_DEBUG' ] endif From patchwork Mon Aug 8 08:05:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114696 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E8219A034C; Mon, 8 Aug 2022 10:06:54 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BA30E42B6D; Mon, 8 Aug 2022 10:06:26 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 8583542C23 for ; Mon, 8 Aug 2022 10:06:25 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277MToSf003845 for ; Mon, 8 Aug 2022 01:06:25 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=ZoncwHScaA1cg6x9JgJyPj146wXnAIm6ACWMN/APXOg=; b=DIvkmCHKzyQSu1l/Ak8DDoQAVHHpahk1P6v8gRyo/HtBpr+oaV9gamArYTNtwH4kEUA+ Ls9hSd5L4EUwy4OyQ3TrGJjc2+qmhKp4t67C2raaBRsPXDPvsDS7waPx5WKtQPO/BWb5 aDpexK+gzhj6BDybEh/3HEpNMupu5cHPTKH9XWJ5cJbSh2uZv5/4aY8Xkww2VoIo6tcx WL0mYZDcNvCvZBXqlJyVezG3BeKnik89cz3jD2F4NYm5Gbfw3wiy5LOFGS2/uVaoiVwu Gg4daxVx6Oc31K8xdJKNtsAUe2pNQGyxmeM0m1mZuGhJ3rdl8Q5O/lgH8Gsdw0vBokty VA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxea-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:24 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Aug 2022 01:06:22 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Mon, 8 Aug 2022 01:06:22 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id 74F863F7043; Mon, 8 Aug 2022 01:06:20 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 06/18] crypto/cnxk: add separate datapath for pdcp cipher operation Date: Mon, 8 Aug 2022 13:35:54 +0530 Message-ID: <20220808080606.220-7-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: GgSO3olVoTujxZntbzfH1-LywyYaS8rh X-Proofpoint-ORIG-GUID: GgSO3olVoTujxZntbzfH1-LywyYaS8rh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add separate datapath for PDCP opcode performing cipher operation. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 19 --- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 27 +--- drivers/crypto/cnxk/cnxk_se.h | 177 +++++++++++++++++++--- 3 files changed, 158 insertions(+), 65 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index bfa6374005..1b70d02e2a 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -77,25 +77,6 @@ cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, return ret; } -static __rte_always_inline int __rte_hot -cpt_sym_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, - struct cnxk_se_sess *sess, struct cpt_inflight_req *infl_req, - struct cpt_inst_s *inst) -{ - uint64_t cpt_op; - int ret; - - cpt_op = sess->cpt_op; - - if (cpt_op & ROC_SE_OP_CIPHER_MASK) - ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst); - else - ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, - inst); - - return ret; -} - static inline int cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], struct cpt_inst_s inst[], struct cpt_inflight_req *infl_req) diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index 2182c1bd2f..3d69723809 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -17,27 +17,6 @@ #include "cnxk_cryptodev_ops.h" #include "cnxk_se.h" -static __rte_always_inline int __rte_hot -cn9k_cpt_sym_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, - struct cnxk_se_sess *sess, - struct cpt_inflight_req *infl_req, - struct cpt_inst_s *inst) -{ - uint64_t cpt_op; - int ret; - - cpt_op = sess->cpt_op; - - if (sess->roc_se_ctx.fc_type == ROC_SE_PDCP_CHAIN) - ret = fill_pdcp_chain_params(op, sess, &qp->meta_info, infl_req, inst); - else if (cpt_op & ROC_SE_OP_CIPHER_MASK) - ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst); - else - ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, inst); - - return ret; -} - static __rte_always_inline int __rte_hot cn9k_cpt_sec_inst_fill(struct rte_crypto_op *op, struct cpt_inflight_req *infl_req, @@ -118,8 +97,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, sym_op = op->sym; sess = get_sym_session_private_data( sym_op->session, cn9k_cryptodev_driver_id); - ret = cn9k_cpt_sym_inst_fill(qp, op, sess, infl_req, - inst); + ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst); inst->w7.u64 = sess->cpt_inst_w7; } else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) ret = cn9k_cpt_sec_inst_fill(op, infl_req, inst); @@ -130,8 +108,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, return -1; } - ret = cn9k_cpt_sym_inst_fill(qp, op, sess, infl_req, - inst); + ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst); if (unlikely(ret)) { sym_session_clear(cn9k_cryptodev_driver_id, op->sym->session); diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 2b477284c0..35d074ea34 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -1865,8 +1865,6 @@ cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, if (likely(fc_type == ROC_SE_FC_GEN)) { ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, fc_params, inst); - } else if (fc_type == ROC_SE_PDCP) { - ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst); } else if (fc_type == ROC_SE_KASUMI) { ret = cpt_kasumi_dec_prep(d_offs, d_lens, fc_params, inst); } @@ -2400,8 +2398,8 @@ prepare_iov_from_pkt_inplace(struct rte_mbuf *pkt, static __rte_always_inline int fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, - struct cpt_qp_meta_info *m_info, - struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst) + struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, + struct cpt_inst_s *inst, const bool is_kasumi) { struct roc_se_ctx *ctx = &sess->roc_se_ctx; uint8_t op_minor = ctx->template_w4.s.opcode_minor; @@ -2424,7 +2422,9 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, int ret; fc_params.cipher_iv_len = sess->iv_length; - fc_params.auth_iv_len = sess->auth_iv_length; + fc_params.auth_iv_len = 0; + fc_params.auth_iv_buf = NULL; + fc_params.iv_buf = NULL; if (likely(sess->iv_length)) { flags |= ROC_SE_VALID_IV_BUF; @@ -2440,13 +2440,15 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } - if (sess->zsk_flag) { + /* Kasumi would need auth IV */ + if (is_kasumi && sess->zsk_flag) { + fc_params.auth_iv_len = sess->auth_iv_length; if (sess->auth_iv_length) fc_params.auth_iv_buf = rte_crypto_op_ctod_offset(cop, uint8_t *, sess->auth_iv_offset); - if (sess->zsk_flag != ROC_SE_ZS_EA) - inplace = 0; + inplace = 0; } + m_src = sym_op->m_src; m_dst = sym_op->m_dst; @@ -2508,14 +2510,6 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, d_lens = ci_data_length; d_lens = (d_lens << 32) | a_data_length; - if (sess->auth_first) - mc_hash_off = a_data_offset + a_data_length; - else - mc_hash_off = ci_data_offset + ci_data_length; - - if (mc_hash_off < (a_data_offset + a_data_length)) { - mc_hash_off = (a_data_offset + a_data_length); - } /* for gmac, salt should be updated like in gcm */ if (unlikely(sess->is_gmac)) { uint8_t *salt; @@ -2529,6 +2523,14 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, if (likely(sess->mac_len)) { struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); + if (sess->auth_first) + mc_hash_off = a_data_offset + a_data_length; + else + mc_hash_off = ci_data_offset + ci_data_length; + + if (mc_hash_off < (a_data_offset + a_data_length)) + mc_hash_off = (a_data_offset + a_data_length); + /* hmac immediately following data is best case */ if (!(op_minor & ROC_SE_FC_MINOR_OP_HMAC_FIRST) && (unlikely(rte_pktmbuf_mtod(m, uint8_t *) + @@ -2599,11 +2601,8 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } if (unlikely(!((flags & ROC_SE_SINGLE_BUF_INPLACE) && - (flags & ROC_SE_SINGLE_BUF_HEADROOM) && - ((ctx->fc_type != ROC_SE_KASUMI) && - (ctx->fc_type != ROC_SE_HASH_HMAC))))) { - mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, - m_info->pool, infl_req); + (flags & ROC_SE_SINGLE_BUF_HEADROOM) && !is_kasumi))) { + mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, m_info->pool, infl_req); if (mdata == NULL) { plt_dp_err("Error allocating meta buffer for request"); return -ENOMEM; @@ -2632,6 +2631,112 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, return ret; } +static __rte_always_inline int +fill_pdcp_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, + struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, + struct cpt_inst_s *inst) +{ + struct rte_crypto_sym_op *sym_op = cop->sym; + struct roc_se_fc_params fc_params; + uint32_t c_data_len, c_data_off; + struct rte_mbuf *m_src, *m_dst; + uint64_t d_offs, d_lens; + char src[SRC_IOV_SIZE]; + char dst[SRC_IOV_SIZE]; + void *mdata = NULL; + uint32_t flags = 0; + int ret; + + /* Cipher only */ + + fc_params.cipher_iv_len = sess->iv_length; + fc_params.auth_iv_len = 0; + fc_params.iv_buf = NULL; + fc_params.auth_iv_buf = NULL; + + if (likely(sess->iv_length)) + fc_params.iv_buf = rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset); + + m_src = sym_op->m_src; + m_dst = sym_op->m_dst; + + c_data_len = sym_op->cipher.data.length; + c_data_off = sym_op->cipher.data.offset; + + d_offs = (uint64_t)c_data_off << 16; + d_lens = (uint64_t)c_data_len << 32; + + fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; + + if (likely(m_dst == NULL || m_src == m_dst)) { + fc_params.dst_iov = fc_params.src_iov = (void *)src; + prepare_iov_from_pkt_inplace(m_src, &fc_params, &flags); + } else { + /* Out of place processing */ + fc_params.src_iov = (void *)src; + fc_params.dst_iov = (void *)dst; + + /* Store SG I/O in the api for reuse */ + if (prepare_iov_from_pkt(m_src, fc_params.src_iov, 0)) { + plt_dp_err("Prepare src iov failed"); + ret = -EINVAL; + goto err_exit; + } + + if (unlikely(m_dst != NULL)) { + uint32_t pkt_len; + + /* Try to make room as much as src has */ + pkt_len = rte_pktmbuf_pkt_len(m_dst); + + if (unlikely(pkt_len < rte_pktmbuf_pkt_len(m_src))) { + pkt_len = rte_pktmbuf_pkt_len(m_src) - pkt_len; + if (!rte_pktmbuf_append(m_dst, pkt_len)) { + plt_dp_err("Not enough space in " + "m_dst %p, need %u" + " more", + m_dst, pkt_len); + ret = -EINVAL; + goto err_exit; + } + } + + if (prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0)) { + plt_dp_err("Prepare dst iov failed for " + "m_dst %p", + m_dst); + ret = -EINVAL; + goto err_exit; + } + } else { + fc_params.dst_iov = (void *)src; + } + } + + if (unlikely(!((flags & ROC_SE_SINGLE_BUF_INPLACE) && + (flags & ROC_SE_SINGLE_BUF_HEADROOM)))) { + mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, m_info->pool, infl_req); + if (mdata == NULL) { + plt_dp_err("Could not allocate meta buffer"); + return -ENOMEM; + } + } + + ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, &fc_params, inst); + if (unlikely(ret)) { + plt_dp_err("Could not prepare instruction"); + goto free_mdata_and_exit; + } + + return 0; + +free_mdata_and_exit: + if (infl_req->op_flags & CPT_OP_FLAGS_METABUF) + rte_mempool_put(m_info->pool, infl_req->mdata); +err_exit: + return ret; +} + static __rte_always_inline int fill_pdcp_chain_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, @@ -2974,4 +3079,34 @@ fill_digest_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, err_exit: return ret; } + +static __rte_always_inline int __rte_hot +cpt_sym_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cnxk_se_sess *sess, + struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst) +{ + uint64_t cpt_op = sess->cpt_op; + int ret; + + if (cpt_op & ROC_SE_OP_CIPHER_MASK) { + switch (sess->roc_se_ctx.fc_type) { + case ROC_SE_PDCP_CHAIN: + ret = fill_pdcp_chain_params(op, sess, &qp->meta_info, infl_req, inst); + break; + case ROC_SE_PDCP: + ret = fill_pdcp_params(op, sess, &qp->meta_info, infl_req, inst); + break; + case ROC_SE_KASUMI: + ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, true); + break; + default: + ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, false); + break; + } + } else { + ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, inst); + } + + return ret; +} + #endif /*_CNXK_SE_H_ */ From patchwork Mon Aug 8 08:05:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114697 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4D471A034C; Mon, 8 Aug 2022 10:07:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0291942C45; Mon, 8 Aug 2022 10:06:29 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 392D942C29 for ; Mon, 8 Aug 2022 10:06:27 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277LTX4g015930 for ; Mon, 8 Aug 2022 01:06:26 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=2IejpeZY/nkwaNuPwnjsg58FbpV/xxhag3Aym+WoklQ=; b=NqVONjcYDkWDSebmNK3rK0k3kO21hpcKHlLpynANlWULvvPX/B+9p1qj4b2FItAbdxwh 5vwizUQN6Q76ctncIoh0/qG0cHUHQV8SriyxEycY3EX3KWdN0I+vpveVyVtlQP7A26gA 3sGbKmzr0YJjkpTnIcu44tpHKoTPfA+MTvu6Q3klZ9xOXDPWPtKy323+KvTPgZ73ETPH jI5RZt2N4NGa2TmfDwVGRmB6VpS9gF8YgHyxFAFP5QvJM1WZhmtdGiEEyhVhdGdAq65J R7u3F8AjbK/z+4Azbvkdtr7RZxwxoKJeXL65U0QrijlV9GHBFQjKLtgpxxlRLzR5SvBy oA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3hsnnqnf7e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:26 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Aug 2022 01:06:24 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 8 Aug 2022 01:06:24 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id E24D43F7051; Mon, 8 Aug 2022 01:06:22 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 07/18] crypto/cnxk: remove MAC len check for AEAD Date: Mon, 8 Aug 2022 13:35:55 +0530 Message-ID: <20220808080606.220-8-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: AWqPsCUrFXXu864xWNA3MCMUu6ulHR0j X-Proofpoint-GUID: AWqPsCUrFXXu864xWNA3MCMUu6ulHR0j X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For both AES-GCM and ChaChaPoly, digest size must be non-zero. So remove the conditional handling in the datapath. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 35d074ea34..2c44dba9a0 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2453,6 +2453,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, m_dst = sym_op->m_dst; if (sess->aes_gcm || sess->chacha_poly) { + struct rte_mbuf *m; uint8_t *salt; uint8_t *aad_data; uint16_t aad_len; @@ -2483,20 +2484,19 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, cpt_fc_salt_update(&sess->roc_se_ctx, salt); sess->salt = *(uint32_t *)salt; } - fc_params.iv_buf = salt + 4; - if (likely(sess->mac_len)) { - struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); - /* hmac immediately following data is best case */ - if (unlikely(rte_pktmbuf_mtod(m, uint8_t *) + - mc_hash_off != - (uint8_t *)sym_op->aead.digest.data)) { - flags |= ROC_SE_VALID_MAC_BUF; - fc_params.mac_buf.size = sess->mac_len; - fc_params.mac_buf.vaddr = - sym_op->aead.digest.data; - inplace = 0; - } + fc_params.iv_buf = PLT_PTR_ADD(salt, 4); + fc_params.mac_buf.size = 0; + fc_params.mac_buf.vaddr = NULL; + m = cpt_m_dst_get(cpt_op, m_src, m_dst); + + /* Digest immediately following data is best case */ + if (unlikely(rte_pktmbuf_mtod(m, uint8_t *) + mc_hash_off != + (uint8_t *)sym_op->aead.digest.data)) { + flags |= ROC_SE_VALID_MAC_BUF; + fc_params.mac_buf.size = sess->mac_len; + fc_params.mac_buf.vaddr = sym_op->aead.digest.data; + inplace = 0; } } else { uint32_t ci_data_length = sym_op->cipher.data.length; From patchwork Mon Aug 8 08:05:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114698 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4F09DA034C; Mon, 8 Aug 2022 10:07:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 29D8442C3C; Mon, 8 Aug 2022 10:06:32 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id B972342BF4 for ; Mon, 8 Aug 2022 10:06:30 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277N0ebX006022 for ; Mon, 8 Aug 2022 01:06:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Lt5En0iNnf3iEtTDhzPIHZ6WpGSX/nQJAm2ATQ3LWuM=; b=HewCZGZOBtY5yi0MTNYOf9KLxnKlohvFOsu5PFqvL+M/9fZTXk7TgZyiFdgAPeqO33hk SpyJBM8zOQ9rtJ+Cna/u0mwQCxB2LC2IgfdAbRdYrk6eQCHm/vv9vAZvCPTJKgZeLJy7 rgWz80TYPnDH9nvM17e4gGRFTujl6oTSl0vcK3niq5WknKjSc7hx6IIZ2UNLzsf9tQvV pvMhlbR3bFyggohi5VNDUlxiCY5BoM545ViqR4mDfKIHAwdr/h7ajkZ1BySNBK6aq0wK rBVwH1fxmgVpl5WEdd/L6fF/7CiIXQx7DInZfv7h6aNr2hyFrpwf9KNAoCOEqyXbZixu Zw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxeg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:29 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 8 Aug 2022 01:06:27 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Mon, 8 Aug 2022 01:06:27 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id 543933F7043; Mon, 8 Aug 2022 01:06:25 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Tejasree Kondoj , Archana Muniganti , Subject: [PATCH 08/18] crypto/cnxk: fix endianness in anti-replay Date: Mon, 8 Aug 2022 13:35:56 +0530 Message-ID: <20220808080606.220-9-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: yhXJVxh8B0i2tkyweTZAi0AfgrOXfUhi X-Proofpoint-ORIG-GUID: yhXJVxh8B0i2tkyweTZAi0AfgrOXfUhi X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Tejasree Kondoj Fixing anti-replay endianness issue in lookaside IPsec. Fixes: c1eac1b966c2 ("crypto/cnxk: add anti-replay as per new firmware") Cc: ktejasree@marvell.com Signed-off-by: Tejasree Kondoj --- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index 3d69723809..b753c1cb4b 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -487,22 +487,19 @@ cn9k_cpt_crypto_adapter_enqueue(uintptr_t base, struct rte_crypto_op *op) } static inline int -ipsec_antireplay_check(struct cn9k_ipsec_sa *sa, uint32_t win_sz, - struct roc_ie_on_inb_hdr *data) +ipsec_antireplay_check(struct cn9k_ipsec_sa *sa, uint32_t win_sz, struct roc_ie_on_inb_hdr *data) { + uint32_t esn_low, esn_hi, seql, seqh = 0; struct roc_ie_on_common_sa *common_sa; struct roc_ie_on_inb_sa *in_sa; - struct roc_ie_on_sa_ctl *ctl; - uint32_t seql, seqh = 0; - uint64_t seq; + uint64_t seq, seq_in_sa; uint8_t esn; int ret; in_sa = &sa->in_sa; common_sa = &in_sa->common_sa; - ctl = &common_sa->ctl; - esn = ctl->esn_en; + esn = common_sa->ctl.esn_en; seql = rte_be_to_cpu_32(data->seql); if (!esn) { @@ -517,9 +514,13 @@ ipsec_antireplay_check(struct cn9k_ipsec_sa *sa, uint32_t win_sz, ret = cnxk_on_anti_replay_check(seq, &sa->ar, win_sz); if (esn && !ret) { - common_sa = &sa->in_sa.common_sa; - if (seq > common_sa->seq_t.u64) - common_sa->seq_t.u64 = seq; + esn_low = rte_be_to_cpu_32(common_sa->seq_t.tl); + esn_hi = rte_be_to_cpu_32(common_sa->seq_t.th); + seq_in_sa = ((uint64_t)esn_hi << 32) | esn_low; + if (seq > seq_in_sa) { + common_sa->seq_t.tl = rte_cpu_to_be_32(seql); + common_sa->seq_t.th = rte_cpu_to_be_32(seqh); + } } return ret; From patchwork Mon Aug 8 08:05:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114699 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA122A034C; Mon, 8 Aug 2022 10:07:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1256442C58; Mon, 8 Aug 2022 10:06:34 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 3BB0742C5B for ; Mon, 8 Aug 2022 10:06:32 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277N0ebY006022 for ; Mon, 8 Aug 2022 01:06:31 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=lwPeCLk5Pfx4jiQXglED89eXS2hVer4wgrcZTsVh7ak=; b=YS+a9kOHMnj/TMNZO3pLw6Fmf0pjMepGJ2PP8hYYVH6KKP6FYM08QIjt79TA9MHmXjbI T2c+HVv9n4OBWUDVkJ6T0iYm7fZk3lzT7D2XJI/eyh9HOUM2hJtpyW+Dchg4IqScAdQC 9JAfPiUopg1MHZNMhc9t5zXbfw2fnVImTp6G8LOJ7W5/4P6Xxgv+MRhywXV+APIx+5g5 1ODmjgF6IHFjo+C/af1bKGJwDV7R7a8go14OQnw53kv1w+rvFLvYNSJZ7oDvA+qJV3kc xbV3tFAl9lCxIRI/E/CuqSNwGQbZ7DYTXMwbWXLySAF3ms3rPdrr7h4pC2X58TKzn3pa JA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxeh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:31 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 8 Aug 2022 01:06:29 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 8 Aug 2022 01:06:29 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id BDE433F708F; Mon, 8 Aug 2022 01:06:27 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 09/18] crypto/cnxk: remove extra indirection for FC and Kasumi Date: Mon, 8 Aug 2022 13:35:57 +0530 Message-ID: <20220808080606.220-10-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 00Hlf-LnPIWXoDxq6gwr9C0hYaidW8et X-Proofpoint-ORIG-GUID: 00Hlf-LnPIWXoDxq6gwr9C0hYaidW8et X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Fill FC params routine now caters to only KASUMI & FC_GEN. Call appropriate functions directly. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 51 +++++++++++------------------------ 1 file changed, 15 insertions(+), 36 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 2c44dba9a0..c176cda0b1 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -1852,36 +1852,9 @@ cpt_kasumi_dec_prep(uint64_t d_offs, uint64_t d_lens, return 0; } -static __rte_always_inline int -cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, - struct roc_se_fc_params *fc_params, - struct cpt_inst_s *inst) -{ - struct roc_se_ctx *ctx = fc_params->ctx_buf.vaddr; - uint8_t fc_type; - int ret = -1; - - fc_type = ctx->fc_type; - - if (likely(fc_type == ROC_SE_FC_GEN)) { - ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, fc_params, inst); - } else if (fc_type == ROC_SE_KASUMI) { - ret = cpt_kasumi_dec_prep(d_offs, d_lens, fc_params, inst); - } - - /* - * For AUTH_ONLY case, - * MC only supports digest generation and verification - * should be done in software by memcmp() - */ - - return ret; -} - static __rte_always_inline int cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, - struct roc_se_fc_params *fc_params, - struct cpt_inst_s *inst) + struct roc_se_fc_params *fc_params, struct cpt_inst_s *inst) { struct roc_se_ctx *ctx = fc_params->ctx_buf.vaddr; uint8_t fc_type; @@ -2600,8 +2573,8 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } - if (unlikely(!((flags & ROC_SE_SINGLE_BUF_INPLACE) && - (flags & ROC_SE_SINGLE_BUF_HEADROOM) && !is_kasumi))) { + if (unlikely(is_kasumi || !((flags & ROC_SE_SINGLE_BUF_INPLACE) && + (flags & ROC_SE_SINGLE_BUF_HEADROOM)))) { mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, m_info->pool, infl_req); if (mdata == NULL) { plt_dp_err("Error allocating meta buffer for request"); @@ -2610,12 +2583,18 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } /* Finally prepare the instruction */ - if (cpt_op & ROC_SE_OP_ENCODE) - ret = cpt_fc_enc_hmac_prep(flags, d_offs, d_lens, &fc_params, - inst); - else - ret = cpt_fc_dec_hmac_prep(flags, d_offs, d_lens, &fc_params, - inst); + + if (is_kasumi) { + if (cpt_op & ROC_SE_OP_ENCODE) + ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, &fc_params, inst); + else + ret = cpt_kasumi_dec_prep(d_offs, d_lens, &fc_params, inst); + } else { + if (cpt_op & ROC_SE_OP_ENCODE) + ret = cpt_enc_hmac_prep(flags, d_offs, d_lens, &fc_params, inst); + else + ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, &fc_params, inst); + } if (unlikely(ret)) { plt_dp_err("Preparing request failed due to bad input arg"); From patchwork Mon Aug 8 08:05:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114700 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31677A034C; Mon, 8 Aug 2022 10:07:21 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 18D2042C62; Mon, 8 Aug 2022 10:06:36 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id BA9E842C60 for ; Mon, 8 Aug 2022 10:06:34 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277N0Na6005832 for ; Mon, 8 Aug 2022 01:06:34 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=S4G5kQUCvZ5krs7SNbFROGTF1EMti7nXQCx79tJdGpQ=; b=VR7I9vdJRC1+3dUp0g0fFzIdj1BGBPiQKdE6wqYQZxH0zbmJKfh/W5NegP/f3sXM1+uC k7k9IqfCxO00/+SWWbvA1lwZucFyxyXiFDKXX3r7czlrLPg2MxdKvSMcz3qBwFn8y9Ny 7Rcip7RRCJS2EVQAwDaP1jVFB1Zxnkxb3zCFXfz3TOzP6Vs9LgGQdLDG8Knif4D83qS5 6fW+OJYsYmhLzJz+FWRkXgj49N3Irh018oebMk4oOGjminggkbUElAw/BcInxj5sPC4l QrzJp6qtCLhAHiQcj09iKfKTolOzP1O1V59H/j62OOUnMhn035b4G/yrE7klShd4THlO zQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxen-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:34 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Aug 2022 01:06:32 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 8 Aug 2022 01:06:32 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id 364E93F7051; Mon, 8 Aug 2022 01:06:29 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 10/18] crypto/cnxk: remove extra digest len check Date: Mon, 8 Aug 2022 13:35:58 +0530 Message-ID: <20220808080606.220-11-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 0150mSKN5DnwqNEFTX0KCWpI1I__bGAx X-Proofpoint-ORIG-GUID: 0150mSKN5DnwqNEFTX0KCWpI1I__bGAx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When the flag ROC_SE_VALID_MAC_BUF is set, digest length will be non-zero. Remove extra check in datapath. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index c176cda0b1..64a66ef911 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -680,11 +680,9 @@ cpt_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, return -1; } } - /* mac_data */ - if (mac_len) { - i = fill_sg_comp_from_buf(scatter_comp, i, - &fc_params->mac_buf); - } + + /* Digest buffer */ + i = fill_sg_comp_from_buf(scatter_comp, i, &fc_params->mac_buf); } else { /* Output including mac */ size = outputlen - iv_len; From patchwork Mon Aug 8 08:05:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114701 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D6F8A034C; Mon, 8 Aug 2022 10:07:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0A50D42C6C; Mon, 8 Aug 2022 10:06:39 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 50B8342C6B for ; Mon, 8 Aug 2022 10:06:37 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277NUUsj007808 for ; Mon, 8 Aug 2022 01:06:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=cPWjJna3CDXddua4ePKjNeKTn+1yfSIRoXca0BfzdKw=; b=Ffgb2HR4VaAVr/bXxzNkTnh703BwnWWoJwGJITLoP923ZEpWdQayOYBS6cDD+2UE4CPr nFbyXBHxk0eY5NGGW5gbB7KWNJHOfQC9UK/Z+4gu4ZSW+GukuumbLdYIorRNItDe0QDU vco4LwDlhAops/WIGDuwZZAw68O/I6AHOWs3AwhRTQvmnw4Rs3X0gHHy7IGHAP1kEe3W NBiHZLQmI/SafotF3Xs2zWfMFcPK5uuJhHKDOy+MALzyMAJeCGjNzE9jnUpdp/gjdeHb 2veEpvtIdDQQX9Xj+hplnCEaQy8sG1wdwwEogVC5xHZ7EkS2FsLtxmsHsiHeyN86uJ0Z 6Q== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxer-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:36 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 8 Aug 2022 01:06:34 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 8 Aug 2022 01:06:34 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id A87A53F7043; Mon, 8 Aug 2022 01:06:32 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 11/18] crypto/cnxk: avoid accessing se ctx in aes gcm path Date: Mon, 8 Aug 2022 13:35:59 +0530 Message-ID: <20220808080606.220-12-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: Z5vVbpzkz1qXoNcO3GxBg4qIW18zOyZZ X-Proofpoint-ORIG-GUID: Z5vVbpzkz1qXoNcO3GxBg4qIW18zOyZZ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Field op_minor is required only for digest encrypted cases with chained ops. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 64a66ef911..55f411d50c 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2372,8 +2372,6 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst, const bool is_kasumi) { - struct roc_se_ctx *ctx = &sess->roc_se_ctx; - uint8_t op_minor = ctx->template_w4.s.opcode_minor; struct rte_crypto_sym_op *sym_op = cop->sym; void *mdata = NULL; uint32_t mc_hash_off; @@ -2474,6 +2472,9 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, uint32_t ci_data_offset = sym_op->cipher.data.offset; uint32_t a_data_length = sym_op->auth.data.length; uint32_t a_data_offset = sym_op->auth.data.offset; + struct roc_se_ctx *ctx = &sess->roc_se_ctx; + + const uint8_t op_minor = ctx->template_w4.s.opcode_minor; d_offs = ci_data_offset; d_offs = (d_offs << 16) | a_data_offset; From patchwork Mon Aug 8 08:06:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114702 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0B606A034C; Mon, 8 Aug 2022 10:07:32 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0A8F142C74; Mon, 8 Aug 2022 10:06:41 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id A838042C71 for ; Mon, 8 Aug 2022 10:06:39 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277N0ebZ006022 for ; Mon, 8 Aug 2022 01:06:39 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=OyJlisHp/7Db/uIC/1g5/cgClk4uBkEjjF8SfU1F8gI=; b=VtDkyxZCA/hMYcrdXOviY0GQ2Lfsg7vBA13n0UlKgLoiwt4pbJmUK0yJ5IjJeTNNFY9Y NdBnDVcdX/5GMszEVbJqmTEqKRKb39b6mB8Djp7gPJJlg2duY/8mDtcFdLfIzH3fvrE0 wGRbCUYLpCg1JxAU5r2Ap7RuuEgtvvTKC4viEEb2jxJsoU9amKNpZZ7AQPx7PBzxGfiE 6utl00z/EltSVKkLNX4tmE4LeGYbq0TD1eixbZ1MgEhNpppe4i0qOBoGOs2uNZWGocQh +brFMmRwrVaJe+G+Qjs640+fmWpyb5p3suu9LE1e+XX/RWNsc8+JBMpoVz1rFSBqHWGD QQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxew-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:39 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 8 Aug 2022 01:06:37 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 8 Aug 2022 01:06:37 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id 31D233F7051; Mon, 8 Aug 2022 01:06:34 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 12/18] crypto/cnxk: remove auth iv from kasumi cipher Date: Mon, 8 Aug 2022 13:36:00 +0530 Message-ID: <20220808080606.220-13-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: z0LUsmQwr1zToQo0TWoDS0ziEyYsvJAr X-Proofpoint-ORIG-GUID: z0LUsmQwr1zToQo0TWoDS0ziEyYsvJAr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Auth IV is not applicable for kasumi cipher operation. Remove the same. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 55f411d50c..3de9718f8b 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2409,13 +2409,11 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } - /* Kasumi would need auth IV */ - if (is_kasumi && sess->zsk_flag) { - fc_params.auth_iv_len = sess->auth_iv_length; - if (sess->auth_iv_length) - fc_params.auth_iv_buf = - rte_crypto_op_ctod_offset(cop, uint8_t *, sess->auth_iv_offset); + /* Kasumi would need SG mode */ + if (is_kasumi) { inplace = 0; + fc_params.mac_buf.vaddr = NULL; + fc_params.mac_buf.size = 0; } m_src = sym_op->m_src; From patchwork Mon Aug 8 08:06:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114703 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 56A09A034C; Mon, 8 Aug 2022 10:07:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 06E6F42C79; Mon, 8 Aug 2022 10:06:43 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id EAC3242C78 for ; Mon, 8 Aug 2022 10:06:41 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277NVRu0029871 for ; Mon, 8 Aug 2022 01:06:41 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=tIc0GqriIATRd3YsoFGFFX/ADaDsq+Z9u4nLJpbomsE=; b=kN0eZcXm/K5e6xn1Ap82ccIlux5koFurgEg7XX8fMtmeNIwQJAnLqxXU1u0h7vVRT9JL CjhLHo1w3B7ZYr76OO62EE34wXcS8E8Ib4H5vtm8YBFJLBhuOGvuPSfqq8bT1527Tsxw N7aYpKFt3ikF6giSLbCkSU/YeX//f2uYLIlLw8h1LWt7IuQ0wyYc+2w89XRMGwOIGbuj Chf7snXIKvfkjTI0Z0YnxsvPOnle6CHzpSGWV/fQPZC232tS57lWNq67Uy3WHpjsHnY0 aXMBT6dsgwQcX9wP6sUzJ/B9FEo/I+yOvUylQ55dktrRGejQ0vZsYKppMaNMlRpfnF3h +A== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3hsnnqnf89-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:41 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Aug 2022 01:06:39 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Mon, 8 Aug 2022 01:06:39 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id A90503F7043; Mon, 8 Aug 2022 01:06:37 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Tejasree Kondoj , Archana Muniganti , Subject: [PATCH 13/18] crypto/cnxk: enable IE engine for Chacha-Poly Date: Mon, 8 Aug 2022 13:36:01 +0530 Message-ID: <20220808080606.220-14-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: S-__keGDxhnAkKYxgD5yyqLxk0K6vACh X-Proofpoint-GUID: S-__keGDxhnAkKYxgD5yyqLxk0K6vACh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Tejasree Kondoj Change engine group of Chacha20-Poly1305 as it is supported on IE engines also. Signed-off-by: Tejasree Kondoj --- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index a73c156d01..85bb1d27a1 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -611,7 +611,7 @@ cnxk_cpt_inst_w7_get(struct cnxk_se_sess *sess, struct roc_cpt *roc_cpt) inst_w7.s.cptr = (uint64_t)&sess->roc_se_ctx.se_ctx; /* Set the engine group */ - if (sess->zsk_flag || sess->chacha_poly || sess->aes_ctr_eea2) + if (sess->zsk_flag || sess->aes_ctr_eea2) inst_w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_SE]; else inst_w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_IE]; From patchwork Mon Aug 8 08:06:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114704 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1D8F2A034C; Mon, 8 Aug 2022 10:07:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 429B842C7F; Mon, 8 Aug 2022 10:06:46 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id AC56742C60 for ; Mon, 8 Aug 2022 10:06:44 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277N0eba006022 for ; Mon, 8 Aug 2022 01:06:44 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=qNQsbbeY4cj7n4wpMdOcUBYpUBDk+Q/rI19BLmkS6sA=; b=lJTK6+LgDVq8fAzqjsYSF8pbu3GmjJC9N3pH3617F7mWFDV6DfNWziyS8ZmTSbjjtXpV zmCmJXSJefUk7gRxR3vZlXAHwm84bswiwuxP+JO22OrupccA/ks+4kqhy1FwJJxhDtBN 4bO/DmT/Bof5fs2DEoURyEiyvUTFsP5GfamefZ67a1G9qt4CtgNoPFluc9QVhJ5SYCuS vBReCD0QMDyljyADNnsAW8CKZ0TJMWAM3Anu28s1CVfaGseBBYqhM++imB6CDP3j2pqL uZxZnsEWzpqzwzvB4Ka91HIpCSmsIdu9nrmOSgooo3sufaD8BeyqRANmcC7HRf7/Ty3+ EA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxf4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:43 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Aug 2022 01:06:42 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Mon, 8 Aug 2022 01:06:42 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id 2B1343F7051; Mon, 8 Aug 2022 01:06:39 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 14/18] crypto/cnxk: use dedicated dp threads depending on operation Date: Mon, 8 Aug 2022 13:36:02 +0530 Message-ID: <20220808080606.220-15-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: tox9xNC9hTAOtTqDMxuWM7CDh1WnDrvW X-Proofpoint-ORIG-GUID: tox9xNC9hTAOtTqDMxuWM7CDh1WnDrvW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Identify the datapath thread to be used during session create. This can be used to call right function early on to avoid multiple session specific checks in datapath functions. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 29 +++++++++++++ drivers/crypto/cnxk/cnxk_se.h | 52 +++++++++++++++--------- 2 files changed, 61 insertions(+), 20 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index 85bb1d27a1..cf91b92c2c 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -625,6 +625,7 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id, struct rte_cryptodev_sym_session *sess, struct rte_mempool *pool) { + enum cpt_dp_thread_type thr_type; struct cnxk_se_sess *sess_priv; void *priv; int ret; @@ -642,6 +643,34 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id, if (ret) goto priv_put; + if (sess_priv->cpt_op & ROC_SE_OP_CIPHER_MASK) { + switch (sess_priv->roc_se_ctx.fc_type) { + case ROC_SE_FC_GEN: + if (sess_priv->aes_gcm || sess_priv->chacha_poly) + thr_type = CPT_DP_THREAD_TYPE_FC_AEAD; + else + thr_type = CPT_DP_THREAD_TYPE_FC_CHAIN; + break; + case ROC_SE_PDCP: + thr_type = CPT_DP_THREAD_TYPE_PDCP; + break; + case ROC_SE_KASUMI: + thr_type = CPT_DP_THREAD_TYPE_KASUMI; + break; + case ROC_SE_PDCP_CHAIN: + thr_type = CPT_DP_THREAD_TYPE_PDCP_CHAIN; + break; + default: + plt_err("Invalid op type"); + ret = -ENOTSUP; + goto priv_put; + } + } else { + thr_type = CPT_DP_THREAD_AUTH_ONLY; + } + + sess_priv->dp_thr_type = thr_type; + if ((sess_priv->roc_se_ctx.fc_type == ROC_SE_HASH_HMAC) && cpt_mac_len_verify(&xform->auth)) { plt_dp_err("MAC length is not supported"); diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 3de9718f8b..dc9621fb84 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -16,6 +16,15 @@ (sizeof(struct roc_se_iov_ptr) + \ (sizeof(struct roc_se_buf_ptr) * ROC_SE_MAX_SG_CNT)) +enum cpt_dp_thread_type { + CPT_DP_THREAD_TYPE_FC_CHAIN = 0x1, + CPT_DP_THREAD_TYPE_FC_AEAD, + CPT_DP_THREAD_TYPE_PDCP, + CPT_DP_THREAD_TYPE_PDCP_CHAIN, + CPT_DP_THREAD_TYPE_KASUMI, + CPT_DP_THREAD_AUTH_ONLY, +}; + struct cnxk_se_sess { uint16_t cpt_op : 4; uint16_t zsk_flag : 4; @@ -29,7 +38,7 @@ struct cnxk_se_sess { uint16_t aes_ctr_eea2 : 1; uint16_t zs_cipher : 4; uint16_t zs_auth : 4; - uint16_t rsvd2 : 8; + uint16_t dp_thr_type : 8; uint16_t aad_length; uint8_t mac_len; uint8_t iv_length; @@ -2370,7 +2379,7 @@ prepare_iov_from_pkt_inplace(struct rte_mbuf *pkt, static __rte_always_inline int fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, - struct cpt_inst_s *inst, const bool is_kasumi) + struct cpt_inst_s *inst, const bool is_kasumi, const bool is_aead) { struct rte_crypto_sym_op *sym_op = cop->sym; void *mdata = NULL; @@ -2419,7 +2428,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, m_src = sym_op->m_src; m_dst = sym_op->m_dst; - if (sess->aes_gcm || sess->chacha_poly) { + if (is_aead) { struct rte_mbuf *m; uint8_t *salt; uint8_t *aad_data; @@ -3060,26 +3069,29 @@ static __rte_always_inline int __rte_hot cpt_sym_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cnxk_se_sess *sess, struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst) { - uint64_t cpt_op = sess->cpt_op; int ret; - if (cpt_op & ROC_SE_OP_CIPHER_MASK) { - switch (sess->roc_se_ctx.fc_type) { - case ROC_SE_PDCP_CHAIN: - ret = fill_pdcp_chain_params(op, sess, &qp->meta_info, infl_req, inst); - break; - case ROC_SE_PDCP: - ret = fill_pdcp_params(op, sess, &qp->meta_info, infl_req, inst); - break; - case ROC_SE_KASUMI: - ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, true); - break; - default: - ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, false); - break; - } - } else { + switch (sess->dp_thr_type) { + case CPT_DP_THREAD_TYPE_PDCP: + ret = fill_pdcp_params(op, sess, &qp->meta_info, infl_req, inst); + break; + case CPT_DP_THREAD_TYPE_FC_CHAIN: + ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, false, false); + break; + case CPT_DP_THREAD_TYPE_FC_AEAD: + ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, false, true); + break; + case CPT_DP_THREAD_TYPE_PDCP_CHAIN: + ret = fill_pdcp_chain_params(op, sess, &qp->meta_info, infl_req, inst); + break; + case CPT_DP_THREAD_TYPE_KASUMI: + ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, true, false); + break; + case CPT_DP_THREAD_AUTH_ONLY: ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, inst); + break; + default: + ret = -EINVAL; } return ret; From patchwork Mon Aug 8 08:06:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114705 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DC904A034C; Mon, 8 Aug 2022 10:07:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 301E942C88; Mon, 8 Aug 2022 10:06:48 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 4CF4B42C83 for ; Mon, 8 Aug 2022 10:06:47 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277N0RZ4005916 for ; Mon, 8 Aug 2022 01:06:46 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=+eR4+SAOOB8HpaHv4i/xB0Vbm/PRHvFcJVxhlZZro14=; b=kaf1nOPt+yNxv1DSs5NShDu1WLC6ACPBbv757hRxPijQ36/otGqymZxatGFeRo0oZ3PG HExo1/o0rN+BM3iDPiFBhA35mA81Vdtgcnmvu2bVlFuPeYwAd1MDDHRfn/d4FjeffrPf 4tYvn0or2KPCGoyD0PCavX4DlIhZiLflwfOEEfsjawz/fboGoMC47Z9cj0d1FOifI8+r mVo3RUhhdDGNt46G+unti4b8vwRMguzpSTUvVpztizEPDef/iVpJKTHA+5hHEPQ2igYk TfofMxbOTMbqhT5+K3/7BSGqsQ91qOs3e+wCL30s1jnqCbHOj13x6/JmYLU/Bu73jtJU jg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxfb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:46 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 8 Aug 2022 01:06:44 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 8 Aug 2022 01:06:44 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id C30AE3F7043; Mon, 8 Aug 2022 01:06:42 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 15/18] crypto/cnxk: remove unused ctx buf len Date: Mon, 8 Aug 2022 13:36:03 +0530 Message-ID: <20220808080606.220-16-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: rEoNluYLLskaRoQCZmWYNcBzpad4GLtv X-Proofpoint-ORIG-GUID: rEoNluYLLskaRoQCZmWYNcBzpad4GLtv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Ctx len buf field is unused. Remove it. Signed-off-by: Anoob Joseph --- drivers/common/cnxk/roc_se.h | 45 +++++++++++++++-------------------- drivers/crypto/cnxk/cnxk_se.h | 27 +++++++++++---------- 2 files changed, 33 insertions(+), 39 deletions(-) diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h index 86bb3aa79d..d1a87a96da 100644 --- a/drivers/common/cnxk/roc_se.h +++ b/drivers/common/cnxk/roc_se.h @@ -287,32 +287,6 @@ struct roc_se_iov_ptr { struct roc_se_buf_ptr bufs[]; }; -struct roc_se_fc_params { - /* 0th cache line */ - union { - struct roc_se_buf_ptr bufs[1]; - struct { - struct roc_se_iov_ptr *src_iov; - struct roc_se_iov_ptr *dst_iov; - }; - }; - void *iv_buf; - void *auth_iv_buf; - struct roc_se_buf_ptr meta_buf; - struct roc_se_buf_ptr ctx_buf; - uint32_t rsvd2; - uint8_t rsvd3; - uint8_t iv_ovr; - uint8_t cipher_iv_len; - uint8_t auth_iv_len; - - /* 1st cache line */ - struct roc_se_buf_ptr aad_buf __plt_cache_aligned; - struct roc_se_buf_ptr mac_buf; -}; - -PLT_STATIC_ASSERT((offsetof(struct roc_se_fc_params, aad_buf) % 128) == 0); - #define ROC_SE_PDCP_ALG_TYPE_ZUC 0 #define ROC_SE_PDCP_ALG_TYPE_SNOW3G 1 #define ROC_SE_PDCP_ALG_TYPE_AES_CTR 2 @@ -349,6 +323,25 @@ struct roc_se_ctx { uint8_t *auth_key; }; +struct roc_se_fc_params { + union { + struct roc_se_buf_ptr bufs[1]; + struct { + struct roc_se_iov_ptr *src_iov; + struct roc_se_iov_ptr *dst_iov; + }; + }; + void *iv_buf; + void *auth_iv_buf; + struct roc_se_ctx *ctx; + struct roc_se_buf_ptr meta_buf; + uint8_t cipher_iv_len; + uint8_t auth_iv_len; + + struct roc_se_buf_ptr aad_buf; + struct roc_se_buf_ptr mac_buf; +}; + static inline void roc_se_zuc_bytes_swap(uint8_t *arr, int len) { diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index dc9621fb84..44b31bfe97 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -334,7 +334,7 @@ cpt_digest_gen_prep(uint32_t flags, uint64_t d_lens, uint32_t g_size_bytes, s_size_bytes; union cpt_inst_w4 cpt_inst_w4; - ctx = params->ctx_buf.vaddr; + ctx = params->ctx; hash_type = ctx->hash_type; mac_len = ctx->mac_len; @@ -478,7 +478,8 @@ cpt_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, aad_len = fc_params->aad_buf.size; aad_buf = &fc_params->aad_buf; } - se_ctx = fc_params->ctx_buf.vaddr; + + se_ctx = fc_params->ctx; cipher_type = se_ctx->enc_cipher; hash_type = se_ctx->hash_type; mac_len = se_ctx->mac_len; @@ -777,7 +778,7 @@ cpt_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, aad_buf = &fc_params->aad_buf; } - se_ctx = fc_params->ctx_buf.vaddr; + se_ctx = fc_params->ctx; hash_type = se_ctx->hash_type; mac_len = se_ctx->mac_len; op_minor = se_ctx->template_w4.s.opcode_minor; @@ -1087,7 +1088,7 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, return -1; } - se_ctx = params->ctx_buf.vaddr; + se_ctx = params->ctx; mac_len = se_ctx->mac_len; pdcp_ci_alg = se_ctx->pdcp_ci_alg; pdcp_auth_alg = se_ctx->pdcp_auth_alg; @@ -1281,7 +1282,7 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, uint8_t pack_iv = 0; union cpt_inst_w4 cpt_inst_w4; - se_ctx = params->ctx_buf.vaddr; + se_ctx = params->ctx; flags = se_ctx->zsk_flags; mac_len = se_ctx->mac_len; @@ -1562,7 +1563,7 @@ cpt_kasumi_enc_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, encr_data_len = ROC_SE_ENCR_DLEN(d_lens); auth_data_len = ROC_SE_AUTH_DLEN(d_lens); - se_ctx = params->ctx_buf.vaddr; + se_ctx = params->ctx; flags = se_ctx->zsk_flags; mac_len = se_ctx->mac_len; @@ -1751,7 +1752,7 @@ cpt_kasumi_dec_prep(uint64_t d_offs, uint64_t d_lens, encr_offset = ROC_SE_ENCR_OFFSET(d_offs) / 8; encr_data_len = ROC_SE_ENCR_DLEN(d_lens); - se_ctx = params->ctx_buf.vaddr; + se_ctx = params->ctx; flags = se_ctx->zsk_flags; cpt_inst_w4.u64 = 0; @@ -1863,7 +1864,7 @@ static __rte_always_inline int cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, struct roc_se_fc_params *fc_params, struct cpt_inst_s *inst) { - struct roc_se_ctx *ctx = fc_params->ctx_buf.vaddr; + struct roc_se_ctx *ctx = fc_params->ctx; uint8_t fc_type; int ret = -1; @@ -2408,7 +2409,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, flags |= ROC_SE_VALID_IV_BUF; fc_params.iv_buf = rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset); - if (sess->aes_ctr && unlikely(sess->iv_length != 16)) { + if (!is_aead && sess->aes_ctr && unlikely(sess->iv_length != 16)) { memcpy((uint8_t *)iv_buf, rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset), @@ -2523,7 +2524,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } } - fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; + fc_params.ctx = &sess->roc_se_ctx; if (!(sess->auth_first) && unlikely(sess->is_null || sess->cpt_op == ROC_SE_OP_DECODE)) inplace = 0; @@ -2651,7 +2652,7 @@ fill_pdcp_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, d_offs = (uint64_t)c_data_off << 16; d_lens = (uint64_t)c_data_len << 32; - fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; + fc_params.ctx = &sess->roc_se_ctx; if (likely(m_dst == NULL || m_src == m_dst)) { fc_params.dst_iov = fc_params.src_iov = (void *)src; @@ -2794,7 +2795,7 @@ fill_pdcp_chain_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, ci_data_length, true); } - fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; + fc_params.ctx = &sess->roc_se_ctx; if (likely((m_dst == NULL || m_dst == m_src)) && inplace) { fc_params.dst_iov = fc_params.src_iov = (void *)src; @@ -2993,7 +2994,7 @@ fill_digest_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, d_lens = sym_op->auth.data.length; - params.ctx_buf.vaddr = &sess->roc_se_ctx; + params.ctx = &sess->roc_se_ctx; if (auth_op == ROC_SE_OP_AUTH_GENERATE) { if (sym_op->auth.digest.data) { From patchwork Mon Aug 8 08:06:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114706 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 122B4A034C; Mon, 8 Aug 2022 10:07:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5239C42C8B; Mon, 8 Aug 2022 10:06:52 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 2C08942C89 for ; Mon, 8 Aug 2022 10:06:50 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277N0RZ5005916 for ; Mon, 8 Aug 2022 01:06:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=yzdtJukYBBK498KjRKiFJONTbglThZ/kE5vR9MfKkvg=; b=Qrag5N8pPzr3J2lKGQbnjXES03vVcW8phPxjul2UJkkLLuiFkrs5ab+rfGaYdljtZzNf Cg9V6hxXzqaknqBoSkModOxXB4tm7+CV3uZ7xz+V74e0GNFsvJJO55V6tiI7DdMi0CSE 4QpAMHPlZ2VsTXVgL2way95h7SJhsXKsyxsDOITibT/lI9WNUGsMOFF+0Lufa0xi4pOJ bfC6NrBwOjey3puW2l0ggdFgIRKK90tBzYqTuZAmAzKOT7+7JCmXL3MsqOXNfbYlSSwP 7F5RD0JEPg2Sznqzm96Hq5YwinKnTRfwr330S1M/q3cb5zAQ8HbZ3arWoOAKzjl90g2v dQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxfh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:49 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Aug 2022 01:06:47 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 8 Aug 2022 01:06:47 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id 4EE083F7051; Mon, 8 Aug 2022 01:06:45 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Shijith Thotton , Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 16/18] drivers: change crypto adapter datapath error print to debug Date: Mon, 8 Aug 2022 13:36:04 +0530 Message-ID: <20220808080606.220-17-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 0XoYeAcr9nKwv37s7GoExcjEcXi--E0I X-Proofpoint-ORIG-GUID: 0XoYeAcr9nKwv37s7GoExcjEcXi--E0I X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Shijith Thotton Errors in crypto adapter datapath can be handled gracefully. So the error print can be moved under debug. Signed-off-by: Shijith Thotton --- drivers/common/cnxk/roc_platform.h | 2 ++ drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 502f243a81..0cde3e0cc3 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -269,6 +269,8 @@ extern int cnxk_logtype_ree; RTE_LOG_DP(ERR, PMD, "%s():%u " fmt "\n", __func__, __LINE__, ##args) #define plt_dp_info(fmt, args...) \ RTE_LOG_DP(INFO, PMD, "%s():%u " fmt "\n", __func__, __LINE__, ##args) +#define plt_dp_dbg(fmt, args...) \ + RTE_LOG_DP(DEBUG, PMD, "%s():%u " fmt "\n", __func__, __LINE__, ##args) #ifdef __cplusplus #define CNXK_PCI_ID(subsystem_dev, dev) \ diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 1b70d02e2a..586941cd70 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -412,7 +412,7 @@ ca_lmtst_burst_submit(struct cn10k_sso_hws *ws, uint64_t w2[], struct cnxk_cpt_q ret = cn10k_cpt_fill_inst(qp, &op[i], inst, infl_req); if (unlikely(ret != 1)) { - plt_dp_err("Could not process op: %p", op[i]); + plt_dp_dbg("Could not process op: %p", op[i]); if (i != 0) goto submit; else From patchwork Mon Aug 8 08:06:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114707 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F1DEAA034C; Mon, 8 Aug 2022 10:08:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3F99F42C60; Mon, 8 Aug 2022 10:06:54 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id CA02042C98 for ; Mon, 8 Aug 2022 10:06:52 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277N0NGX005827 for ; Mon, 8 Aug 2022 01:06:52 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=4XaXNmXDy6LZe8U10TIjmztsFqO/SLT0I/gX7GfL2DQ=; b=UIVcYpOeB+bqQlOHc8oW0APMBNpj2kX29sVlxHNBxCCCTslU2/j85iyOdPRY0f2sWrso o4w+G8kGU3yv2lv4AxgAFdQ8loDYV5uU91DWP0fA9iRwhJg179wnnqZjvee8QRIj/ElE DLkpF0v8kovi3qa5cuKq1eo3gyEbmT6bRIKkH8stXq17ygGEB5+IxvhsGM4mWcEvrA4D KEowJ3eArNstObXOIojtn9+7F2fvVcngsP1pwvlCFvOmDXBIM6jzcybXyIkZbfpTSK71 /lcpE5SW8QudNdps9HKdd8r/bm+9t/bVipih7NYuTGw/Zd5OqKXavE932yNUW0y+xWXI KQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hsqtmmxfu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:52 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 8 Aug 2022 01:06:50 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 8 Aug 2022 01:06:50 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id 09D753F705F; Mon, 8 Aug 2022 01:06:47 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Vamsi Attunuru , Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 17/18] crypto/cnxk: update flow label copy capability Date: Mon, 8 Aug 2022 13:36:05 +0530 Message-ID: <20220808080606.220-18-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: nwxNgxeF-QeDsrBsJGrL1XcECB73mXd9 X-Proofpoint-ORIG-GUID: nwxNgxeF-QeDsrBsJGrL1XcECB73mXd9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vamsi Attunuru Update flow label copy capability in IPsec options. Signed-off-by: Vamsi Attunuru --- drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index fdc646a6fc..ffb0c289a0 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -1294,6 +1294,7 @@ cn10k_sec_caps_update(struct rte_security_capability *sec_cap) sec_cap->ipsec.options.l4_csum_enable = 1; sec_cap->ipsec.options.stats = 1; sec_cap->ipsec.options.esn = 1; + sec_cap->ipsec.options.copy_flabel = 1; sec_cap->ipsec.replay_win_sz_max = ROC_AR_WIN_SIZE_MAX; } From patchwork Mon Aug 8 08:06:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114708 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DD064A0540; Mon, 8 Aug 2022 10:08:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2F66F40A79; Mon, 8 Aug 2022 10:06:56 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 60A364067B for ; Mon, 8 Aug 2022 10:06:55 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 277NrifB014579 for ; Mon, 8 Aug 2022 01:06:54 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=3KlGUUUdcOZf1DIrad0yrIKI+MclmtvGATrYZHUBNwQ=; b=kReXVTCkd5gFsjUwgQ/xkiN45r5ltf4KAs/fu0HmBQ6gQd2inLqRhdtuIpktWBtI8Oob gRWr//0xD8MIAkRKgN75s1b0fZ2+6zedacQape0VIbK6BkqyqPz7WkzXLl6pQAZZhd2k +ynl1ip+EmKCTecfm0+gW1U+9BiCKO1IPigBfq2d+sbH76Nra+7u5zrZEX4fHAl215Vu Tb2nIjbXfO0xzKnGdUbHe+7krd6irZSiZzZjweNYzMFXwv8Pmt5OjSaixNTVlD9op51p Dua5FFCvhg6vlR9oV5+OwxQUyiGJnEbmQgubXN1pfccL1jUWm8uVtq9yrFgG8N2UsHoQ QA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3hsnnqnf9e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 08 Aug 2022 01:06:54 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Aug 2022 01:06:52 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 8 Aug 2022 01:06:53 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.160.62]) by maili.marvell.com (Postfix) with ESMTP id BD8263F7051; Mon, 8 Aug 2022 01:06:50 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Volodymyr Fialko , Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 18/18] crypto/cnxk: add support for DOCSIS algorithm Date: Mon, 8 Aug 2022 13:36:06 +0530 Message-ID: <20220808080606.220-19-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808080606.220-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 8tIMgiISKDLCgIQPFgGN-yaj1aquokPZ X-Proofpoint-GUID: 8tIMgiISKDLCgIQPFgGN-yaj1aquokPZ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-08_05,2022-08-05_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko Add support for offloading RTE_CRYPTO_CIPHER_AES_DOCSISBPI and RTE_CRYPTO_CIPHER_DES_DOCSISBPI algorithms to symmetric crypto session. Signed-off-by: Volodymyr Fialko --- doc/guides/cryptodevs/features/cn9k.ini | 2 + doc/guides/rel_notes/release_22_11.rst | 1 + drivers/common/cnxk/roc_se.c | 25 ++++++++- drivers/common/cnxk/roc_se.h | 11 ++-- drivers/crypto/cnxk/cnxk_cryptodev.h | 2 +- .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 52 +++++++++++++++++++ drivers/crypto/cnxk/cnxk_se.h | 12 +++++ 7 files changed, 99 insertions(+), 6 deletions(-) diff --git a/doc/guides/cryptodevs/features/cn9k.ini b/doc/guides/cryptodevs/features/cn9k.ini index 98ad7cf10a..c3d131db1a 100644 --- a/doc/guides/cryptodevs/features/cn9k.ini +++ b/doc/guides/cryptodevs/features/cn9k.ini @@ -35,6 +35,8 @@ DES CBC = Y KASUMI F8 = Y SNOW3G UEA2 = Y ZUC EEA3 = Y +AES DOCSIS BPI = Y +DES DOCSIS BPI = Y ; ; Supported authentication algorithms of 'cn9k' crypto driver. diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index 333f66bef3..7fab9d6550 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -58,6 +58,7 @@ New Features * **Updated Marvell cnxk crypto driver.** * Added AES-CCM support in lookaside protocol (IPsec) for CN9K & CN10K. + * Added AES & DES DOCSIS algorithm support in lookaside crypto for CN9K. Removed Items diff --git a/drivers/common/cnxk/roc_se.c b/drivers/common/cnxk/roc_se.c index 8d6446c3a0..2663480099 100644 --- a/drivers/common/cnxk/roc_se.c +++ b/drivers/common/cnxk/roc_se.c @@ -63,6 +63,7 @@ cpt_ciph_type_set(roc_se_cipher_type type, struct roc_se_ctx *ctx, break; case ROC_SE_DES3_CBC: case ROC_SE_DES3_ECB: + case ROC_SE_DES_DOCSISBPI: fc_type = ROC_SE_FC_GEN; break; case ROC_SE_AES_CBC: @@ -70,6 +71,7 @@ cpt_ciph_type_set(roc_se_cipher_type type, struct roc_se_ctx *ctx, case ROC_SE_AES_CFB: case ROC_SE_AES_CTR: case ROC_SE_AES_GCM: + case ROC_SE_AES_DOCSISBPI: if (unlikely(cpt_ciph_aes_key_validate(key_len) != 0)) return -1; fc_type = ROC_SE_FC_GEN; @@ -451,7 +453,7 @@ roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, uint8_t *zuc_const; uint32_t keyx[4]; uint8_t *ci_key; - int ret; + int i, ret; zs_ch_ctx = &se_ctx->se_ctx.zs_ch_ctx; @@ -531,6 +533,27 @@ roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, memset(fctx->hmac.ipad, 0, sizeof(fctx->hmac.ipad)); memcpy(fctx->hmac.ipad, &key[key_len], key_len); break; + case ROC_SE_AES_DOCSISBPI: + /* + * DOCSIS uses the combination of AES-CBC and residual termination blocks that are + * less than 128. Pass it as regular AES-CBC cipher to CPT, but keep type in + * se_ctx as AES_DOCSISBPI to skip block size checks in instruction preparation. + */ + cpt_ciph_aes_key_type_set(fctx, key_len); + fctx->enc.enc_cipher = ROC_SE_AES_CBC; + memcpy(fctx->enc.encr_key, key, key_len); + goto success; + case ROC_SE_DES_DOCSISBPI: + /* See case ROC_SE_DES3_CBC: for explanation */ + for (i = 0; i < 3; i++) + memcpy(fctx->enc.encr_key + key_len * i, key, key_len); + /* + * DOCSIS uses DES-CBC mode with special handling of residual termination blocks + * that are less than 64 bits. Pass it as regular DES-CBC, but keep type in + * se_ctx as DES_DOCSISBPI to skip block size checks in instruction preparation. + */ + fctx->enc.enc_cipher = ROC_SE_DES3_CBC; + goto success; case ROC_SE_SNOW3G_UEA2: if (chained_op == true) { struct roc_se_onk_zuc_chain_ctx *ctx = diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h index d1a87a96da..e70a197d4f 100644 --- a/drivers/common/cnxk/roc_se.h +++ b/drivers/common/cnxk/roc_se.h @@ -10,6 +10,7 @@ #define ROC_SE_FC_MINOR_OP_ENCRYPT 0x0 #define ROC_SE_FC_MINOR_OP_DECRYPT 0x1 #define ROC_SE_FC_MINOR_OP_HMAC_FIRST 0x10 +#define ROC_SE_FC_MINOR_OP_DOCSIS 0x40 #define ROC_SE_MAJOR_OP_HASH 0x34 #define ROC_SE_MAJOR_OP_HMAC 0x35 @@ -17,10 +18,10 @@ #define ROC_SE_MAJOR_OP_KASUMI 0x38 #define ROC_SE_MAJOR_OP_PDCP_CHAIN 0x3C -#define ROC_SE_MAJOR_OP_MISC 0x01 -#define ROC_SE_MISC_MINOR_OP_PASSTHROUGH 0x03 -#define ROC_SE_MISC_MINOR_OP_DUMMY 0x04 -#define ROC_SE_MISC_MINOR_OP_HW_SUPPORT 0x08 +#define ROC_SE_MAJOR_OP_MISC 0x01ULL +#define ROC_SE_MISC_MINOR_OP_PASSTHROUGH 0x03ULL +#define ROC_SE_MISC_MINOR_OP_DUMMY 0x04ULL +#define ROC_SE_MISC_MINOR_OP_HW_SUPPORT 0x08ULL #define ROC_SE_MAX_AAD_SIZE 64 #define ROC_SE_MAX_MAC_LEN 64 @@ -125,6 +126,8 @@ typedef enum { ROC_SE_AES_CTR_EEA2 = 0x92, ROC_SE_KASUMI_F8_CBC = 0x93, ROC_SE_KASUMI_F8_ECB = 0x94, + ROC_SE_AES_DOCSISBPI = 0x95, + ROC_SE_DES_DOCSISBPI = 0x96, } roc_se_cipher_type; typedef enum { diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index a3dcfbfa6d..588760cfb0 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -10,7 +10,7 @@ #include "roc_cpt.h" -#define CNXK_CPT_MAX_CAPS 35 +#define CNXK_CPT_MAX_CAPS 37 #define CNXK_SEC_CRYPTO_MAX_CAPS 14 #define CNXK_SEC_MAX_CAPS 9 #define CNXK_AE_EC_ID_MAX 8 diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index ffb0c289a0..1fb35f54cd 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -697,6 +697,49 @@ static const struct rte_cryptodev_capabilities caps_des[] = { }, }; +static const struct rte_cryptodev_capabilities caps_docsis[] = { + { /* AES DOCSIS BPI */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_DOCSISBPI, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 16 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* DES DOCSIS BPI */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_DES_DOCSISBPI, + .block_size = 8, + .key_size = { + .min = 8, + .max = 8, + .increment = 0 + }, + .iv_size = { + .min = 8, + .max = 8, + .increment = 0 + } + }, } + }, } + }, +}; + static const struct rte_cryptodev_capabilities caps_null[] = { { /* NULL (AUTH) */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, @@ -1158,6 +1201,12 @@ cn10k_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[]) } } +static void +cn9k_crypto_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos) +{ + cpt_caps_add(cnxk_caps, cur_pos, caps_docsis, RTE_DIM(caps_docsis)); +} + static void crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[], union cpt_eng_caps *hw_caps) @@ -1172,6 +1221,9 @@ crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[], CPT_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, kasumi); CPT_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, des); + if (!roc_model_is_cn10k()) + cn9k_crypto_caps_add(cnxk_caps, &cur_pos); + cpt_caps_add(cnxk_caps, &cur_pos, caps_null, RTE_DIM(caps_null)); cpt_caps_add(cnxk_caps, &cur_pos, caps_end, RTE_DIM(caps_end)); diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 44b31bfe97..b7d4b19d01 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2032,6 +2032,18 @@ fill_sess_cipher(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) enc_type = ROC_SE_AES_ECB; cipher_key_len = 16; break; + case RTE_CRYPTO_CIPHER_AES_DOCSISBPI: + /* Set DOCSIS flag */ + sess->roc_se_ctx.template_w4.s.opcode_minor |= ROC_SE_FC_MINOR_OP_DOCSIS; + enc_type = ROC_SE_AES_DOCSISBPI; + cipher_key_len = 16; + break; + case RTE_CRYPTO_CIPHER_DES_DOCSISBPI: + /* Set DOCSIS flag */ + sess->roc_se_ctx.template_w4.s.opcode_minor |= ROC_SE_FC_MINOR_OP_DOCSIS; + enc_type = ROC_SE_DES_DOCSISBPI; + cipher_key_len = 8; + break; case RTE_CRYPTO_CIPHER_3DES_CTR: case RTE_CRYPTO_CIPHER_AES_F8: case RTE_CRYPTO_CIPHER_ARC4: