From patchwork Thu Dec 21 12:35:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135431 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 441C043750; Thu, 21 Dec 2023 13:35:56 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D927C40A81; Thu, 21 Dec 2023 13:35:53 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 84E3040A7A for ; Thu, 21 Dec 2023 13:35:52 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS2u019305 for ; Thu, 21 Dec 2023 04:35:51 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=rTDiIAj8REKV1Wc9IF8ejh75L/NXp+/kd0RwR0pH7ro=; b=NPG AH4CinaMgAYZ4yG+NVqf0xOQTiMFW73mdSKUf/8AZvcHdQjBtLQ/hnrItYIn9k+v MbYy/H3BwQnPdN83V7ccMskKfyxKTvHZPMx6x+INfH7eO8ueZTLm3u8VY8yGHo+F Cgz9efxeOmYpOmfscLZKjfRRwj2/PLKNL9GIlCWRyt99TY/HaddJdbxSOjEvZtqf +mY02y9waEg3XFDBnC9s8C2H5y8Ve10hCfNRxPL3wfOuDJf1CNH+Bbdx12U9wL27 W0osfJVXz9ycQ3d/UShjh189er1uxQKyXypQJ7yeWhEX0xgZZlDFcAx7XfAZlHWA hRkcWP4MT/aeL6fxYKA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0fh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:35:51 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:35:50 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:35:50 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 46B453F7079; Thu, 21 Dec 2023 04:35:46 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Jerin Jacob , Vidya Sagar Velumuri , Tejasree Kondoj , Subject: [PATCH 01/24] common/cnxk: fix memory leak Date: Thu, 21 Dec 2023 18:05:22 +0530 Message-ID: <20231221123545.510-2-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: FlikiKwoYkh9rk5dzT75QS_dQKg8durh X-Proofpoint-ORIG-GUID: FlikiKwoYkh9rk5dzT75QS_dQKg8durh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Akhil Goyal dev_init() acquires some resources which need to be cleaned in case a failure is observed afterwards. Fixes: c045d2e5cbbc ("common/cnxk: add CPT configuration") Signed-off-by: Akhil Goyal --- drivers/common/cnxk/roc_cpt.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/common/cnxk/roc_cpt.c b/drivers/common/cnxk/roc_cpt.c index 981e85a204..4e23d8c135 100644 --- a/drivers/common/cnxk/roc_cpt.c +++ b/drivers/common/cnxk/roc_cpt.c @@ -756,7 +756,7 @@ roc_cpt_dev_init(struct roc_cpt *roc_cpt) rc = dev_init(dev, pci_dev); if (rc) { plt_err("Failed to init roc device"); - goto fail; + return rc; } cpt->pci_dev = pci_dev; @@ -788,6 +788,7 @@ roc_cpt_dev_init(struct roc_cpt *roc_cpt) return 0; fail: + dev_fini(dev, pci_dev); return rc; } From patchwork Thu Dec 21 12:35:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135432 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 46A8643750; Thu, 21 Dec 2023 13:36:02 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 041DA40EF0; Thu, 21 Dec 2023 13:35:57 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 5B1EA40EE2 for ; Thu, 21 Dec 2023 13:35:55 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVojB019540 for ; Thu, 21 Dec 2023 04:35:54 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=u6aVUvW6PSC8Q+8d0o8t16mCOT9uvvXh7wpGraD1YxA=; b=Y9x h0rSGXsn7CB5Bthk/bKrlD7Psau0hi9xdR/PKCERGqV52vZ5QRMcpxjpz5zMg6Kh NaqTLHY/LBQJcqEO2gljswcy7/aFsd/kO9dpjhj5LxkDRKJnI+qpmDG3ph98p3e7 aRAmxHQ67+gg7CtGpKcFjVPjp3cnaPGt1D2N0Kr6wEUCm/hyZvQRURp8f5dE5maY FQx/rdrQosQyrUqWxXB916L6gY0Bz5Jn9Ze8btp55cvcuGACFiEOMsakK3FUUfyQ Rp5RbIMdSk1DjiOCEDvQ29RImW7OsppnrM4gdiZIfzVIM3GXscNHhFP4PmCgDNGi 0YO7YoMMZuG0HYslpoA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0fu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:35:54 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:35:52 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:35:52 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 11E7C3F707E; Thu, 21 Dec 2023 04:35:49 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Jerin Jacob , Vidya Sagar Velumuri , Tejasree Kondoj , Subject: [PATCH 02/24] crypto/cnxk: use common macro Date: Thu, 21 Dec 2023 18:05:23 +0530 Message-ID: <20231221123545.510-3-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: sgz4O2vgL8RvU1OxXedF9W0SHMQSCKRh X-Proofpoint-ORIG-GUID: sgz4O2vgL8RvU1OxXedF9W0SHMQSCKRh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Having different macros for same purpose may cause issues if one is updated without updating the other. Use same macro by including the header. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_cryptodev.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index d0ad881f2f..f5374131bf 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -8,12 +8,12 @@ #include #include +#include "roc_ae.h" #include "roc_cpt.h" #define CNXK_CPT_MAX_CAPS 55 #define CNXK_SEC_CRYPTO_MAX_CAPS 16 #define CNXK_SEC_MAX_CAPS 9 -#define CNXK_AE_EC_ID_MAX 9 /** * Device private data */ @@ -23,8 +23,8 @@ struct cnxk_cpt_vf { struct rte_cryptodev_capabilities sec_crypto_caps[CNXK_SEC_CRYPTO_MAX_CAPS]; struct rte_security_capability sec_caps[CNXK_SEC_MAX_CAPS]; - uint64_t cnxk_fpm_iova[CNXK_AE_EC_ID_MAX]; - struct roc_ae_ec_group *ec_grp[CNXK_AE_EC_ID_MAX]; + uint64_t cnxk_fpm_iova[ROC_AE_EC_ID_PMAX]; + struct roc_ae_ec_group *ec_grp[ROC_AE_EC_ID_PMAX]; uint16_t max_qps_limit; }; From patchwork Thu Dec 21 12:35:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135433 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7DD6D43750; Thu, 21 Dec 2023 13:36:09 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 22782410F9; Thu, 21 Dec 2023 13:35:59 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 120B7410F9 for ; Thu, 21 Dec 2023 13:35:57 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVlnE019466 for ; Thu, 21 Dec 2023 04:35:57 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=1k7/JdQc45c13GDhhDPO6GJGPVEtfByRsVi6rRBpVuo=; b=QG0 VxbYPCWrIGCPhyn6hwv1nGcbwlDTAYWODd7uUi/pOZXMxoJt0z1ciTmiujUJubrv HZ2XrrclG1iWYzEMwHO36w9+9PpJkn4N41voU7PGl/Ts/E8Tc0HNdW7QKcpBtogX PzleObd7183zrAtvblHBF2LNYgBd/sUfQU73c4h9SZx9LGq/xoya7t94kqATgKr/ ZFCqlfp7k0W+BnttOQKws0BvvGA7mFxYo3WnWB9chUI6HgwTowA2gNBq/y1E8wRa bgPKXWhKHe0KW63M8cnx0GCEyveVnVeyDKl4KSTgAWGUDUcvYUYxphgmF88bfNI1 QGD15StU5G0NKwNz9kw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0g2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:35:57 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:35:55 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:35:55 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id C31023F707D; Thu, 21 Dec 2023 04:35:52 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Tejasree Kondoj , Jerin Jacob , Vidya Sagar Velumuri , Subject: [PATCH 03/24] crypto/cnxk: fallback to SG if headroom is not available Date: Thu, 21 Dec 2023 18:05:24 +0530 Message-ID: <20231221123545.510-4-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: xOM-CHyG_h3k6TzcZO8wD6nknARG_AQt X-Proofpoint-ORIG-GUID: xOM-CHyG_h3k6TzcZO8wD6nknARG_AQt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Tejasree Kondoj Falling back to SG mode for cn9k lookaside IPsec if headroom is not available. Signed-off-by: Tejasree Kondoj --- drivers/crypto/cnxk/cn9k_ipsec_la_ops.h | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h index 85aacb803f..3d0db72775 100644 --- a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h +++ b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h @@ -82,19 +82,13 @@ process_outb_sa(struct cpt_qp_meta_info *m_info, struct rte_crypto_op *cop, extend_tail = rlen - dlen; pkt_len += extend_tail; - if (likely(m_src->next == NULL)) { + if (likely((m_src->next == NULL) && (hdr_len <= data_off))) { if (unlikely(extend_tail > rte_pktmbuf_tailroom(m_src))) { plt_dp_err("Not enough tail room (required: %d, available: %d)", extend_tail, rte_pktmbuf_tailroom(m_src)); return -ENOMEM; } - if (unlikely(hdr_len > data_off)) { - plt_dp_err("Not enough head room (required: %d, available: %d)", hdr_len, - rte_pktmbuf_headroom(m_src)); - return -ENOMEM; - } - m_src->data_len = pkt_len; hdr = PLT_PTR_ADD(m_src->buf_addr, data_off - hdr_len); From patchwork Thu Dec 21 12:35:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135434 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0A24F43750; Thu, 21 Dec 2023 13:36:16 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2DD7842E59; Thu, 21 Dec 2023 13:36:02 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 06FAC42E52 for ; Thu, 21 Dec 2023 13:36:00 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVT4R019308 for ; Thu, 21 Dec 2023 04:36:00 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=eMrG15f0J+RTkmetX/sYwDC1hWDdC42KJi2xXhw68n0=; b=Iya lM34BuJ5OvtfeCtwwIP7Ft6KZlmju1EnHYFvI9AQBKHDtYlvvYtmd1VXes2sKf58 We/bWs5dVf1muZ2uSrKVjxODRXtwmYKd8lcHv7UUtowLDI2E10riNxTRlTbLlxGx OMowDu8qPo10M96FzIwG9DpYcgB99HoAIAnfViTtDgzus8qlBd4+oLl50IJ9jhTl BJEsnqQcOvysuUkuaneeDSWgUgiJVoraabgzyHsHumbCWpkuF3zaJRz7PhG4gWyy jRDEOVgnBG6q1I+H0TbF+91Dcg2nveHVQYOIvmKbVn3wMOTV8Q32AEP97uX3DGIS ZoV0y7/Xj4l/VBtXQOg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0g7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:00 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:35:58 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:35:58 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 9199C3F7073; Thu, 21 Dec 2023 04:35:55 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Jerin Jacob , Vidya Sagar Velumuri , Tejasree Kondoj , Subject: [PATCH 04/24] crypto/cnxk: return microcode completion code Date: Thu, 21 Dec 2023 18:05:25 +0530 Message-ID: <20231221123545.510-5-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: MgGIBR4Kj-tf0zwwmwskUjS4zT3aTAOH X-Proofpoint-ORIG-GUID: MgGIBR4Kj-tf0zwwmwskUjS4zT3aTAOH X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Return microcode completion code in case of errors. This allows applications to check the failure reasons in more granularity. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 997110e3d3..bef7b75810 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -823,6 +823,7 @@ cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *re break; default: cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + cop->aux_flags = res->uc_compcode; return; } @@ -884,6 +885,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, plt_dp_info("Request failed with microcode error"); plt_dp_info("MC completion code 0x%x", res->uc_compcode); + cop->aux_flags = uc_compcode; goto temp_sess_free; } From patchwork Thu Dec 21 12:35:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135435 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5973243750; Thu, 21 Dec 2023 13:36:24 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BFA2942E6A; Thu, 21 Dec 2023 13:36:05 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 0FCD742E5F for ; Thu, 21 Dec 2023 13:36:03 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS2v019305 for ; Thu, 21 Dec 2023 04:36:03 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=M19+VUuhXbw7nKtU90bv15ZdUBveh8Dz5xMICdE7VkM=; b=jXT 3rU41KoI4Uo4gKiZHs3PQR3TsYmEGs/q/j6mxY/F2clotHqh8G5nQXlNzd9Lct2b QcBpvY4tuDTnpjtZk45Sat/c98JtHMxmL+0kwKPNPslGLT+KjTotHRsSe+L+b3yo bGXunH4qjoVSHIL33k+ZwzdNrTMjDpJSGS0Ki97JulTZq0/YWDf6LIp4eIHyLMxe RUJiClnQuimNSyPfhy1YiuWg3KaHXdpVOlUTuYBWro2i8OStX5KBmoOvH1/5/C+g iNPL0lJxAbOLDzRc+g7gmtO5UELiuJd0LhXTOjUg1J5T1uY2xnR50Y7Wx8Ca4BMP BkO9eTFcNn6+PS+i27g== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0ge-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:03 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:01 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:01 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 6978F3F7079; Thu, 21 Dec 2023 04:35:58 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Gowrishankar Muthukrishnan , Jerin Jacob , Vidya Sagar Velumuri , "Tejasree Kondoj" , Subject: [PATCH 05/24] crypto/cnxk: fix ECDH pubkey verify in cn9k Date: Thu, 21 Dec 2023 18:05:26 +0530 Message-ID: <20231221123545.510-6-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: UblRbDkhip4sENj7ky2jxfMRaeEk1rcy X-Proofpoint-ORIG-GUID: UblRbDkhip4sENj7ky2jxfMRaeEk1rcy X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Gowrishankar Muthukrishnan Fix ECDH pubkey verify in cn9k. Fixes: baae0994fa96 ("crypto/cnxk: support ECDH") Signed-off-by: Gowrishankar Muthukrishnan --- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index 34d40b07d4..442cd8e5a9 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -578,7 +578,17 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, if (unlikely(res->uc_compcode)) { if (res->uc_compcode == ROC_SE_ERR_GC_ICV_MISCOMPARE) cop->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; - else + else if (cop->type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC && + cop->sess_type == RTE_CRYPTO_OP_WITH_SESSION && + cop->asym->ecdh.ke_type == RTE_CRYPTO_ASYM_KE_PUB_KEY_VERIFY) { + if (res->uc_compcode == ROC_AE_ERR_ECC_POINT_NOT_ON_CURVE) { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + return; + } else if (res->uc_compcode == ROC_AE_ERR_ECC_PAI) { + cop->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + return; + } + } else cop->status = RTE_CRYPTO_OP_STATUS_ERROR; plt_dp_info("Request failed with microcode error"); From patchwork Thu Dec 21 12:35:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135436 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9AAB143750; Thu, 21 Dec 2023 13:36:31 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EB09A42E64; Thu, 21 Dec 2023 13:36:08 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 5E62D40EE2 for ; Thu, 21 Dec 2023 13:36:07 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVelR019408 for ; Thu, 21 Dec 2023 04:36:06 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=3T+J0XDAPcFQvougORyVAxOLCI8Yx7Ki2u20vKyJsxA=; b=f+E GzQtSfEQM9KCDgCNwa1IjIWnxRaNjx2k1Xym82u+alzyCuxZJ/zIRb2raA96lVYr sgaUYotP7p1zLad2WBQcWkPEyq2l7OYLmjuWXf38ebwlCgmuTiJlQIYGoIFmP4T3 mEwhhhCJlVyQAoOlUPFRNIzxV+jGdo24Iz2Hv4sS6fl6tvFTLWCxsbf3zbTrkAz8 KYmKvlasg2Ki/CexGW7Qqvt4vPq/i4217bag8K6IkmcKLo0wQOhU7Wa7KQjO0hbc jzzlj35nqlWYmgWsFvpFGyUYu8wahXciOYUoZeDtU1lP+Os0ni4Vn+gzYlA+vwI7 PKPvjCsUlN9m0jMAs7A== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0gy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:05 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:04 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:04 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 5D9E23F7098; Thu, 21 Dec 2023 04:36:01 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Aakash Sasidharan , Jerin Jacob , Vidya Sagar Velumuri , "Tejasree Kondoj" , Subject: [PATCH 06/24] crypto/cnxk: enable digest gen for zero len input Date: Thu, 21 Dec 2023 18:05:27 +0530 Message-ID: <20231221123545.510-7-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 3sbs-2yaBSZZDJnrePclR8202DMDUcWA X-Proofpoint-ORIG-GUID: 3sbs-2yaBSZZDJnrePclR8202DMDUcWA X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Aakash Sasidharan With zero length input, digest generation fails with incorrect value. Fix this by completely avoiding the gather component when the input packet has zero data length. Signed-off-by: Aakash Sasidharan --- drivers/crypto/cnxk/cnxk_se.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index c2a807fa94..1aec7dea9f 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2479,7 +2479,7 @@ prepare_iov_from_pkt(struct rte_mbuf *pkt, struct roc_se_iov_ptr *iovec, uint32_ void *seg_data = NULL; int32_t seg_size = 0; - if (!pkt) { + if (!pkt || pkt->data_len == 0) { iovec->buf_cnt = 0; return 0; } From patchwork Thu Dec 21 12:35:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135437 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B271043750; Thu, 21 Dec 2023 13:36:38 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 012EA42E63; Thu, 21 Dec 2023 13:36:12 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id E71A542E77 for ; Thu, 21 Dec 2023 13:36:09 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVT4S019308 for ; Thu, 21 Dec 2023 04:36:09 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=Gu59vUeOcpHvaYbxTR6MW+f0fL9p39zpUavGJiEzI1U=; b=A+c 6MztqLoVgI17qSdYqYXaUPrY9j3pR3WaFH+Cc2fB44lYQlyQu1VAb4v1XltKjYrJ 4XYy7EwXbqBF3fDntfmNM3vrjEMKjtCRn2pdqTPxyo3ehYYaVgN0RCc2FTT7eoAQ Euk1omgyU2q2fUCZ+sFoLeGJlF8b7ziw9eab1yGZYo8uleqUyOPFYmOv1QvdHBC9 g74vJ2wnUkXMJxt0kFq1R62SF9PhGNo+a6u/wZafJ1hDTTNspvnn5XfbBy2etY/S 1EhFacUZMGZrsT3MTJGlNSIkJWxjWmARk40HIUsI20ioVmt4Lc5v4ynagNSJxETW HeHDrg9cZPMX1DXmKzw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0h4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:08 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:07 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:07 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 5E8033F7073; Thu, 21 Dec 2023 04:36:04 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 07/24] crypto/cnxk: enable Rx inject in security lookaside Date: Thu, 21 Dec 2023 18:05:28 +0530 Message-ID: <20231221123545.510-8-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 5dvcKMOkf_8nCNt5urTQRVLzVwHILl24 X-Proofpoint-ORIG-GUID: 5dvcKMOkf_8nCNt5urTQRVLzVwHILl24 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Add Rx inject fastpath API. Add devargs to specify an LF to be used for Rx inject. When the RX inject feature flag is enabled: 1. Reserve a CPT LF to use for RX Inject mode. 2. Enable RXC and disable full packet mode for that LF. Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri --- doc/guides/cryptodevs/cnxk.rst | 12 ++ doc/guides/rel_notes/release_24_03.rst | 3 + drivers/common/cnxk/hw/cpt.h | 9 ++ drivers/common/cnxk/roc_cpt.c | 11 +- drivers/common/cnxk/roc_cpt.h | 3 +- drivers/common/cnxk/roc_cpt_priv.h | 2 +- drivers/common/cnxk/roc_ie_ot.c | 14 +-- drivers/common/cnxk/roc_mbox.h | 2 + drivers/common/cnxk/roc_nix_inl.c | 2 +- drivers/common/cnxk/roc_nix_inl_dev.c | 2 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 123 +++++++++++++++++++ drivers/crypto/cnxk/cn10k_cryptodev_ops.h | 8 ++ drivers/crypto/cnxk/cn10k_ipsec.c | 4 + drivers/crypto/cnxk/cn10k_ipsec.h | 2 + drivers/crypto/cnxk/cnxk_cryptodev.c | 3 + drivers/crypto/cnxk/cnxk_cryptodev.h | 3 + drivers/crypto/cnxk/cnxk_cryptodev_devargs.c | 31 +++++ drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 27 +++- drivers/crypto/cnxk/version.map | 3 + 19 files changed, 249 insertions(+), 15 deletions(-) diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst index fbe67475be..8dc745dccd 100644 --- a/doc/guides/cryptodevs/cnxk.rst +++ b/doc/guides/cryptodevs/cnxk.rst @@ -187,6 +187,18 @@ Runtime Config Options With the above configuration, the number of maximum queue pairs supported by the device is limited to 4. +- ``LF ID for RX injection in case of fallback mechanism`` (default ``60``) + + LF ID for RX Injection in fallback mechanism of security. + Can be configured during runtime by using ``rx_inj_lf`` ``devargs`` parameter. + + For example:: + + -a 0002:20:00.1,rx_inj_lf=20 + + With the above configuration, LF 20 will be used by the device for RX Injection + in security in fallback mechanism secnario. + Debugging Options ----------------- diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst index e9c9717706..fa30b46ead 100644 --- a/doc/guides/rel_notes/release_24_03.rst +++ b/doc/guides/rel_notes/release_24_03.rst @@ -55,6 +55,9 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Updated Marvell cnxk crypto driver.** + + * Added support for Rx inject in crypto_cn10k. Removed Items ------------- diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h index cf9046bbfb..edab8a5d83 100644 --- a/drivers/common/cnxk/hw/cpt.h +++ b/drivers/common/cnxk/hw/cpt.h @@ -237,6 +237,15 @@ struct cpt_inst_s { uint64_t doneint : 1; uint64_t nixtx_addr : 60; } s; + struct { + uint64_t nixtxl : 3; + uint64_t doneint : 1; + uint64_t chan : 12; + uint64_t l2_len : 8; + uint64_t et_offset : 8; + uint64_t match_id : 16; + uint64_t sso_pf_func : 16; + } hw_s; uint64_t u64; } w0; diff --git a/drivers/common/cnxk/roc_cpt.c b/drivers/common/cnxk/roc_cpt.c index 4e23d8c135..38e46d65c1 100644 --- a/drivers/common/cnxk/roc_cpt.c +++ b/drivers/common/cnxk/roc_cpt.c @@ -463,7 +463,7 @@ cpt_available_lfs_get(struct dev *dev, uint16_t *nb_lf) int cpt_lfs_alloc(struct dev *dev, uint8_t eng_grpmsk, uint8_t blkaddr, bool inl_dev_sso, - bool ctx_ilen_valid, uint8_t ctx_ilen) + bool ctx_ilen_valid, uint8_t ctx_ilen, bool rxc_ena, uint16_t rx_inj_lf) { struct cpt_lf_alloc_req_msg *req; struct mbox *mbox = mbox_get(dev->mbox); @@ -489,6 +489,10 @@ cpt_lfs_alloc(struct dev *dev, uint8_t eng_grpmsk, uint8_t blkaddr, bool inl_dev req->blkaddr = blkaddr; req->ctx_ilen_valid = ctx_ilen_valid; req->ctx_ilen = ctx_ilen; + if (rxc_ena) { + req->rxc_ena = 1; + req->rxc_ena_lf_id = rx_inj_lf; + } rc = mbox_process(mbox); exit: @@ -586,7 +590,7 @@ cpt_iq_init(struct roc_cpt_lf *lf) } int -roc_cpt_dev_configure(struct roc_cpt *roc_cpt, int nb_lf) +roc_cpt_dev_configure(struct roc_cpt *roc_cpt, int nb_lf, bool rxc_ena, uint16_t rx_inj_lf) { struct cpt *cpt = roc_cpt_to_cpt_priv(roc_cpt); uint8_t blkaddr[ROC_CPT_MAX_BLKS]; @@ -630,7 +634,8 @@ roc_cpt_dev_configure(struct roc_cpt *roc_cpt, int nb_lf) ctx_ilen = (PLT_ALIGN(ROC_OT_IPSEC_SA_SZ_MAX, ROC_ALIGN) / 128) - 1; } - rc = cpt_lfs_alloc(&cpt->dev, eng_grpmsk, blkaddr[blknum], false, ctx_ilen_valid, ctx_ilen); + rc = cpt_lfs_alloc(&cpt->dev, eng_grpmsk, blkaddr[blknum], false, ctx_ilen_valid, ctx_ilen, + rxc_ena, rx_inj_lf); if (rc) goto lfs_detach; diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h index 787bccb27d..001e71c55e 100644 --- a/drivers/common/cnxk/roc_cpt.h +++ b/drivers/common/cnxk/roc_cpt.h @@ -171,7 +171,8 @@ int __roc_api roc_cpt_dev_init(struct roc_cpt *roc_cpt); int __roc_api roc_cpt_dev_fini(struct roc_cpt *roc_cpt); int __roc_api roc_cpt_eng_grp_add(struct roc_cpt *roc_cpt, enum cpt_eng_type eng_type); -int __roc_api roc_cpt_dev_configure(struct roc_cpt *roc_cpt, int nb_lf); +int __roc_api roc_cpt_dev_configure(struct roc_cpt *roc_cpt, int nb_lf, bool rxc_ena, + uint16_t rx_inj_lf); void __roc_api roc_cpt_dev_clear(struct roc_cpt *roc_cpt); int __roc_api roc_cpt_lf_init(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf); void __roc_api roc_cpt_lf_fini(struct roc_cpt_lf *lf); diff --git a/drivers/common/cnxk/roc_cpt_priv.h b/drivers/common/cnxk/roc_cpt_priv.h index 4ed87c857b..fa4986d671 100644 --- a/drivers/common/cnxk/roc_cpt_priv.h +++ b/drivers/common/cnxk/roc_cpt_priv.h @@ -22,7 +22,7 @@ int cpt_lfs_attach(struct dev *dev, uint8_t blkaddr, bool modify, uint16_t nb_lf); int cpt_lfs_detach(struct dev *dev); int cpt_lfs_alloc(struct dev *dev, uint8_t eng_grpmsk, uint8_t blk, bool inl_dev_sso, - bool ctx_ilen_valid, uint8_t ctx_ilen); + bool ctx_ilen_valid, uint8_t ctx_ilen, bool rxc_ena, uint16_t rx_inj_lf); int cpt_lfs_free(struct dev *dev); int cpt_lf_init(struct roc_cpt_lf *lf); void cpt_lf_fini(struct roc_cpt_lf *lf); diff --git a/drivers/common/cnxk/roc_ie_ot.c b/drivers/common/cnxk/roc_ie_ot.c index d0b7ad38f1..465b2bc1fb 100644 --- a/drivers/common/cnxk/roc_ie_ot.c +++ b/drivers/common/cnxk/roc_ie_ot.c @@ -12,13 +12,13 @@ roc_ot_ipsec_inb_sa_init(struct roc_ot_ipsec_inb_sa *sa, bool is_inline) memset(sa, 0, sizeof(struct roc_ot_ipsec_inb_sa)); - if (is_inline) { - sa->w0.s.pkt_output = ROC_IE_OT_SA_PKT_OUTPUT_NO_FRAG; - sa->w0.s.pkt_format = ROC_IE_OT_SA_PKT_FMT_META; - sa->w0.s.pkind = ROC_IE_OT_CPT_PKIND; - sa->w0.s.et_ovrwr = 1; - sa->w2.s.l3hdr_on_err = 1; - } + sa->w0.s.pkt_output = ROC_IE_OT_SA_PKT_OUTPUT_NO_FRAG; + sa->w0.s.pkt_format = ROC_IE_OT_SA_PKT_FMT_META; + sa->w0.s.pkind = ROC_IE_OT_CPT_PKIND; + sa->w0.s.et_ovrwr = 1; + sa->w2.s.l3hdr_on_err = 1; + + PLT_SET_USED(is_inline); offset = offsetof(struct roc_ot_ipsec_inb_sa, ctx); sa->w0.s.hw_ctx_off = offset / ROC_CTX_UNIT_8B; diff --git a/drivers/common/cnxk/roc_mbox.h b/drivers/common/cnxk/roc_mbox.h index 05434aec5a..0ad8b738c6 100644 --- a/drivers/common/cnxk/roc_mbox.h +++ b/drivers/common/cnxk/roc_mbox.h @@ -2022,6 +2022,8 @@ struct cpt_lf_alloc_req_msg { uint8_t __io blkaddr; uint8_t __io ctx_ilen_valid : 1; uint8_t __io ctx_ilen : 7; + uint8_t __io rxc_ena : 1; + uint8_t __io rxc_ena_lf_id : 7; }; #define CPT_INLINE_INBOUND 0 diff --git a/drivers/common/cnxk/roc_nix_inl.c b/drivers/common/cnxk/roc_nix_inl.c index 750fd08355..07a90133ca 100644 --- a/drivers/common/cnxk/roc_nix_inl.c +++ b/drivers/common/cnxk/roc_nix_inl.c @@ -986,7 +986,7 @@ roc_nix_inl_outb_init(struct roc_nix *roc_nix) 1ULL << ROC_CPT_DFLT_ENG_GRP_SE_IE | 1ULL << ROC_CPT_DFLT_ENG_GRP_AE); rc = cpt_lfs_alloc(dev, eng_grpmask, blkaddr, - !roc_nix->ipsec_out_sso_pffunc, ctx_ilen_valid, ctx_ilen); + !roc_nix->ipsec_out_sso_pffunc, ctx_ilen_valid, ctx_ilen, false, 0); if (rc) { plt_err("Failed to alloc CPT LF resources, rc=%d", rc); goto lf_detach; diff --git a/drivers/common/cnxk/roc_nix_inl_dev.c b/drivers/common/cnxk/roc_nix_inl_dev.c index dc1306c093..f6991de051 100644 --- a/drivers/common/cnxk/roc_nix_inl_dev.c +++ b/drivers/common/cnxk/roc_nix_inl_dev.c @@ -194,7 +194,7 @@ nix_inl_cpt_setup(struct nix_inl_dev *inl_dev, bool inl_dev_sso) } rc = cpt_lfs_alloc(dev, eng_grpmask, RVU_BLOCK_ADDR_CPT0, inl_dev_sso, ctx_ilen_valid, - ctx_ilen); + ctx_ilen, false, 0); if (rc) { plt_err("Failed to alloc CPT LF resources, rc=%d", rc); return rc; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index bef7b75810..c86d47239b 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -7,6 +7,8 @@ #include #include +#include + #include "roc_cpt.h" #if defined(__aarch64__) #include "roc_io.h" @@ -1057,6 +1059,103 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return i; } +uint16_t __rte_hot +cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, + struct rte_security_session **sess, uint16_t nb_pkts) +{ + uint16_t l2_len, pf_func, lmt_id, count = 0; + uint64_t lmt_base, lmt_arg, io_addr; + struct cn10k_sec_session *sec_sess; + struct rte_cryptodev *cdev = dev; + union cpt_res_s *hw_res = NULL; + struct cpt_inst_s *inst; + struct cnxk_cpt_vf *vf; + struct rte_mbuf *m; + uint64_t dptr; + int i; + + const union cpt_res_s res = { + .cn10k.compcode = CPT_COMP_NOT_DONE, + }; + + vf = cdev->data->dev_private; + + lmt_base = vf->rx_inj_lmtline.lmt_base; + io_addr = vf->rx_inj_lmtline.io_addr; + + ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); + pf_func = vf->rx_inj_pf_func; + +again: + inst = (struct cpt_inst_s *)lmt_base; + for (i = 0; i < RTE_MIN(PKTS_PER_LOOP, nb_pkts); i++) { + + m = pkts[i]; + sec_sess = (struct cn10k_sec_session *)sess[i]; + + if (unlikely(rte_pktmbuf_headroom(m) < 32)) { + plt_dp_err("No space for CPT res_s"); + break; + } + + if (unlikely(!rte_pktmbuf_is_contiguous(m))) { + plt_dp_err("Multi seg is not supported"); + break; + } + + l2_len = m->l2_len; + + *rte_security_dynfield(m) = (uint64_t)sec_sess->userdata; + + hw_res = rte_pktmbuf_mtod(m, void *); + hw_res = RTE_PTR_SUB(hw_res, 32); + hw_res = RTE_PTR_ALIGN_CEIL(hw_res, 16); + + /* Prepare CPT instruction */ + inst->w0.u64 = 0; + inst->w2.u64 = 0; + inst->w2.s.rvu_pf_func = pf_func; + inst->w3.u64 = (((uint64_t)m + sizeof(struct rte_mbuf)) >> 3) << 3 | 1; + + inst->w4.u64 = sec_sess->inst.w4 | (rte_pktmbuf_pkt_len(m)); + dptr = (uint64_t)rte_pktmbuf_iova(m); + inst->dptr = dptr; + inst->rptr = dptr; + + inst->w0.hw_s.l2_len = l2_len; + inst->w0.hw_s.et_offset = l2_len - 2; + + inst->res_addr = (uint64_t)hw_res; + rte_atomic_store_explicit(&hw_res->u64[0], res.u64[0], rte_memory_order_relaxed); + + inst->w7.u64 = sec_sess->inst.w7; + + inst += 2; + } + + if (i > PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (PKTS_PER_STEORL - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, io_addr); + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + PKTS_PER_STEORL); + roc_lmt_submit_steorl(lmt_arg, io_addr); + } else { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, io_addr); + } + + rte_io_wmb(); + + if (nb_pkts - i > 0 && i == PKTS_PER_LOOP) { + nb_pkts -= i; + pkts += i; + count += i; + goto again; + } + + return count + i; +} + void cn10k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev, struct cnxk_cpt_vf *vf) { @@ -1535,6 +1634,30 @@ cn10k_sym_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, return 0; } +int +cn10k_cryptodev_sec_rx_inject_configure(void *device, uint16_t port_id, bool enable) +{ + struct rte_cryptodev *crypto_dev = device; + struct rte_eth_dev *eth_dev; + int ret; + + if (!rte_eth_dev_is_valid_port(port_id)) + return -EINVAL; + + if (!(crypto_dev->feature_flags & RTE_CRYPTODEV_FF_SECURITY_RX_INJECT)) + return -ENOTSUP; + + eth_dev = &rte_eth_devices[port_id]; + + ret = strncmp(eth_dev->device->driver->name, "net_cn10k", 8); + if (ret) + return -ENOTSUP; + + RTE_SET_USED(enable); + + return 0; +} + struct rte_cryptodev_ops cn10k_cpt_ops = { /* Device control ops */ .dev_configure = cnxk_cpt_dev_config, diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h index befbfcdfad..34becede3c 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h @@ -16,6 +16,14 @@ extern struct rte_cryptodev_ops cn10k_cpt_ops; void cn10k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev, struct cnxk_cpt_vf *vf); +__rte_internal +uint16_t __rte_hot cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, + struct rte_security_session **sess, + uint16_t nb_pkts); + +__rte_internal +int cn10k_cryptodev_sec_rx_inject_configure(void *device, uint16_t port_id, bool enable); + __rte_internal uint16_t __rte_hot cn10k_cpt_sg_ver1_crypto_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_events); diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c index ffd3f50eed..2d098fdd24 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.c +++ b/drivers/crypto/cnxk/cn10k_ipsec.c @@ -10,6 +10,7 @@ #include #include +#include "cn10k_cryptodev_ops.h" #include "cn10k_ipsec.h" #include "cnxk_cryptodev.h" #include "cnxk_cryptodev_ops.h" @@ -297,6 +298,7 @@ cn10k_sec_session_create(void *device, struct rte_security_session_conf *conf, if (conf->protocol != RTE_SECURITY_PROTOCOL_IPSEC) return -ENOTSUP; + ((struct cn10k_sec_session *)sess)->userdata = conf->userdata; return cn10k_ipsec_session_create(device, &conf->ipsec, conf->crypto_xform, sess); } @@ -458,4 +460,6 @@ cn10k_sec_ops_override(void) cnxk_sec_ops.session_get_size = cn10k_sec_session_get_size; cnxk_sec_ops.session_stats_get = cn10k_sec_session_stats_get; cnxk_sec_ops.session_update = cn10k_sec_session_update; + cnxk_sec_ops.inb_pkt_rx_inject = cn10k_cryptodev_sec_inb_rx_inject; + cnxk_sec_ops.rx_inject_configure = cn10k_cryptodev_sec_rx_inject_configure; } diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h index 8a93d74062..03ac994001 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.h +++ b/drivers/crypto/cnxk/cn10k_ipsec.h @@ -38,6 +38,8 @@ struct cn10k_sec_session { bool is_outbound; /** Queue pair */ struct cnxk_cpt_qp *qp; + /** Userdata to be set for Rx inject */ + void *userdata; /** * End of SW mutable area diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.c b/drivers/crypto/cnxk/cnxk_cryptodev.c index 4819a14184..b1684e56a7 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev.c @@ -24,6 +24,9 @@ cnxk_cpt_default_ff_get(void) if (roc_model_is_cn10k()) ff |= RTE_CRYPTODEV_FF_SECURITY_INNER_CSUM | RTE_CRYPTODEV_FF_SYM_RAW_DP; + if (roc_model_is_cn10ka_b0()) + ff |= RTE_CRYPTODEV_FF_SECURITY_RX_INJECT; + return ff; } diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index f5374131bf..fedae53736 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -18,6 +18,8 @@ * Device private data */ struct cnxk_cpt_vf { + struct roc_cpt_lmtline rx_inj_lmtline; + uint16_t rx_inj_pf_func; struct roc_cpt cpt; struct rte_cryptodev_capabilities crypto_caps[CNXK_CPT_MAX_CAPS]; struct rte_cryptodev_capabilities @@ -26,6 +28,7 @@ struct cnxk_cpt_vf { uint64_t cnxk_fpm_iova[ROC_AE_EC_ID_PMAX]; struct roc_ae_ec_group *ec_grp[ROC_AE_EC_ID_PMAX]; uint16_t max_qps_limit; + uint16_t rx_inj_lf; }; uint64_t cnxk_cpt_default_ff_get(void); diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_devargs.c b/drivers/crypto/cnxk/cnxk_cryptodev_devargs.c index c3e9bdb2d1..f5a76d83ed 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_devargs.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_devargs.c @@ -9,6 +9,23 @@ #define CNXK_MAX_QPS_LIMIT "max_qps_limit" #define CNXK_MAX_QPS_LIMIT_MIN 1 #define CNXK_MAX_QPS_LIMIT_MAX (ROC_CPT_MAX_LFS - 1) +#define CNXK_RX_INJ_LF "rx_inj_lf" + +static int +parse_rx_inj_lf(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + if (val < CNXK_MAX_QPS_LIMIT_MIN || val > CNXK_MAX_QPS_LIMIT_MAX) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} static int parse_max_qps_limit(const char *key, const char *value, void *extra_args) @@ -31,8 +48,12 @@ cnxk_cpt_parse_devargs(struct rte_devargs *devargs, struct cnxk_cpt_vf *vf) { uint16_t max_qps_limit = CNXK_MAX_QPS_LIMIT_MAX; struct rte_kvargs *kvlist; + uint16_t rx_inj_lf; int rc; + /* Set to max value as default so that the feature is disabled by default. */ + rx_inj_lf = CNXK_MAX_QPS_LIMIT_MAX; + if (devargs == NULL) goto null_devargs; @@ -48,10 +69,20 @@ cnxk_cpt_parse_devargs(struct rte_devargs *devargs, struct cnxk_cpt_vf *vf) rte_kvargs_free(kvlist); goto exit; } + + rc = rte_kvargs_process(kvlist, CNXK_RX_INJ_LF, parse_rx_inj_lf, &rx_inj_lf); + if (rc < 0) { + plt_err("rx_inj_lf should in the range <%d-%d>", CNXK_MAX_QPS_LIMIT_MIN, + max_qps_limit - 1); + rte_kvargs_free(kvlist); + goto exit; + } + rte_kvargs_free(kvlist); null_devargs: vf->max_qps_limit = max_qps_limit; + vf->rx_inj_lf = rx_inj_lf; return 0; exit: diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index 82938c77c8..c0733ddbfb 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "roc_ae_fpm_tables.h" #include "roc_cpt.h" @@ -95,6 +96,7 @@ cnxk_cpt_dev_config(struct rte_cryptodev *dev, struct rte_cryptodev_config *conf struct cnxk_cpt_vf *vf = dev->data->dev_private; struct roc_cpt *roc_cpt = &vf->cpt; uint16_t nb_lf_avail, nb_lf; + bool rxc_ena = false; int ret; /* If this is a reconfigure attempt, clear the device and configure again */ @@ -111,7 +113,13 @@ cnxk_cpt_dev_config(struct rte_cryptodev *dev, struct rte_cryptodev_config *conf if (nb_lf > nb_lf_avail) return -ENOTSUP; - ret = roc_cpt_dev_configure(roc_cpt, nb_lf); + if (dev->feature_flags & RTE_CRYPTODEV_FF_SECURITY_RX_INJECT) { + if (rte_security_dynfield_register() < 0) + return -ENOTSUP; + rxc_ena = true; + } + + ret = roc_cpt_dev_configure(roc_cpt, nb_lf, rxc_ena, vf->rx_inj_lf); if (ret) { plt_err("Could not configure device"); return ret; @@ -208,6 +216,10 @@ cnxk_cpt_dev_info_get(struct rte_cryptodev *dev, info->sym.max_nb_sessions = 0; info->min_mbuf_headroom_req = CNXK_CPT_MIN_HEADROOM_REQ; info->min_mbuf_tailroom_req = CNXK_CPT_MIN_TAILROOM_REQ; + + /* If the LF ID for RX Inject is less than the available lfs. */ + if (vf->rx_inj_lf > info->max_nb_queue_pairs) + info->feature_flags &= ~RTE_CRYPTODEV_FF_SECURITY_RX_INJECT; } static void @@ -452,6 +464,19 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, qp->sess_mp = conf->mp_session; dev->data->queue_pairs[qp_id] = qp; + if (qp_id == vf->rx_inj_lf) { + ret = roc_cpt_lmtline_init(roc_cpt, &vf->rx_inj_lmtline, vf->rx_inj_lf); + if (ret) { + plt_err("Could not init lmtline Rx inject"); + goto exit; + } + + vf->rx_inj_pf_func = qp->lf.pf_func; + + /* Block the queue for other submissions */ + qp->pend_q.pq_mask = 0; + } + return 0; exit: diff --git a/drivers/crypto/cnxk/version.map b/drivers/crypto/cnxk/version.map index d13209feec..5789a6bfc9 100644 --- a/drivers/crypto/cnxk/version.map +++ b/drivers/crypto/cnxk/version.map @@ -8,5 +8,8 @@ INTERNAL { cn10k_cpt_crypto_adapter_dequeue; cn10k_cpt_crypto_adapter_vector_dequeue; + cn10k_cryptodev_sec_inb_rx_inject; + cn10k_cryptodev_sec_rx_inject_configure; + local: *; }; From patchwork Thu Dec 21 12:35:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135438 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5FD1E43750; Thu, 21 Dec 2023 13:36:49 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8CCEC42E88; Thu, 21 Dec 2023 13:36:14 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id DCA5642E7C for ; Thu, 21 Dec 2023 13:36:12 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVNkX019126 for ; Thu, 21 Dec 2023 04:36:12 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=Wv2/vcWtLrKPO++fLhkLGugM6aZYNlkG1IpKk8ZHB5E=; b=MDf 5jUlTcARacZhujW4KiBGbYeKPSm7tLYLzy643fOg/pVSGcCAlejt/uVXkv10S35C ONJI05/pmvfG/20NvryKVNnZZxLwZhbfrCoMHzWFmYVHdLH20SKN1N8CqctvPUkr qBWxdsdFLvwkDV1ulcuOmV8Hal2t3wEBNDtCUtqBA65x7HPRu9Fz5+BAu6H5uN1y 1Ay/2Bs+5ik1BV5s37eE57O9DNseNGAN5gCp0EAT154fnga4bzsvNdm24bMgHELO 0DbHuniMCuhnO4B/IbZMiG2GPYnI4pRJ7fDSTyQwSbTGMzIWLTjw/CP+URfYlSx6 D4YpMos2R22PpjBmU7g== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0h7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:11 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:10 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:10 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 52D373F7079; Thu, 21 Dec 2023 04:36:07 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Rahul Bhansali , Jerin Jacob , Vidya Sagar Velumuri , Tejasree Kondoj , Subject: [PATCH 08/24] common/cnxk: add Rx inject configs Date: Thu, 21 Dec 2023 18:05:29 +0530 Message-ID: <20231221123545.510-9-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: THGtzYJ_apEWS6uFHkz6RhAv2JdU7kYT X-Proofpoint-ORIG-GUID: THGtzYJ_apEWS6uFHkz6RhAv2JdU7kYT X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Rahul Bhansali Add Rx inject config for feature enable/disable, and store Rx chan value per port. Signed-off-by: Rahul Bhansali --- drivers/common/cnxk/roc_idev.c | 44 +++++++++++++++++++++++++++++ drivers/common/cnxk/roc_idev.h | 5 ++++ drivers/common/cnxk/roc_idev_priv.h | 6 ++++ drivers/common/cnxk/roc_nix.c | 2 ++ drivers/common/cnxk/version.map | 4 +++ 5 files changed, 61 insertions(+) diff --git a/drivers/common/cnxk/roc_idev.c b/drivers/common/cnxk/roc_idev.c index e6c6b34d78..48df3518b0 100644 --- a/drivers/common/cnxk/roc_idev.c +++ b/drivers/common/cnxk/roc_idev.c @@ -310,3 +310,47 @@ roc_idev_nix_inl_meta_aura_get(void) return idev->inl_cfg.meta_aura; return 0; } + +uint8_t +roc_idev_nix_rx_inject_get(uint16_t port) +{ + struct idev_cfg *idev; + + idev = idev_get_cfg(); + if (idev != NULL && port < PLT_MAX_ETHPORTS) + return idev->inl_rx_inj_cfg.rx_inject_en[port]; + + return 0; +} + +void +roc_idev_nix_rx_inject_set(uint16_t port, uint8_t enable) +{ + struct idev_cfg *idev; + + idev = idev_get_cfg(); + if (idev != NULL && port < PLT_MAX_ETHPORTS) + __atomic_store_n(&idev->inl_rx_inj_cfg.rx_inject_en[port], enable, + __ATOMIC_RELEASE); +} + +uint16_t * +roc_idev_nix_rx_chan_base_get(void) +{ + struct idev_cfg *idev = idev_get_cfg(); + + if (idev != NULL) + return (uint16_t *)&idev->inl_rx_inj_cfg.chan; + + return NULL; +} + +void +roc_idev_nix_rx_chan_set(uint16_t port, uint16_t chan) +{ + struct idev_cfg *idev; + + idev = idev_get_cfg(); + if (idev != NULL && port < PLT_MAX_ETHPORTS) + __atomic_store_n(&idev->inl_rx_inj_cfg.chan[port], chan, __ATOMIC_RELEASE); +} diff --git a/drivers/common/cnxk/roc_idev.h b/drivers/common/cnxk/roc_idev.h index aea7f5279d..00664eaed6 100644 --- a/drivers/common/cnxk/roc_idev.h +++ b/drivers/common/cnxk/roc_idev.h @@ -22,4 +22,9 @@ struct roc_nix_list *__roc_api roc_idev_nix_list_get(void); struct roc_mcs *__roc_api roc_idev_mcs_get(uint8_t mcs_idx); void __roc_api roc_idev_mcs_set(struct roc_mcs *mcs); void __roc_api roc_idev_mcs_free(struct roc_mcs *mcs); + +uint8_t __roc_api roc_idev_nix_rx_inject_get(uint16_t port); +void __roc_api roc_idev_nix_rx_inject_set(uint16_t port, uint8_t enable); +uint16_t *__roc_api roc_idev_nix_rx_chan_base_get(void); +void __roc_api roc_idev_nix_rx_chan_set(uint16_t port, uint16_t chan); #endif /* _ROC_IDEV_H_ */ diff --git a/drivers/common/cnxk/roc_idev_priv.h b/drivers/common/cnxk/roc_idev_priv.h index 80f8465e1c..8dc1cb25bf 100644 --- a/drivers/common/cnxk/roc_idev_priv.h +++ b/drivers/common/cnxk/roc_idev_priv.h @@ -19,6 +19,11 @@ struct idev_nix_inl_cfg { uint32_t refs; }; +struct idev_nix_inl_rx_inj_cfg { + uint16_t chan[PLT_MAX_ETHPORTS]; + uint8_t rx_inject_en[PLT_MAX_ETHPORTS]; +}; + struct idev_cfg { uint16_t sso_pf_func; uint16_t npa_pf_func; @@ -35,6 +40,7 @@ struct idev_cfg { struct nix_inl_dev *nix_inl_dev; struct idev_nix_inl_cfg inl_cfg; struct roc_nix_list roc_nix_list; + struct idev_nix_inl_rx_inj_cfg inl_rx_inj_cfg; plt_spinlock_t nix_inl_dev_lock; plt_spinlock_t npa_dev_lock; }; diff --git a/drivers/common/cnxk/roc_nix.c b/drivers/common/cnxk/roc_nix.c index f64933a1d9..97c0ae3e25 100644 --- a/drivers/common/cnxk/roc_nix.c +++ b/drivers/common/cnxk/roc_nix.c @@ -223,6 +223,8 @@ roc_nix_lf_alloc(struct roc_nix *roc_nix, uint32_t nb_rxq, uint32_t nb_txq, nix->nb_rx_queues = nb_rxq; nix->nb_tx_queues = nb_txq; + roc_idev_nix_rx_chan_set(roc_nix->port_id, rsp->rx_chan_base); + nix->rqs = plt_zmalloc(sizeof(struct roc_nix_rq *) * nb_rxq, 0); if (!nix->rqs) { rc = -ENOMEM; diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index aa884a8fe2..f84382c401 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -105,6 +105,10 @@ INTERNAL { roc_idev_num_lmtlines_get; roc_idev_nix_inl_meta_aura_get; roc_idev_nix_list_get; + roc_idev_nix_rx_chan_base_get; + roc_idev_nix_rx_chan_set; + roc_idev_nix_rx_inject_get; + roc_idev_nix_rx_inject_set; roc_ml_reg_read64; roc_ml_reg_write64; roc_ml_reg_read32; From patchwork Thu Dec 21 12:35:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135439 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4D6AE43750; Thu, 21 Dec 2023 13:36:56 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A877142E81; Thu, 21 Dec 2023 13:36:17 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id BE24842E83 for ; Thu, 21 Dec 2023 13:36:15 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVelT019408 for ; Thu, 21 Dec 2023 04:36:15 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=9Rk4cx/n4UxxjLqxOu8e/V5MbjxtGbyGtXm6HyefoNY=; b=cwp tPjI27VHzduKJBygf1BuGHyTQwebLFaAE0OX0qTIeRrWvaBzS52TxB9e5JndDPva UYFCaVns2l5uvtmjawxQ8xtzSKETQMgL4vag+qVpY4OAIgngD1dh8Lg/9e2cfJdZ art8cI7PwRluDX35B27wfMOnTobI2tJLX8+nqvsYV4c9RknwVd36BFOpW2z0leRJ zn0Hvez/7nEU6De9REvGOEkVeFhY5RhONu8LcEkQLOrtC64FOjOYKZm5NLLwYraN dawGXxClekDGfDU8LSnKq5UkRM4XOQDFNOlE/46lRLVl7MwQcjdY9e0ZhttCRAI+ xbMQ3XccZZd6tPNjgQw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0he-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:14 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:13 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:13 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 450043F7073; Thu, 21 Dec 2023 04:36:09 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Rahul Bhansali , Jerin Jacob , Vidya Sagar Velumuri , Tejasree Kondoj , Subject: [PATCH 09/24] crypto/cnxk: Rx inject config update Date: Thu, 21 Dec 2023 18:05:30 +0530 Message-ID: <20231221123545.510-10-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: CBBjvpDTHqrKFfWpI4H94_jclRoly00Y X-Proofpoint-ORIG-GUID: CBBjvpDTHqrKFfWpI4H94_jclRoly00Y X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Rahul Bhansali - Update chan in CPT inst from port's Rx chan - Set Rx inject config in Idev struct Signed-off-by: Rahul Bhansali --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 4 +++- drivers/crypto/cnxk/cn10k_ipsec.c | 3 +++ drivers/crypto/cnxk/cnxk_cryptodev.h | 1 + drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 2 ++ 4 files changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index c86d47239b..53a33aab49 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -15,6 +15,7 @@ #else #include "roc_io_generic.h" #endif +#include "roc_idev.h" #include "roc_sso.h" #include "roc_sso_dp.h" @@ -1122,6 +1123,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, inst->dptr = dptr; inst->rptr = dptr; + inst->w0.hw_s.chan = *(vf->rx_chan_base + m->port); inst->w0.hw_s.l2_len = l2_len; inst->w0.hw_s.et_offset = l2_len - 2; @@ -1653,7 +1655,7 @@ cn10k_cryptodev_sec_rx_inject_configure(void *device, uint16_t port_id, bool ena if (ret) return -ENOTSUP; - RTE_SET_USED(enable); + roc_idev_nix_rx_inject_set(port_id, enable); return 0; } diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c index 2d098fdd24..d08a1067ca 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.c +++ b/drivers/crypto/cnxk/cn10k_ipsec.c @@ -192,6 +192,9 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, sec_sess->is_outbound = false; sec_sess->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, in_sa); + /* Save index/SPI in cookie, specific required for Rx Inject */ + sa_dptr->w1.s.cookie = 0xFFFFFFFF; + /* pre-populate CPT INST word 4 */ inst_w4.u64 = 0; inst_w4.s.opcode_major = ROC_IE_OT_MAJOR_OP_PROCESS_INBOUND_IPSEC | ROC_IE_OT_INPLACE_BIT; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index fedae53736..2ae81d2f90 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -20,6 +20,7 @@ struct cnxk_cpt_vf { struct roc_cpt_lmtline rx_inj_lmtline; uint16_t rx_inj_pf_func; + uint16_t *rx_chan_base; struct roc_cpt cpt; struct rte_cryptodev_capabilities crypto_caps[CNXK_CPT_MAX_CAPS]; struct rte_cryptodev_capabilities diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index c0733ddbfb..fd44155955 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -10,6 +10,7 @@ #include "roc_ae_fpm_tables.h" #include "roc_cpt.h" #include "roc_errata.h" +#include "roc_idev.h" #include "roc_ie_on.h" #include "cnxk_ae.h" @@ -117,6 +118,7 @@ cnxk_cpt_dev_config(struct rte_cryptodev *dev, struct rte_cryptodev_config *conf if (rte_security_dynfield_register() < 0) return -ENOTSUP; rxc_ena = true; + vf->rx_chan_base = roc_idev_nix_rx_chan_base_get(); } ret = roc_cpt_dev_configure(roc_cpt, nb_lf, rxc_ena, vf->rx_inj_lf); From patchwork Thu Dec 21 12:35:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135440 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 44B8243750; Thu, 21 Dec 2023 13:37:03 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B61AB42E8F; Thu, 21 Dec 2023 13:36:19 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 6F50E42E8B for ; Thu, 21 Dec 2023 13:36:18 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVJZC019120 for ; Thu, 21 Dec 2023 04:36:17 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=V+W1ulhrCI5x/cFgjoUuInf9nze6tgb9YRdyTItuzak=; b=IVX NX7PK8gQLzwtaaBA7HdWOrpbwLEF7jpJxQ68chHpGZ3QGOH2brseSq18AjwXQT/l C0Gvu5fM3pLuZeGRNqN15uBZnQS7LtXMVIGLoRWnx+5cqVUFr0vy8tCsxTNHUQ27 97KB+ddJupQ7JiD6ufTExWn9BLp67VdNl8U5+4/rOik5v74kglt/w8gWVri9vITM zFGOOfywVfX8b6HDTN6vQeP2hV0jm6fXWhbcxVwl0uD0Wd4Iwmr7wFphH2id2VnE zSR0uuCZ2yowb2b9a/TgQyij7AJgPo+01g8f5AdFJ2R43ppAu1cgbMACTSKQWM3g jTwE6zmFAvuGJwij6dQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0hk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:17 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:16 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:16 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 44E2E3F7079; Thu, 21 Dec 2023 04:36:12 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 10/24] crypto/cnxk: enable Rx inject for 103 Date: Thu, 21 Dec 2023 18:05:31 +0530 Message-ID: <20231221123545.510-11-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: A1MZug0b848LkN1nJEdx3ibfSCwayHFB X-Proofpoint-ORIG-GUID: A1MZug0b848LkN1nJEdx3ibfSCwayHFB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Enable Rx inject feature for 103XX Signed-off-by: Vidya Sagar Velumuri --- drivers/crypto/cnxk/cnxk_cryptodev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.c b/drivers/crypto/cnxk/cnxk_cryptodev.c index b1684e56a7..1eede2e59c 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev.c @@ -24,7 +24,7 @@ cnxk_cpt_default_ff_get(void) if (roc_model_is_cn10k()) ff |= RTE_CRYPTODEV_FF_SECURITY_INNER_CSUM | RTE_CRYPTODEV_FF_SYM_RAW_DP; - if (roc_model_is_cn10ka_b0()) + if (roc_model_is_cn10ka_b0() || roc_model_is_cn10kb()) ff |= RTE_CRYPTODEV_FF_SECURITY_RX_INJECT; return ff; From patchwork Thu Dec 21 12:35:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135441 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 369B043750; Thu, 21 Dec 2023 13:37:09 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E44CB40A6E; Thu, 21 Dec 2023 13:36:23 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id ED20542E94 for ; Thu, 21 Dec 2023 13:36:21 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVJZE019120 for ; Thu, 21 Dec 2023 04:36:21 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=03D/K55Dos4vyJlmXAgu21lCDHLbR2xQz4K5vNDgvPQ=; b=Wc/ a7XQz3dCpht7ai21ivEKRhWIPC+UWEB77erMVahL6inuWFP+YoYuPjY8/sUqLuyt xpR/oFz72TziiFGaBpJbzI3iKfy4psh2kzIMEgKIsnIFaDZ1a0aAeaNKYnf8IAYI 3f0lrJqI6d+AobUrcv7PSnkhHM1Ppts+OxdtbterdJHv38qLfAYVWTWzDJKnkwUO vG2gasdMiDV8IS35O1Y4EG5QxEOIzBYl0V6PYtbA24W64eXQ6Nvwh5Xy3q7toE5g noWRv0NbqPNkCeE0gviAJTi8JLt6Dkvz0rof75epv3uEcFR62zOt/WUQwe+s4lLf gwewCzltrQj9/3zU1qQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0ht-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:20 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:19 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:19 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id F0B8D3F7092; Thu, 21 Dec 2023 04:36:15 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 11/24] crypto/cnxk: rename security caps as IPsec security caps Date: Thu, 21 Dec 2023 18:05:32 +0530 Message-ID: <20231221123545.510-12-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: hSUQflkVRjQLI9CnFOdit9SagZSUI7pw X-Proofpoint-ORIG-GUID: hSUQflkVRjQLI9CnFOdit9SagZSUI7pw X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Security capabilities would vary between IPsec and other new offloads. Rename existing security caps to indicate that they are IPsec specific ones. Rename and change the scope of common functions, inorder to avoid code duplication. These functions can be used by both IPsec and TLS Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri --- drivers/common/cnxk/cnxk_security.c | 13 ++-- drivers/common/cnxk/cnxk_security.h | 17 +++-- drivers/common/cnxk/version.map | 1 + drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 18 ++++- drivers/crypto/cnxk/cn10k_ipsec.c | 46 +++++++----- drivers/crypto/cnxk/cn10k_ipsec.h | 9 ++- drivers/crypto/cnxk/cn10k_ipsec_la_ops.h | 18 ++--- drivers/crypto/cnxk/cn9k_ipsec_la_ops.h | 8 +- drivers/crypto/cnxk/cnxk_cryptodev.h | 10 +-- .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 73 ++++++++++--------- drivers/crypto/cnxk/cnxk_sg.h | 4 +- 11 files changed, 123 insertions(+), 94 deletions(-) diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c index a8c3ba90cd..81991c4697 100644 --- a/drivers/common/cnxk/cnxk_security.c +++ b/drivers/common/cnxk/cnxk_security.c @@ -8,9 +8,8 @@ #include "roc_api.h" -static void -ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, - uint8_t *hmac_opad_ipad) +void +cnxk_sec_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, uint8_t *hmac_opad_ipad) { const uint8_t *key = auth_xform->auth.key.data; uint32_t length = auth_xform->auth.key.length; @@ -192,7 +191,7 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, const uint8_t *auth_key = auth_xfrm->auth.key.data; roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad); } else { - ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad); + cnxk_sec_opad_ipad_gen(auth_xfrm, hmac_opad_ipad); } tmp_key = (uint64_t *)hmac_opad_ipad; @@ -741,7 +740,7 @@ onf_ipsec_sa_common_param_fill(struct roc_ie_onf_sa_ctl *ctl, uint8_t *salt, key = cipher_xfrm->cipher.key.data; length = cipher_xfrm->cipher.key.length; - ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad); + cnxk_sec_opad_ipad_gen(auth_xfrm, hmac_opad_ipad); } switch (length) { @@ -1374,7 +1373,7 @@ cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipsec, roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad); } else if (auth_xform->auth.algo != RTE_CRYPTO_AUTH_NULL) { - ipsec_hmac_opad_ipad_gen(auth_xform, hmac_opad_ipad); + cnxk_sec_opad_ipad_gen(auth_xform, hmac_opad_ipad); } } @@ -1441,7 +1440,7 @@ cnxk_on_ipsec_inb_sa_create(struct rte_security_ipsec_xform *ipsec, roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad); } else if (auth_xform->auth.algo != RTE_CRYPTO_AUTH_NULL) { - ipsec_hmac_opad_ipad_gen(auth_xform, hmac_opad_ipad); + cnxk_sec_opad_ipad_gen(auth_xform, hmac_opad_ipad); } } diff --git a/drivers/common/cnxk/cnxk_security.h b/drivers/common/cnxk/cnxk_security.h index 2277ce9144..fabf694df4 100644 --- a/drivers/common/cnxk/cnxk_security.h +++ b/drivers/common/cnxk/cnxk_security.h @@ -61,14 +61,15 @@ bool __roc_api cnxk_onf_ipsec_inb_sa_valid(struct roc_onf_ipsec_inb_sa *sa); bool __roc_api cnxk_onf_ipsec_outb_sa_valid(struct roc_onf_ipsec_outb_sa *sa); /* [CN9K] */ -int __roc_api -cnxk_on_ipsec_inb_sa_create(struct rte_security_ipsec_xform *ipsec, - struct rte_crypto_sym_xform *crypto_xform, - struct roc_ie_on_inb_sa *in_sa); +int __roc_api cnxk_on_ipsec_inb_sa_create(struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *crypto_xform, + struct roc_ie_on_inb_sa *in_sa); -int __roc_api -cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipsec, - struct rte_crypto_sym_xform *crypto_xform, - struct roc_ie_on_outb_sa *out_sa); +int __roc_api cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *crypto_xform, + struct roc_ie_on_outb_sa *out_sa); + +__rte_internal +void cnxk_sec_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, uint8_t *hmac_opad_ipad); #endif /* _CNXK_SECURITY_H__ */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index f84382c401..15fd5710d2 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -1,6 +1,7 @@ INTERNAL { global: + cnxk_sec_opad_ipad_gen; cnxk_ipsec_icvlen_get; cnxk_ipsec_ivlen_get; cnxk_ipsec_outb_rlens_get; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 53a33aab49..f105a431f8 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -80,8 +80,9 @@ cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op) } static __rte_always_inline int __rte_hot -cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cn10k_sec_session *sess, - struct cpt_inst_s *inst, struct cpt_inflight_req *infl_req, const bool is_sg_ver2) +cpt_sec_ipsec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, + struct cn10k_sec_session *sess, struct cpt_inst_s *inst, + struct cpt_inflight_req *infl_req, const bool is_sg_ver2) { struct rte_crypto_sym_op *sym_op = op->sym; int ret; @@ -91,7 +92,7 @@ cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cn10k return -ENOTSUP; } - if (sess->is_outbound) + if (sess->ipsec.is_outbound) ret = process_outb_sa(&qp->lf, op, sess, &qp->meta_info, infl_req, inst, is_sg_ver2); else @@ -100,6 +101,17 @@ cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cn10k return ret; } +static __rte_always_inline int __rte_hot +cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cn10k_sec_session *sess, + struct cpt_inst_s *inst, struct cpt_inflight_req *infl_req, const bool is_sg_ver2) +{ + + if (sess->proto == RTE_SECURITY_PROTOCOL_IPSEC) + return cpt_sec_ipsec_inst_fill(qp, op, sess, &inst[0], infl_req, is_sg_ver2); + + return 0; +} + static inline int cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], struct cpt_inst_s inst[], struct cpt_inflight_req *infl_req, const bool is_sg_ver2) diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c index d08a1067ca..a9c673ea83 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.c +++ b/drivers/crypto/cnxk/cn10k_ipsec.c @@ -20,7 +20,7 @@ #include "roc_api.h" static uint64_t -ipsec_cpt_inst_w7_get(struct roc_cpt *roc_cpt, void *sa) +cpt_inst_w7_get(struct roc_cpt *roc_cpt, void *sa) { union cpt_inst_w7 w7; @@ -64,7 +64,7 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, goto sa_dptr_free; } - sec_sess->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, out_sa); + sec_sess->inst.w7 = cpt_inst_w7_get(roc_cpt, out_sa); #ifdef LA_IPSEC_DEBUG /* Use IV from application in debug mode */ @@ -89,7 +89,7 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, } #endif - sec_sess->is_outbound = true; + sec_sess->ipsec.is_outbound = true; /* Get Rlen calculation data */ ret = cnxk_ipsec_outb_rlens_get(&rlens, ipsec_xfrm, crypto_xfrm); @@ -150,6 +150,7 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, /* Trigger CTX flush so that data is written back to DRAM */ roc_cpt_lf_ctx_flush(lf, out_sa, false); + sec_sess->proto = RTE_SECURITY_PROTOCOL_IPSEC; plt_atomic_thread_fence(__ATOMIC_SEQ_CST); sa_dptr_free: @@ -189,8 +190,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, goto sa_dptr_free; } - sec_sess->is_outbound = false; - sec_sess->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, in_sa); + sec_sess->ipsec.is_outbound = false; + sec_sess->inst.w7 = cpt_inst_w7_get(roc_cpt, in_sa); /* Save index/SPI in cookie, specific required for Rx Inject */ sa_dptr->w1.s.cookie = 0xFFFFFFFF; @@ -209,7 +210,7 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, */ if (ipsec_xfrm->options.ip_csum_enable) { param1.s.ip_csum_disable = ROC_IE_OT_SA_INNER_PKT_IP_CSUM_ENABLE; - sec_sess->ip_csum = RTE_MBUF_F_RX_IP_CKSUM_GOOD; + sec_sess->ipsec.ip_csum = RTE_MBUF_F_RX_IP_CKSUM_GOOD; } /* Disable L4 checksum verification by default */ @@ -250,6 +251,7 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, /* Trigger CTX flush so that data is written back to DRAM */ roc_cpt_lf_ctx_flush(lf, in_sa, true); + sec_sess->proto = RTE_SECURITY_PROTOCOL_IPSEC; plt_atomic_thread_fence(__ATOMIC_SEQ_CST); sa_dptr_free: @@ -298,16 +300,15 @@ cn10k_sec_session_create(void *device, struct rte_security_session_conf *conf, if (conf->action_type != RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL) return -EINVAL; - if (conf->protocol != RTE_SECURITY_PROTOCOL_IPSEC) - return -ENOTSUP; - - ((struct cn10k_sec_session *)sess)->userdata = conf->userdata; - return cn10k_ipsec_session_create(device, &conf->ipsec, - conf->crypto_xform, sess); + if (conf->protocol == RTE_SECURITY_PROTOCOL_IPSEC) { + ((struct cn10k_sec_session *)sess)->userdata = conf->userdata; + return cn10k_ipsec_session_create(device, &conf->ipsec, conf->crypto_xform, sess); + } + return -ENOTSUP; } static int -cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess) +cn10k_sec_ipsec_session_destroy(void *dev, struct rte_security_session *sec_sess) { struct rte_cryptodev *crypto_dev = dev; union roc_ot_ipsec_sa_word2 *w2; @@ -318,9 +319,6 @@ cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess) void *sa_dptr = NULL; int ret; - if (unlikely(sec_sess == NULL)) - return -EINVAL; - sess = (struct cn10k_sec_session *)sec_sess; qp = crypto_dev->data->queue_pairs[0]; @@ -336,7 +334,7 @@ cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess) ret = -1; - if (sess->is_outbound) { + if (sess->ipsec.is_outbound) { sa_dptr = plt_zmalloc(sizeof(struct roc_ot_ipsec_outb_sa), 8); if (sa_dptr != NULL) { roc_ot_ipsec_outb_sa_init(sa_dptr); @@ -376,6 +374,18 @@ cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess) return 0; } +static int +cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess) +{ + if (unlikely(sec_sess == NULL)) + return -EINVAL; + + if (((struct cn10k_sec_session *)sec_sess)->proto == RTE_SECURITY_PROTOCOL_IPSEC) + return cn10k_sec_ipsec_session_destroy(dev, sec_sess); + + return -EINVAL; +} + static unsigned int cn10k_sec_session_get_size(void *device __rte_unused) { @@ -405,7 +415,7 @@ cn10k_sec_session_stats_get(void *device, struct rte_security_session *sess, stats->protocol = RTE_SECURITY_PROTOCOL_IPSEC; sa = &priv->sa; - if (priv->is_outbound) { + if (priv->ipsec.is_outbound) { out_sa = &sa->out_sa; roc_cpt_lf_ctx_flush(&qp->lf, out_sa, false); rte_delay_ms(1); diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h index 03ac994001..2b7a3e6acf 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.h +++ b/drivers/crypto/cnxk/cn10k_ipsec.h @@ -29,13 +29,18 @@ struct cn10k_sec_session { /** PMD private space */ + enum rte_security_session_protocol proto; /** Pre-populated CPT inst words */ struct cnxk_cpt_inst_tmpl inst; uint16_t max_extended_len; uint16_t iv_offset; uint8_t iv_length; - uint8_t ip_csum; - bool is_outbound; + union { + struct { + uint8_t ip_csum; + bool is_outbound; + } ipsec; + }; /** Queue pair */ struct cnxk_cpt_qp *qp; /** Userdata to be set for Rx inject */ diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h index 8e208eb2ca..af2c85022e 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h +++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h @@ -121,7 +121,7 @@ process_outb_sa(struct roc_cpt_lf *lf, struct rte_crypto_op *cop, struct cn10k_s i = 0; gather_comp = (struct roc_sglist_comp *)((uint8_t *)m_data + 8); - i = fill_ipsec_sg_comp_from_pkt(gather_comp, i, m_src); + i = fill_sg_comp_from_pkt(gather_comp, i, m_src); ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); @@ -132,7 +132,7 @@ process_outb_sa(struct roc_cpt_lf *lf, struct rte_crypto_op *cop, struct cn10k_s i = 0; scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); - i = fill_ipsec_sg_comp_from_pkt(scatter_comp, i, m_src); + i = fill_sg_comp_from_pkt(scatter_comp, i, m_src); ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); @@ -170,7 +170,7 @@ process_outb_sa(struct roc_cpt_lf *lf, struct rte_crypto_op *cop, struct cn10k_s i = 0; gather_comp = (struct roc_sg2list_comp *)((uint8_t *)m_data); - i = fill_ipsec_sg2_comp_from_pkt(gather_comp, i, m_src); + i = fill_sg2_comp_from_pkt(gather_comp, i, m_src); cpt_inst_w5.s.gather_sz = ((i + 2) / 3); g_size_bytes = ((i + 2) / 3) * sizeof(struct roc_sg2list_comp); @@ -181,7 +181,7 @@ process_outb_sa(struct roc_cpt_lf *lf, struct rte_crypto_op *cop, struct cn10k_s i = 0; scatter_comp = (struct roc_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); - i = fill_ipsec_sg2_comp_from_pkt(scatter_comp, i, m_src); + i = fill_sg2_comp_from_pkt(scatter_comp, i, m_src); cpt_inst_w6.s.scatter_sz = ((i + 2) / 3); @@ -211,7 +211,7 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_sec_session *sess, struct inst->w4.u64 = sess->inst.w4 | rte_pktmbuf_pkt_len(m_src); dptr = rte_pktmbuf_mtod(m_src, uint64_t); inst->dptr = dptr; - m_src->ol_flags |= (uint64_t)sess->ip_csum; + m_src->ol_flags |= (uint64_t)sess->ipsec.ip_csum; } else if (is_sg_ver2 == false) { struct roc_sglist_comp *scatter_comp, *gather_comp; uint32_t g_size_bytes, s_size_bytes; @@ -234,7 +234,7 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_sec_session *sess, struct /* Input Gather List */ i = 0; gather_comp = (struct roc_sglist_comp *)((uint8_t *)m_data + 8); - i = fill_ipsec_sg_comp_from_pkt(gather_comp, i, m_src); + i = fill_sg_comp_from_pkt(gather_comp, i, m_src); ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); @@ -242,7 +242,7 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_sec_session *sess, struct /* Output Scatter List */ i = 0; scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); - i = fill_ipsec_sg_comp_from_pkt(scatter_comp, i, m_src); + i = fill_sg_comp_from_pkt(scatter_comp, i, m_src); ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); @@ -270,7 +270,7 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_sec_session *sess, struct i = 0; gather_comp = (struct roc_sg2list_comp *)((uint8_t *)m_data); - i = fill_ipsec_sg2_comp_from_pkt(gather_comp, i, m_src); + i = fill_sg2_comp_from_pkt(gather_comp, i, m_src); cpt_inst_w5.s.gather_sz = ((i + 2) / 3); g_size_bytes = ((i + 2) / 3) * sizeof(struct roc_sg2list_comp); @@ -278,7 +278,7 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_sec_session *sess, struct /* Output Scatter List */ i = 0; scatter_comp = (struct roc_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); - i = fill_ipsec_sg2_comp_from_pkt(scatter_comp, i, m_src); + i = fill_sg2_comp_from_pkt(scatter_comp, i, m_src); cpt_inst_w6.s.scatter_sz = ((i + 2) / 3); diff --git a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h index 3d0db72775..3e9f1e7efb 100644 --- a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h +++ b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h @@ -132,7 +132,7 @@ process_outb_sa(struct cpt_qp_meta_info *m_info, struct rte_crypto_op *cop, gather_comp = (struct roc_sglist_comp *)((uint8_t *)m_data + 8); i = fill_sg_comp(gather_comp, i, (uint64_t)hdr, hdr_len); - i = fill_ipsec_sg_comp_from_pkt(gather_comp, i, m_src); + i = fill_sg_comp_from_pkt(gather_comp, i, m_src); ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); @@ -146,7 +146,7 @@ process_outb_sa(struct cpt_qp_meta_info *m_info, struct rte_crypto_op *cop, scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); i = fill_sg_comp(scatter_comp, i, (uint64_t)hdr, hdr_len); - i = fill_ipsec_sg_comp_from_pkt(scatter_comp, i, m_src); + i = fill_sg_comp_from_pkt(scatter_comp, i, m_src); ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); @@ -228,7 +228,7 @@ process_inb_sa(struct cpt_qp_meta_info *m_info, struct rte_crypto_op *cop, */ i = 0; gather_comp = (struct roc_sglist_comp *)((uint8_t *)m_data + 8); - i = fill_ipsec_sg_comp_from_pkt(gather_comp, i, m_src); + i = fill_sg_comp_from_pkt(gather_comp, i, m_src); ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); @@ -239,7 +239,7 @@ process_inb_sa(struct cpt_qp_meta_info *m_info, struct rte_crypto_op *cop, i = 0; scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); i = fill_sg_comp(scatter_comp, i, (uint64_t)hdr, hdr_len); - i = fill_ipsec_sg_comp_from_pkt(scatter_comp, i, m_src); + i = fill_sg_comp_from_pkt(scatter_comp, i, m_src); ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index 2ae81d2f90..a5c4365631 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -11,9 +11,10 @@ #include "roc_ae.h" #include "roc_cpt.h" -#define CNXK_CPT_MAX_CAPS 55 -#define CNXK_SEC_CRYPTO_MAX_CAPS 16 -#define CNXK_SEC_MAX_CAPS 9 +#define CNXK_CPT_MAX_CAPS 55 +#define CNXK_SEC_IPSEC_CRYPTO_MAX_CAPS 16 +#define CNXK_SEC_MAX_CAPS 9 + /** * Device private data */ @@ -23,8 +24,7 @@ struct cnxk_cpt_vf { uint16_t *rx_chan_base; struct roc_cpt cpt; struct rte_cryptodev_capabilities crypto_caps[CNXK_CPT_MAX_CAPS]; - struct rte_cryptodev_capabilities - sec_crypto_caps[CNXK_SEC_CRYPTO_MAX_CAPS]; + struct rte_cryptodev_capabilities sec_ipsec_crypto_caps[CNXK_SEC_IPSEC_CRYPTO_MAX_CAPS]; struct rte_security_capability sec_caps[CNXK_SEC_MAX_CAPS]; uint64_t cnxk_fpm_iova[ROC_AE_EC_ID_PMAX]; struct roc_ae_ec_group *ec_grp[ROC_AE_EC_ID_PMAX]; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index 2676b52832..178f510a63 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -20,13 +20,14 @@ RTE_DIM(caps_##name)); \ } while (0) -#define SEC_CAPS_ADD(cnxk_caps, cur_pos, hw_caps, name) \ +#define SEC_IPSEC_CAPS_ADD(cnxk_caps, cur_pos, hw_caps, name) \ do { \ if ((hw_caps[CPT_ENG_TYPE_SE].name) || \ (hw_caps[CPT_ENG_TYPE_IE].name) || \ (hw_caps[CPT_ENG_TYPE_AE].name)) \ - sec_caps_add(cnxk_caps, cur_pos, sec_caps_##name, \ - RTE_DIM(sec_caps_##name)); \ + sec_ipsec_caps_add(cnxk_caps, cur_pos, \ + sec_ipsec_caps_##name, \ + RTE_DIM(sec_ipsec_caps_##name)); \ } while (0) static const struct rte_cryptodev_capabilities caps_mul[] = { @@ -1184,7 +1185,7 @@ static const struct rte_cryptodev_capabilities caps_end[] = { RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() }; -static const struct rte_cryptodev_capabilities sec_caps_aes[] = { +static const struct rte_cryptodev_capabilities sec_ipsec_caps_aes[] = { { /* AES GCM */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, {.sym = { @@ -1332,7 +1333,7 @@ static const struct rte_cryptodev_capabilities sec_caps_aes[] = { }, }; -static const struct rte_cryptodev_capabilities sec_caps_des[] = { +static const struct rte_cryptodev_capabilities sec_ipsec_caps_des[] = { { /* DES */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, {.sym = { @@ -1375,7 +1376,7 @@ static const struct rte_cryptodev_capabilities sec_caps_des[] = { }, }; -static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = { +static const struct rte_cryptodev_capabilities sec_ipsec_caps_sha1_sha2[] = { { /* SHA1 HMAC */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, {.sym = { @@ -1478,7 +1479,7 @@ static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = { }, }; -static const struct rte_cryptodev_capabilities sec_caps_null[] = { +static const struct rte_cryptodev_capabilities sec_ipsec_caps_null[] = { { /* NULL (CIPHER) */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, {.sym = { @@ -1691,29 +1692,28 @@ cnxk_crypto_capabilities_get(struct cnxk_cpt_vf *vf) } static void -sec_caps_limit_check(int *cur_pos, int nb_caps) +sec_ipsec_caps_limit_check(int *cur_pos, int nb_caps) { - PLT_VERIFY(*cur_pos + nb_caps <= CNXK_SEC_CRYPTO_MAX_CAPS); + PLT_VERIFY(*cur_pos + nb_caps <= CNXK_SEC_IPSEC_CRYPTO_MAX_CAPS); } static void -sec_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos, - const struct rte_cryptodev_capabilities *caps, int nb_caps) +sec_ipsec_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos, + const struct rte_cryptodev_capabilities *caps, int nb_caps) { - sec_caps_limit_check(cur_pos, nb_caps); + sec_ipsec_caps_limit_check(cur_pos, nb_caps); memcpy(&cnxk_caps[*cur_pos], caps, nb_caps * sizeof(caps[0])); *cur_pos += nb_caps; } static void -cn10k_sec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[], - int *cur_pos) +cn10k_sec_ipsec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos) { const struct rte_cryptodev_capabilities *cap; unsigned int i; - sec_caps_limit_check(cur_pos, 1); + sec_ipsec_caps_limit_check(cur_pos, 1); /* NULL auth */ for (i = 0; i < RTE_DIM(caps_null); i++) { @@ -1727,7 +1727,7 @@ cn10k_sec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[], } static void -cn9k_sec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[]) +cn9k_sec_ipsec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[]) { struct rte_cryptodev_capabilities *caps; @@ -1747,27 +1747,26 @@ cn9k_sec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[]) } static void -sec_crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[], - union cpt_eng_caps *hw_caps) +sec_ipsec_crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[], + union cpt_eng_caps *hw_caps) { int cur_pos = 0; - SEC_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, aes); - SEC_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, des); - SEC_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, sha1_sha2); + SEC_IPSEC_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, aes); + SEC_IPSEC_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, des); + SEC_IPSEC_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, sha1_sha2); if (roc_model_is_cn10k()) - cn10k_sec_crypto_caps_update(cnxk_caps, &cur_pos); + cn10k_sec_ipsec_crypto_caps_update(cnxk_caps, &cur_pos); else - cn9k_sec_crypto_caps_update(cnxk_caps); + cn9k_sec_ipsec_crypto_caps_update(cnxk_caps); - sec_caps_add(cnxk_caps, &cur_pos, sec_caps_null, - RTE_DIM(sec_caps_null)); - sec_caps_add(cnxk_caps, &cur_pos, caps_end, RTE_DIM(caps_end)); + sec_ipsec_caps_add(cnxk_caps, &cur_pos, sec_ipsec_caps_null, RTE_DIM(sec_ipsec_caps_null)); + sec_ipsec_caps_add(cnxk_caps, &cur_pos, caps_end, RTE_DIM(caps_end)); } static void -cnxk_sec_caps_update(struct rte_security_capability *sec_cap) +cnxk_sec_ipsec_caps_update(struct rte_security_capability *sec_cap) { sec_cap->ipsec.options.udp_encap = 1; sec_cap->ipsec.options.copy_df = 1; @@ -1775,7 +1774,7 @@ cnxk_sec_caps_update(struct rte_security_capability *sec_cap) } static void -cn10k_sec_caps_update(struct rte_security_capability *sec_cap) +cn10k_sec_ipsec_caps_update(struct rte_security_capability *sec_cap) { if (sec_cap->ipsec.direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { #ifdef LA_IPSEC_DEBUG @@ -1797,7 +1796,7 @@ cn10k_sec_caps_update(struct rte_security_capability *sec_cap) } static void -cn9k_sec_caps_update(struct rte_security_capability *sec_cap) +cn9k_sec_ipsec_caps_update(struct rte_security_capability *sec_cap) { if (sec_cap->ipsec.direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { #ifdef LA_IPSEC_DEBUG @@ -1814,22 +1813,24 @@ cnxk_cpt_caps_populate(struct cnxk_cpt_vf *vf) unsigned long i; crypto_caps_populate(vf->crypto_caps, vf->cpt.hw_caps); - sec_crypto_caps_populate(vf->sec_crypto_caps, vf->cpt.hw_caps); + sec_ipsec_crypto_caps_populate(vf->sec_ipsec_crypto_caps, vf->cpt.hw_caps); PLT_STATIC_ASSERT(RTE_DIM(sec_caps_templ) <= RTE_DIM(vf->sec_caps)); memcpy(vf->sec_caps, sec_caps_templ, sizeof(sec_caps_templ)); for (i = 0; i < RTE_DIM(sec_caps_templ) - 1; i++) { - vf->sec_caps[i].crypto_capabilities = vf->sec_crypto_caps; - cnxk_sec_caps_update(&vf->sec_caps[i]); + if (vf->sec_caps[i].protocol == RTE_SECURITY_PROTOCOL_IPSEC) { + vf->sec_caps[i].crypto_capabilities = vf->sec_ipsec_crypto_caps; - if (roc_model_is_cn10k()) - cn10k_sec_caps_update(&vf->sec_caps[i]); + cnxk_sec_ipsec_caps_update(&vf->sec_caps[i]); - if (roc_model_is_cn9k()) - cn9k_sec_caps_update(&vf->sec_caps[i]); + if (roc_model_is_cn10k()) + cn10k_sec_ipsec_caps_update(&vf->sec_caps[i]); + if (roc_model_is_cn9k()) + cn9k_sec_ipsec_caps_update(&vf->sec_caps[i]); + } } } diff --git a/drivers/crypto/cnxk/cnxk_sg.h b/drivers/crypto/cnxk/cnxk_sg.h index 65244199bd..aa074581d7 100644 --- a/drivers/crypto/cnxk/cnxk_sg.h +++ b/drivers/crypto/cnxk/cnxk_sg.h @@ -129,7 +129,7 @@ fill_sg_comp_from_iov(struct roc_sglist_comp *list, uint32_t i, struct roc_se_io } static __rte_always_inline uint32_t -fill_ipsec_sg_comp_from_pkt(struct roc_sglist_comp *list, uint32_t i, struct rte_mbuf *pkt) +fill_sg_comp_from_pkt(struct roc_sglist_comp *list, uint32_t i, struct rte_mbuf *pkt) { uint32_t buf_sz; void *vaddr; @@ -150,7 +150,7 @@ fill_ipsec_sg_comp_from_pkt(struct roc_sglist_comp *list, uint32_t i, struct rte } static __rte_always_inline uint32_t -fill_ipsec_sg2_comp_from_pkt(struct roc_sg2list_comp *list, uint32_t i, struct rte_mbuf *pkt) +fill_sg2_comp_from_pkt(struct roc_sg2list_comp *list, uint32_t i, struct rte_mbuf *pkt) { uint32_t buf_sz; void *vaddr; From patchwork Thu Dec 21 12:35:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135442 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C4D8F43750; Thu, 21 Dec 2023 13:37:17 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 715F842EA8; Thu, 21 Dec 2023 13:36:25 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 4076742E95 for ; Thu, 21 Dec 2023 13:36:24 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVojD019540 for ; Thu, 21 Dec 2023 04:36:23 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=a/m61A7KcO7M8e6waN3f5dpUCBcE8jJT+cLfAh0RxTc=; b=JsO R/U4KalYF80xzvOG4+nxaSVKk8gAOjDzUE51vNtz38ls8HYntTo1b5Vqua4IaIxg KdfEse79V6A7Dh6lkr09bgEte4JyBqmGMkuqzahFH7723VtMor3fzpW/bUBVYIzw G/MX5GUVi5JyPkYr9S6Qa0Y/pQlvnqXbkE+SIvcorp+gn1mYrtBcupicF4TnY+uS af3wFW8S31UoZRjoV3cw1PMj07EtKa1MIVyVkAquXYBmutZXeq1GMNDF+vyGAnX8 3f2R/te64MDnJaQ1QxUlD/Y2PbevjT45o9MwgnQSiKhGLliRfgIVC9Qea058ZyHa 3XQqG1Vx+zjrDCoNm7w== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0j2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:23 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:21 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:21 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id E38AF3F7073; Thu, 21 Dec 2023 04:36:18 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Jerin Jacob , Vidya Sagar Velumuri , Tejasree Kondoj , Subject: [PATCH 12/24] common/cnxk: update opad-ipad gen to handle TLS Date: Thu, 21 Dec 2023 18:05:33 +0530 Message-ID: <20231221123545.510-13-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: A_OsX-83xDQDC06KK1jzxD1WZpwIrnA0 X-Proofpoint-ORIG-GUID: A_OsX-83xDQDC06KK1jzxD1WZpwIrnA0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For TLS opcodes, ipad is at the offset 64 as compared to the packed implementation for IPsec. Extend the function to handle TLS contexts as well. Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri --- drivers/common/cnxk/cnxk_security.c | 15 ++++++++------- drivers/common/cnxk/cnxk_security.h | 3 ++- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c index 81991c4697..bdb04fe142 100644 --- a/drivers/common/cnxk/cnxk_security.c +++ b/drivers/common/cnxk/cnxk_security.c @@ -9,7 +9,8 @@ #include "roc_api.h" void -cnxk_sec_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, uint8_t *hmac_opad_ipad) +cnxk_sec_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, uint8_t *hmac_opad_ipad, + bool is_tls) { const uint8_t *key = auth_xform->auth.key.data; uint32_t length = auth_xform->auth.key.length; @@ -29,11 +30,11 @@ cnxk_sec_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, uint8_t *hmac_op switch (auth_xform->auth.algo) { case RTE_CRYPTO_AUTH_MD5_HMAC: roc_hash_md5_gen(opad, (uint32_t *)&hmac_opad_ipad[0]); - roc_hash_md5_gen(ipad, (uint32_t *)&hmac_opad_ipad[24]); + roc_hash_md5_gen(ipad, (uint32_t *)&hmac_opad_ipad[is_tls ? 64 : 24]); break; case RTE_CRYPTO_AUTH_SHA1_HMAC: roc_hash_sha1_gen(opad, (uint32_t *)&hmac_opad_ipad[0]); - roc_hash_sha1_gen(ipad, (uint32_t *)&hmac_opad_ipad[24]); + roc_hash_sha1_gen(ipad, (uint32_t *)&hmac_opad_ipad[is_tls ? 64 : 24]); break; case RTE_CRYPTO_AUTH_SHA256_HMAC: roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0], 256); @@ -191,7 +192,7 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, const uint8_t *auth_key = auth_xfrm->auth.key.data; roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad); } else { - cnxk_sec_opad_ipad_gen(auth_xfrm, hmac_opad_ipad); + cnxk_sec_opad_ipad_gen(auth_xfrm, hmac_opad_ipad, false); } tmp_key = (uint64_t *)hmac_opad_ipad; @@ -740,7 +741,7 @@ onf_ipsec_sa_common_param_fill(struct roc_ie_onf_sa_ctl *ctl, uint8_t *salt, key = cipher_xfrm->cipher.key.data; length = cipher_xfrm->cipher.key.length; - cnxk_sec_opad_ipad_gen(auth_xfrm, hmac_opad_ipad); + cnxk_sec_opad_ipad_gen(auth_xfrm, hmac_opad_ipad, false); } switch (length) { @@ -1373,7 +1374,7 @@ cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipsec, roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad); } else if (auth_xform->auth.algo != RTE_CRYPTO_AUTH_NULL) { - cnxk_sec_opad_ipad_gen(auth_xform, hmac_opad_ipad); + cnxk_sec_opad_ipad_gen(auth_xform, hmac_opad_ipad, false); } } @@ -1440,7 +1441,7 @@ cnxk_on_ipsec_inb_sa_create(struct rte_security_ipsec_xform *ipsec, roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad); } else if (auth_xform->auth.algo != RTE_CRYPTO_AUTH_NULL) { - cnxk_sec_opad_ipad_gen(auth_xform, hmac_opad_ipad); + cnxk_sec_opad_ipad_gen(auth_xform, hmac_opad_ipad, false); } } diff --git a/drivers/common/cnxk/cnxk_security.h b/drivers/common/cnxk/cnxk_security.h index fabf694df4..86ec657cb0 100644 --- a/drivers/common/cnxk/cnxk_security.h +++ b/drivers/common/cnxk/cnxk_security.h @@ -70,6 +70,7 @@ int __roc_api cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipse struct roc_ie_on_outb_sa *out_sa); __rte_internal -void cnxk_sec_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, uint8_t *hmac_opad_ipad); +void cnxk_sec_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, uint8_t *hmac_opad_ipad, + bool is_tls); #endif /* _CNXK_SECURITY_H__ */ From patchwork Thu Dec 21 12:35:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135443 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2EDAC43750; Thu, 21 Dec 2023 13:37:24 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8EE2142E99; Thu, 21 Dec 2023 13:36:28 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 3789042EAC for ; Thu, 21 Dec 2023 13:36:27 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVJZF019120 for ; Thu, 21 Dec 2023 04:36:26 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=UKXKPDO+soD3v87UrKQCMng1X1fCg9UxOv0GO+tJkLs=; b=Z6K 7aPHScTORkKj7Hs0aqpCX2/qQvdHKOk5t3KjuyjT1rwzbtjO0XTTrmObegLDRg7z jsOVKoZWK+TmgpdbfP2drRFmJO0BPZeK9wARkuYJPS+vjvvEwRwAYVQ/GN7ntaML RRdRNhUvuupDR35GOwy5yLxdSwpS+hQrMoDd/lvZCIX3c990u1kiNG64w1AWRT5a mxYcsPDMH5WL+9Vn6OprFptrfPdXXcWI1OlCi4UIT6ao38X0Lb4VlKnr2FQHWOlO p0Q3XmJPX3vhV33vOMVp6fdI8i8yYbrN7/+g5ug26a0akg5F0NTF6//kRXD000Zk QampRqaoP7TXuuYitRw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0j6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:25 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:24 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:24 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 937E83F7079; Thu, 21 Dec 2023 04:36:21 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Jerin Jacob , Vidya Sagar Velumuri , Tejasree Kondoj , Subject: [PATCH 13/24] common/cnxk: add TLS record contexts Date: Thu, 21 Dec 2023 18:05:34 +0530 Message-ID: <20231221123545.510-14-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: jpVOciLjPRxfN5Knvy6w3Qiqn1w5h4pJ X-Proofpoint-ORIG-GUID: jpVOciLjPRxfN5Knvy6w3Qiqn1w5h4pJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add TLS record read and write contexts. Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri --- drivers/common/cnxk/roc_cpt.h | 4 +- drivers/common/cnxk/roc_ie_ot_tls.h | 199 ++++++++++++++++++++++++++++ drivers/common/cnxk/roc_se.h | 11 ++ 3 files changed, 211 insertions(+), 3 deletions(-) create mode 100644 drivers/common/cnxk/roc_ie_ot_tls.h diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h index 001e71c55e..5a2b5caeb0 100644 --- a/drivers/common/cnxk/roc_cpt.h +++ b/drivers/common/cnxk/roc_cpt.h @@ -55,6 +55,7 @@ #define ROC_CPT_AES_CBC_IV_LEN 16 #define ROC_CPT_SHA1_HMAC_LEN 12 #define ROC_CPT_SHA2_HMAC_LEN 16 +#define ROC_CPT_DES_IV_LEN 8 #define ROC_CPT_DES3_KEY_LEN 24 #define ROC_CPT_AES128_KEY_LEN 16 @@ -71,9 +72,6 @@ #define ROC_CPT_DES_BLOCK_LENGTH 8 #define ROC_CPT_AES_BLOCK_LENGTH 16 -#define ROC_CPT_AES_GCM_ROUNDUP_BYTE_LEN 4 -#define ROC_CPT_AES_CBC_ROUNDUP_BYTE_LEN 16 - /* Salt length for AES-CTR/GCM/CCM and AES-GMAC */ #define ROC_CPT_SALT_LEN 4 diff --git a/drivers/common/cnxk/roc_ie_ot_tls.h b/drivers/common/cnxk/roc_ie_ot_tls.h new file mode 100644 index 0000000000..61955ef4d1 --- /dev/null +++ b/drivers/common/cnxk/roc_ie_ot_tls.h @@ -0,0 +1,199 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef __ROC_IE_OT_TLS_H__ +#define __ROC_IE_OT_TLS_H__ + +#include "roc_platform.h" + +#define ROC_IE_OT_TLS_CTX_ILEN 1 +#define ROC_IE_OT_TLS_CTX_HDR_SIZE 1 +#define ROC_IE_OT_TLS_AR_WIN_SIZE_MAX 4096 +#define ROC_IE_OT_TLS_LOG_MIN_AR_WIN_SIZE_M1 5 + +/* u64 array size to fit anti replay window bits */ +#define ROC_IE_OT_TLS_AR_WINBITS_SZ \ + (PLT_ALIGN_CEIL(ROC_IE_OT_TLS_AR_WIN_SIZE_MAX, BITS_PER_LONG_LONG) / BITS_PER_LONG_LONG) + +/* CN10K TLS opcodes */ +#define ROC_IE_OT_TLS_MAJOR_OP_RECORD_ENC 0x16UL +#define ROC_IE_OT_TLS_MAJOR_OP_RECORD_DEC 0x17UL + +#define ROC_IE_OT_TLS_CTX_MAX_OPAD_IPAD_LEN 128 +#define ROC_IE_OT_TLS_CTX_MAX_KEY_IV_LEN 48 +#define ROC_IE_OT_TLS_CTX_MAX_IV_LEN 16 + +enum roc_ie_ot_tls_mac_type { + ROC_IE_OT_TLS_MAC_MD5 = 1, + ROC_IE_OT_TLS_MAC_SHA1 = 2, + ROC_IE_OT_TLS_MAC_SHA2_256 = 4, + ROC_IE_OT_TLS_MAC_SHA2_384 = 5, + ROC_IE_OT_TLS_MAC_SHA2_512 = 6, +}; + +enum roc_ie_ot_tls_cipher_type { + ROC_IE_OT_TLS_CIPHER_3DES = 1, + ROC_IE_OT_TLS_CIPHER_AES_CBC = 3, + ROC_IE_OT_TLS_CIPHER_AES_GCM = 7, + ROC_IE_OT_TLS_CIPHER_AES_CCM = 10, +}; + +enum roc_ie_ot_tls_ver { + ROC_IE_OT_TLS_VERSION_TLS_12 = 1, + ROC_IE_OT_TLS_VERSION_DTLS_12 = 2, +}; + +enum roc_ie_ot_tls_aes_key_len { + ROC_IE_OT_TLS_AES_KEY_LEN_128 = 1, + ROC_IE_OT_TLS_AES_KEY_LEN_256 = 3, +}; + +enum { + ROC_IE_OT_TLS_IV_SRC_DEFAULT = 0, + ROC_IE_OT_TLS_IV_SRC_FROM_SA = 1, +}; + +struct roc_ie_ot_tls_read_ctx_update_reg { + uint64_t ar_base; + uint64_t ar_valid_mask; + uint64_t hard_life; + uint64_t soft_life; + uint64_t mib_octs; + uint64_t mib_pkts; + uint64_t ar_winbits[ROC_IE_OT_TLS_AR_WINBITS_SZ]; +}; + +union roc_ie_ot_tls_param2 { + uint16_t u16; + struct { + uint8_t msg_type; + uint8_t rsvd; + } s; +}; + +struct roc_ie_ot_tls_read_sa { + /* Word0 */ + union { + struct { + uint64_t ar_win : 3; + uint64_t hard_life_dec : 1; + uint64_t soft_life_dec : 1; + uint64_t count_glb_octets : 1; + uint64_t count_glb_pkts : 1; + uint64_t count_mib_bytes : 1; + + uint64_t count_mib_pkts : 1; + uint64_t hw_ctx_off : 7; + + uint64_t ctx_id : 16; + + uint64_t orig_pkt_fabs : 1; + uint64_t orig_pkt_free : 1; + uint64_t pkind : 6; + + uint64_t rsvd0 : 1; + uint64_t et_ovrwr : 1; + uint64_t pkt_output : 2; + uint64_t pkt_format : 1; + uint64_t defrag_opt : 2; + uint64_t x2p_dst : 1; + + uint64_t ctx_push_size : 7; + uint64_t rsvd1 : 1; + + uint64_t ctx_hdr_size : 2; + uint64_t aop_valid : 1; + uint64_t rsvd2 : 1; + uint64_t ctx_size : 4; + } s; + uint64_t u64; + } w0; + + /* Word1 */ + uint64_t w1_rsvd3; + + /* Word2 */ + union { + struct { + uint64_t version_select : 4; + uint64_t aes_key_len : 2; + uint64_t cipher_select : 4; + uint64_t mac_select : 4; + uint64_t rsvd4 : 50; + } s; + uint64_t u64; + } w2; + + /* Word3 */ + uint64_t w3_rsvd5; + + /* Word4 - Word9 */ + uint8_t cipher_key[ROC_IE_OT_TLS_CTX_MAX_KEY_IV_LEN]; + + /* Word10 - Word25 */ + uint8_t opad_ipad[ROC_IE_OT_TLS_CTX_MAX_OPAD_IPAD_LEN]; + + /* Word26 - Word32 */ + struct roc_ie_ot_tls_read_ctx_update_reg ctx; +}; + +struct roc_ie_ot_tls_write_sa { + /* Word0 */ + union { + struct { + uint64_t rsvd0 : 3; + uint64_t hard_life_dec : 1; + uint64_t soft_life_dec : 1; + uint64_t count_glb_octets : 1; + uint64_t count_glb_pkts : 1; + uint64_t count_mib_bytes : 1; + + uint64_t count_mib_pkts : 1; + uint64_t hw_ctx_off : 7; + + uint64_t rsvd1 : 32; + + uint64_t ctx_push_size : 7; + uint64_t rsvd2 : 1; + + uint64_t ctx_hdr_size : 2; + uint64_t aop_valid : 1; + uint64_t rsvd3 : 1; + uint64_t ctx_size : 4; + } s; + uint64_t u64; + } w0; + + /* Word1 */ + uint64_t w1_rsvd4; + + /* Word2 */ + union { + struct { + uint64_t version_select : 4; + uint64_t aes_key_len : 2; + uint64_t cipher_select : 4; + uint64_t mac_select : 4; + uint64_t iv_at_cptr : 1; + uint64_t rsvd5 : 49; + } s; + uint64_t u64; + } w2; + + /* Word3 */ + uint64_t w3_rsvd6; + + /* Word4 - Word9 */ + uint8_t cipher_key[ROC_IE_OT_TLS_CTX_MAX_KEY_IV_LEN]; + + /* Word10 - Word25 */ + uint8_t opad_ipad[ROC_IE_OT_TLS_CTX_MAX_OPAD_IPAD_LEN]; + + /* Word26 */ + uint64_t w26_rsvd7; + + /* Word27 */ + uint64_t seq_num; +}; +#endif /* __ROC_IE_OT_TLS_H__ */ diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h index d8cbd58c9a..abb8c6a149 100644 --- a/drivers/common/cnxk/roc_se.h +++ b/drivers/common/cnxk/roc_se.h @@ -5,6 +5,8 @@ #ifndef __ROC_SE_H__ #define __ROC_SE_H__ +#include "roc_constants.h" + /* SE opcodes */ #define ROC_SE_MAJOR_OP_FC 0x33 #define ROC_SE_FC_MINOR_OP_ENCRYPT 0x0 @@ -162,6 +164,15 @@ typedef enum { ROC_SE_ERR_GC_ICV_MISCOMPARE = 0x4c, ROC_SE_ERR_GC_DATA_UNALIGNED = 0x4d, + ROC_SE_ERR_SSL_RECORD_LEN_INVALID = 0x82, + ROC_SE_ERR_SSL_CTX_LEN_INVALID = 0x83, + ROC_SE_ERR_SSL_CIPHER_UNSUPPORTED = 0x84, + ROC_SE_ERR_SSL_MAC_UNSUPPORTED = 0x85, + ROC_SE_ERR_SSL_VERSION_UNSUPPORTED = 0x86, + ROC_SE_ERR_SSL_MAC_MISMATCH = 0x89, + ROC_SE_ERR_SSL_PKT_REPLAY_SEQ_OUT_OF_WINDOW = 0xC1, + ROC_SE_ERR_SSL_PKT_REPLAY_SEQ = 0xC9, + /* API Layer */ ROC_SE_ERR_REQ_PENDING = 0xfe, ROC_SE_ERR_REQ_TIMEOUT = 0xff, From patchwork Thu Dec 21 12:35:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135444 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6F5143750; Thu, 21 Dec 2023 13:37:29 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D21B542E7C; Thu, 21 Dec 2023 13:36:30 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 8641C42EB0 for ; Thu, 21 Dec 2023 13:36:29 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVT4V019308 for ; Thu, 21 Dec 2023 04:36:28 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=cMICx3xSH7h8+wI7RU/qhkuhn+dc7a/7NJls34bYNXg=; b=IC0 xdwzhGELlCEdd/InrRCEbbArnHo5HJBoXXmGxpN/gQH2Ah0+nZj4MNiFz9awg0Ir /BWs+sOQDbj5x49sRky3HAj1fyDYcQBiKkvXmJlpJKUAh2IzFzrSB8xgk5RnILKs EdZuT3cwRhxmObuWra9A2X6j5Rpmm/Jak6g2H49MBn4rx6SS4uqw1GRx/FdW6GFY ppAHVzD3mRgX7LSS5YYwCs9MK3AHT7pRk8UR5eU6G0P1e2aOieVUTsJk2CG38/VX 30fG9A4cfqeRKOwRXPrmMfEdd4CVUK63C9hUvIxRpSMUvFVSP9IrEfhopdmiPDKN mS476QMlG6cc54kgK6w== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0jd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:28 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:27 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:27 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 41A643F7073; Thu, 21 Dec 2023 04:36:23 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Jerin Jacob , Vidya Sagar Velumuri , Tejasree Kondoj , Subject: [PATCH 14/24] crypto/cnxk: separate IPsec from security common code Date: Thu, 21 Dec 2023 18:05:35 +0530 Message-ID: <20231221123545.510-15-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: JvwZICjLeXQaghH2MlLewFfSaIAtHEy1 X-Proofpoint-ORIG-GUID: JvwZICjLeXQaghH2MlLewFfSaIAtHEy1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The current structs and functions assume only IPsec offload. Separate it out to allow for addition of TLS. Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri --- drivers/crypto/cnxk/cn10k_cryptodev.c | 2 +- drivers/crypto/cnxk/cn10k_cryptodev_sec.c | 127 ++++++++++++++++++++++ drivers/crypto/cnxk/cn10k_cryptodev_sec.h | 61 +++++++++++ drivers/crypto/cnxk/cn10k_ipsec.c | 127 +++------------------- drivers/crypto/cnxk/cn10k_ipsec.h | 45 +++----- drivers/crypto/cnxk/cn10k_ipsec_la_ops.h | 1 + drivers/crypto/cnxk/meson.build | 1 + 7 files changed, 218 insertions(+), 146 deletions(-) create mode 100644 drivers/crypto/cnxk/cn10k_cryptodev_sec.c create mode 100644 drivers/crypto/cnxk/cn10k_cryptodev_sec.h diff --git a/drivers/crypto/cnxk/cn10k_cryptodev.c b/drivers/crypto/cnxk/cn10k_cryptodev.c index 2fd4df3c5d..5ed918e18e 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev.c @@ -12,7 +12,7 @@ #include "cn10k_cryptodev.h" #include "cn10k_cryptodev_ops.h" -#include "cn10k_ipsec.h" +#include "cn10k_cryptodev_sec.h" #include "cnxk_cryptodev.h" #include "cnxk_cryptodev_capabilities.h" #include "cnxk_cryptodev_sec.h" diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_sec.c b/drivers/crypto/cnxk/cn10k_cryptodev_sec.c new file mode 100644 index 0000000000..0fd0a5b03c --- /dev/null +++ b/drivers/crypto/cnxk/cn10k_cryptodev_sec.c @@ -0,0 +1,127 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include + +#include "cn10k_cryptodev_ops.h" +#include "cn10k_cryptodev_sec.h" +#include "cnxk_cryptodev_ops.h" + +static int +cn10k_sec_session_create(void *dev, struct rte_security_session_conf *conf, + struct rte_security_session *sess) +{ + struct rte_cryptodev *crypto_dev = dev; + struct cnxk_cpt_vf *vf; + struct cnxk_cpt_qp *qp; + + if (conf->action_type != RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL) + return -EINVAL; + + qp = crypto_dev->data->queue_pairs[0]; + if (qp == NULL) { + plt_err("Setup cryptodev queue pair before creating security session"); + return -EPERM; + } + + vf = crypto_dev->data->dev_private; + + if (conf->protocol == RTE_SECURITY_PROTOCOL_IPSEC) { + ((struct cn10k_sec_session *)sess)->userdata = conf->userdata; + return cn10k_ipsec_session_create(vf, qp, &conf->ipsec, conf->crypto_xform, sess); + } + + return -ENOTSUP; +} + +static int +cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess) +{ + struct cn10k_sec_session *cn10k_sec_sess; + struct rte_cryptodev *crypto_dev = dev; + struct cnxk_cpt_qp *qp; + + if (unlikely(sec_sess == NULL)) + return -EINVAL; + + qp = crypto_dev->data->queue_pairs[0]; + if (unlikely(qp == NULL)) + return -ENOTSUP; + + cn10k_sec_sess = (struct cn10k_sec_session *)sec_sess; + + if (cn10k_sec_sess->proto == RTE_SECURITY_PROTOCOL_IPSEC) + return cn10k_sec_ipsec_session_destroy(qp, cn10k_sec_sess); + + return -EINVAL; +} + +static unsigned int +cn10k_sec_session_get_size(void *dev __rte_unused) +{ + return sizeof(struct cn10k_sec_session) - sizeof(struct rte_security_session); +} + +static int +cn10k_sec_session_stats_get(void *dev, struct rte_security_session *sec_sess, + struct rte_security_stats *stats) +{ + struct cn10k_sec_session *cn10k_sec_sess; + struct rte_cryptodev *crypto_dev = dev; + struct cnxk_cpt_qp *qp; + + if (unlikely(sec_sess == NULL)) + return -EINVAL; + + qp = crypto_dev->data->queue_pairs[0]; + if (unlikely(qp == NULL)) + return -ENOTSUP; + + cn10k_sec_sess = (struct cn10k_sec_session *)sec_sess; + + if (cn10k_sec_sess->proto == RTE_SECURITY_PROTOCOL_IPSEC) + return cn10k_ipsec_stats_get(qp, cn10k_sec_sess, stats); + + return -ENOTSUP; +} + +static int +cn10k_sec_session_update(void *dev, struct rte_security_session *sec_sess, + struct rte_security_session_conf *conf) +{ + struct cn10k_sec_session *cn10k_sec_sess; + struct rte_cryptodev *crypto_dev = dev; + struct cnxk_cpt_qp *qp; + struct cnxk_cpt_vf *vf; + + if (sec_sess == NULL) + return -EINVAL; + + qp = crypto_dev->data->queue_pairs[0]; + if (qp == NULL) + return -EINVAL; + + vf = crypto_dev->data->dev_private; + + cn10k_sec_sess = (struct cn10k_sec_session *)sec_sess; + + if (cn10k_sec_sess->proto == RTE_SECURITY_PROTOCOL_IPSEC) + return cn10k_ipsec_session_update(vf, qp, cn10k_sec_sess, conf); + + return -ENOTSUP; +} + +/* Update platform specific security ops */ +void +cn10k_sec_ops_override(void) +{ + /* Update platform specific ops */ + cnxk_sec_ops.session_create = cn10k_sec_session_create; + cnxk_sec_ops.session_destroy = cn10k_sec_session_destroy; + cnxk_sec_ops.session_get_size = cn10k_sec_session_get_size; + cnxk_sec_ops.session_stats_get = cn10k_sec_session_stats_get; + cnxk_sec_ops.session_update = cn10k_sec_session_update; + cnxk_sec_ops.inb_pkt_rx_inject = cn10k_cryptodev_sec_inb_rx_inject; + cnxk_sec_ops.rx_inject_configure = cn10k_cryptodev_sec_rx_inject_configure; +} diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_sec.h b/drivers/crypto/cnxk/cn10k_cryptodev_sec.h new file mode 100644 index 0000000000..02fd35eab7 --- /dev/null +++ b/drivers/crypto/cnxk/cn10k_cryptodev_sec.h @@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef __CN10K_CRYPTODEV_SEC_H__ +#define __CN10K_CRYPTODEV_SEC_H__ + +#include + +#include "roc_constants.h" +#include "roc_cpt.h" + +#include "cn10k_ipsec.h" + +struct cn10k_sec_session { + struct rte_security_session rte_sess; + + /** PMD private space */ + + enum rte_security_session_protocol proto; + /** Pre-populated CPT inst words */ + struct cnxk_cpt_inst_tmpl inst; + uint16_t max_extended_len; + uint16_t iv_offset; + uint8_t iv_length; + union { + struct { + uint8_t ip_csum; + bool is_outbound; + } ipsec; + }; + /** Queue pair */ + struct cnxk_cpt_qp *qp; + /** Userdata to be set for Rx inject */ + void *userdata; + + /** + * End of SW mutable area + */ + union { + struct cn10k_ipsec_sa sa; + }; +} __rte_aligned(ROC_ALIGN); + +static inline uint64_t +cpt_inst_w7_get(struct roc_cpt *roc_cpt, void *cptr) +{ + union cpt_inst_w7 w7; + + w7.u64 = 0; + w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_IE]; + w7.s.ctx_val = 1; + w7.s.cptr = (uint64_t)cptr; + rte_mb(); + + return w7.u64; +} + +void cn10k_sec_ops_override(void); + +#endif /* __CN10K_CRYPTODEV_SEC_H__ */ diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c index a9c673ea83..74d6cd70d1 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.c +++ b/drivers/crypto/cnxk/cn10k_ipsec.c @@ -11,6 +11,7 @@ #include #include "cn10k_cryptodev_ops.h" +#include "cn10k_cryptodev_sec.h" #include "cn10k_ipsec.h" #include "cnxk_cryptodev.h" #include "cnxk_cryptodev_ops.h" @@ -19,20 +20,6 @@ #include "roc_api.h" -static uint64_t -cpt_inst_w7_get(struct roc_cpt *roc_cpt, void *sa) -{ - union cpt_inst_w7 w7; - - w7.u64 = 0; - w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_IE]; - w7.s.ctx_val = 1; - w7.s.cptr = (uint64_t)sa; - rte_mb(); - - return w7.u64; -} - static int cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, struct rte_security_ipsec_xform *ipsec_xfrm, @@ -260,29 +247,19 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, return ret; } -static int -cn10k_ipsec_session_create(void *dev, +int +cn10k_ipsec_session_create(struct cnxk_cpt_vf *vf, struct cnxk_cpt_qp *qp, struct rte_security_ipsec_xform *ipsec_xfrm, struct rte_crypto_sym_xform *crypto_xfrm, struct rte_security_session *sess) { - struct rte_cryptodev *crypto_dev = dev; struct roc_cpt *roc_cpt; - struct cnxk_cpt_vf *vf; - struct cnxk_cpt_qp *qp; int ret; - qp = crypto_dev->data->queue_pairs[0]; - if (qp == NULL) { - plt_err("Setup cpt queue pair before creating security session"); - return -EPERM; - } - ret = cnxk_ipsec_xform_verify(ipsec_xfrm, crypto_xfrm); if (ret) return ret; - vf = crypto_dev->data->dev_private; roc_cpt = &vf->cpt; if (ipsec_xfrm->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS) @@ -293,38 +270,15 @@ cn10k_ipsec_session_create(void *dev, (struct cn10k_sec_session *)sess); } -static int -cn10k_sec_session_create(void *device, struct rte_security_session_conf *conf, - struct rte_security_session *sess) -{ - if (conf->action_type != RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL) - return -EINVAL; - - if (conf->protocol == RTE_SECURITY_PROTOCOL_IPSEC) { - ((struct cn10k_sec_session *)sess)->userdata = conf->userdata; - return cn10k_ipsec_session_create(device, &conf->ipsec, conf->crypto_xform, sess); - } - return -ENOTSUP; -} - -static int -cn10k_sec_ipsec_session_destroy(void *dev, struct rte_security_session *sec_sess) +int +cn10k_sec_ipsec_session_destroy(struct cnxk_cpt_qp *qp, struct cn10k_sec_session *sess) { - struct rte_cryptodev *crypto_dev = dev; union roc_ot_ipsec_sa_word2 *w2; - struct cn10k_sec_session *sess; struct cn10k_ipsec_sa *sa; - struct cnxk_cpt_qp *qp; struct roc_cpt_lf *lf; void *sa_dptr = NULL; int ret; - sess = (struct cn10k_sec_session *)sec_sess; - - qp = crypto_dev->data->queue_pairs[0]; - if (unlikely(qp == NULL)) - return -ENOTSUP; - lf = &qp->lf; sa = &sess->sa; @@ -374,48 +328,18 @@ cn10k_sec_ipsec_session_destroy(void *dev, struct rte_security_session *sec_sess return 0; } -static int -cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess) +int +cn10k_ipsec_stats_get(struct cnxk_cpt_qp *qp, struct cn10k_sec_session *sess, + struct rte_security_stats *stats) { - if (unlikely(sec_sess == NULL)) - return -EINVAL; - - if (((struct cn10k_sec_session *)sec_sess)->proto == RTE_SECURITY_PROTOCOL_IPSEC) - return cn10k_sec_ipsec_session_destroy(dev, sec_sess); - - return -EINVAL; -} - -static unsigned int -cn10k_sec_session_get_size(void *device __rte_unused) -{ - return sizeof(struct cn10k_sec_session) - sizeof(struct rte_security_session); -} - -static int -cn10k_sec_session_stats_get(void *device, struct rte_security_session *sess, - struct rte_security_stats *stats) -{ - struct rte_cryptodev *crypto_dev = device; struct roc_ot_ipsec_outb_sa *out_sa; struct roc_ot_ipsec_inb_sa *in_sa; - struct cn10k_sec_session *priv; struct cn10k_ipsec_sa *sa; - struct cnxk_cpt_qp *qp; - - if (unlikely(sess == NULL)) - return -EINVAL; - - priv = (struct cn10k_sec_session *)sess; - - qp = crypto_dev->data->queue_pairs[0]; - if (qp == NULL) - return -EINVAL; stats->protocol = RTE_SECURITY_PROTOCOL_IPSEC; - sa = &priv->sa; + sa = &sess->sa; - if (priv->ipsec.is_outbound) { + if (sess->ipsec.is_outbound) { out_sa = &sa->out_sa; roc_cpt_lf_ctx_flush(&qp->lf, out_sa, false); rte_delay_ms(1); @@ -432,23 +356,13 @@ cn10k_sec_session_stats_get(void *device, struct rte_security_session *sess, return 0; } -static int -cn10k_sec_session_update(void *device, struct rte_security_session *sess, - struct rte_security_session_conf *conf) +int +cn10k_ipsec_session_update(struct cnxk_cpt_vf *vf, struct cnxk_cpt_qp *qp, + struct cn10k_sec_session *sess, struct rte_security_session_conf *conf) { - struct rte_cryptodev *crypto_dev = device; struct roc_cpt *roc_cpt; - struct cnxk_cpt_qp *qp; - struct cnxk_cpt_vf *vf; int ret; - if (sess == NULL) - return -EINVAL; - - qp = crypto_dev->data->queue_pairs[0]; - if (qp == NULL) - return -EINVAL; - if (conf->ipsec.direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS) return -ENOTSUP; @@ -456,23 +370,8 @@ cn10k_sec_session_update(void *device, struct rte_security_session *sess, if (ret) return ret; - vf = crypto_dev->data->dev_private; roc_cpt = &vf->cpt; return cn10k_ipsec_outb_sa_create(roc_cpt, &qp->lf, &conf->ipsec, conf->crypto_xform, (struct cn10k_sec_session *)sess); } - -/* Update platform specific security ops */ -void -cn10k_sec_ops_override(void) -{ - /* Update platform specific ops */ - cnxk_sec_ops.session_create = cn10k_sec_session_create; - cnxk_sec_ops.session_destroy = cn10k_sec_session_destroy; - cnxk_sec_ops.session_get_size = cn10k_sec_session_get_size; - cnxk_sec_ops.session_stats_get = cn10k_sec_session_stats_get; - cnxk_sec_ops.session_update = cn10k_sec_session_update; - cnxk_sec_ops.inb_pkt_rx_inject = cn10k_cryptodev_sec_inb_rx_inject; - cnxk_sec_ops.rx_inject_configure = cn10k_cryptodev_sec_rx_inject_configure; -} diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h index 2b7a3e6acf..0d1e14a065 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.h +++ b/drivers/crypto/cnxk/cn10k_ipsec.h @@ -11,9 +11,12 @@ #include "roc_constants.h" #include "roc_ie_ot.h" +#include "cnxk_cryptodev.h" +#include "cnxk_cryptodev_ops.h" #include "cnxk_ipsec.h" -typedef void *CN10K_SA_CONTEXT_MARKER[0]; +/* Forward declaration */ +struct cn10k_sec_session; struct cn10k_ipsec_sa { union { @@ -24,34 +27,14 @@ struct cn10k_ipsec_sa { }; } __rte_aligned(ROC_ALIGN); -struct cn10k_sec_session { - struct rte_security_session rte_sess; - - /** PMD private space */ - - enum rte_security_session_protocol proto; - /** Pre-populated CPT inst words */ - struct cnxk_cpt_inst_tmpl inst; - uint16_t max_extended_len; - uint16_t iv_offset; - uint8_t iv_length; - union { - struct { - uint8_t ip_csum; - bool is_outbound; - } ipsec; - }; - /** Queue pair */ - struct cnxk_cpt_qp *qp; - /** Userdata to be set for Rx inject */ - void *userdata; - - /** - * End of SW mutable area - */ - struct cn10k_ipsec_sa sa; -} __rte_aligned(ROC_ALIGN); - -void cn10k_sec_ops_override(void); - +int cn10k_ipsec_session_create(struct cnxk_cpt_vf *vf, struct cnxk_cpt_qp *qp, + struct rte_security_ipsec_xform *ipsec_xfrm, + struct rte_crypto_sym_xform *crypto_xfrm, + struct rte_security_session *sess); +int cn10k_sec_ipsec_session_destroy(struct cnxk_cpt_qp *qp, struct cn10k_sec_session *sess); +int cn10k_ipsec_stats_get(struct cnxk_cpt_qp *qp, struct cn10k_sec_session *sess, + struct rte_security_stats *stats); +int cn10k_ipsec_session_update(struct cnxk_cpt_vf *vf, struct cnxk_cpt_qp *qp, + struct cn10k_sec_session *sess, + struct rte_security_session_conf *conf); #endif /* __CN10K_IPSEC_H__ */ diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h index af2c85022e..a30b8e413d 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h +++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h @@ -11,6 +11,7 @@ #include "roc_ie.h" #include "cn10k_cryptodev.h" +#include "cn10k_cryptodev_sec.h" #include "cn10k_ipsec.h" #include "cnxk_cryptodev.h" #include "cnxk_cryptodev_ops.h" diff --git a/drivers/crypto/cnxk/meson.build b/drivers/crypto/cnxk/meson.build index 3d9a0dbbf0..d6fafd43d9 100644 --- a/drivers/crypto/cnxk/meson.build +++ b/drivers/crypto/cnxk/meson.build @@ -14,6 +14,7 @@ sources = files( 'cn9k_ipsec.c', 'cn10k_cryptodev.c', 'cn10k_cryptodev_ops.c', + 'cn10k_cryptodev_sec.c', 'cn10k_ipsec.c', 'cnxk_cryptodev.c', 'cnxk_cryptodev_capabilities.c', From patchwork Thu Dec 21 12:35:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135445 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9DB6F43750; Thu, 21 Dec 2023 13:37:38 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5A20C42EB4; Thu, 21 Dec 2023 13:36:34 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 63D6742E60 for ; Thu, 21 Dec 2023 13:36:32 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVGSv018988 for ; Thu, 21 Dec 2023 04:36:31 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=LiLbsF2Ofdjy1TH4nwMswJUf9xW7c2YsSYs/IMLH5b4=; b=cpN XutCLAkfJVu29jVWG39lYZrtmNeyRdUPZP7ugH8HvlID529u3H12gqfz1syQvqs1 CIHiOS50xY7Go9n0cz4PGrRbTIW6n/Ecc90UEETb6d9pRQ52rIRXSGxWgChihV7p LLcOLVo7aVsNmiOoU51MH/wn7hay8dyS2c0xmIVPR7outEYa5eEPny6kXnbpa2zj oPwEGQWwgkHCyxDoBMchvYVDhkWba/H1TFeSZDTYUX5m2eMOhIJUIgZq3eXeQEnU hPW6n1QzPVwawLwtCdh17vvQR5b9F2hmo9sYlRyCLV6hrZn5K2ZkbHMxN1qwRx/Q iPOkI45RBidEeDCjvVA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0jm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:31 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:29 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:29 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id E35FB3F7079; Thu, 21 Dec 2023 04:36:26 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 15/24] crypto/cnxk: add TLS record session ops Date: Thu, 21 Dec 2023 18:05:36 +0530 Message-ID: <20231221123545.510-16-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 1d9LY2E2dE5AFcDPUV6caggOE-9CzRN- X-Proofpoint-ORIG-GUID: 1d9LY2E2dE5AFcDPUV6caggOE-9CzRN- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Add TLS record session ops for creating and destroying security sessions. Add support for both read and write sessions. Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri --- drivers/crypto/cnxk/cn10k_cryptodev_sec.h | 8 + drivers/crypto/cnxk/cn10k_tls.c | 758 ++++++++++++++++++++++ drivers/crypto/cnxk/cn10k_tls.h | 35 + drivers/crypto/cnxk/meson.build | 1 + 4 files changed, 802 insertions(+) create mode 100644 drivers/crypto/cnxk/cn10k_tls.c create mode 100644 drivers/crypto/cnxk/cn10k_tls.h diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_sec.h b/drivers/crypto/cnxk/cn10k_cryptodev_sec.h index 02fd35eab7..33fd3aa398 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_sec.h +++ b/drivers/crypto/cnxk/cn10k_cryptodev_sec.h @@ -11,6 +11,7 @@ #include "roc_cpt.h" #include "cn10k_ipsec.h" +#include "cn10k_tls.h" struct cn10k_sec_session { struct rte_security_session rte_sess; @@ -28,6 +29,12 @@ struct cn10k_sec_session { uint8_t ip_csum; bool is_outbound; } ipsec; + struct { + uint8_t enable_padding : 1; + uint8_t hdr_len : 4; + uint8_t rvsd : 3; + bool is_write; + } tls; }; /** Queue pair */ struct cnxk_cpt_qp *qp; @@ -39,6 +46,7 @@ struct cn10k_sec_session { */ union { struct cn10k_ipsec_sa sa; + struct cn10k_tls_record tls_rec; }; } __rte_aligned(ROC_ALIGN); diff --git a/drivers/crypto/cnxk/cn10k_tls.c b/drivers/crypto/cnxk/cn10k_tls.c new file mode 100644 index 0000000000..e1ed65b06a --- /dev/null +++ b/drivers/crypto/cnxk/cn10k_tls.c @@ -0,0 +1,758 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include +#include +#include + +#include + +#include "roc_cpt.h" +#include "roc_se.h" + +#include "cn10k_cryptodev_sec.h" +#include "cn10k_tls.h" +#include "cnxk_cryptodev.h" +#include "cnxk_cryptodev_ops.h" +#include "cnxk_security.h" + +static int +tls_xform_cipher_verify(struct rte_crypto_sym_xform *crypto_xform) +{ + enum rte_crypto_cipher_algorithm c_algo = crypto_xform->cipher.algo; + uint16_t keylen = crypto_xform->cipher.key.length; + + if (((c_algo == RTE_CRYPTO_CIPHER_NULL) && (keylen == 0)) || + ((c_algo == RTE_CRYPTO_CIPHER_3DES_CBC) && (keylen == 24)) || + ((c_algo == RTE_CRYPTO_CIPHER_AES_CBC) && ((keylen == 16) || (keylen == 32)))) + return 0; + + return -EINVAL; +} + +static int +tls_xform_auth_verify(struct rte_crypto_sym_xform *crypto_xform) +{ + enum rte_crypto_auth_algorithm a_algo = crypto_xform->auth.algo; + uint16_t keylen = crypto_xform->auth.key.length; + + if (((a_algo == RTE_CRYPTO_AUTH_MD5_HMAC) && (keylen == 16)) || + ((a_algo == RTE_CRYPTO_AUTH_SHA1_HMAC) && (keylen == 20)) || + ((a_algo == RTE_CRYPTO_AUTH_SHA256_HMAC) && (keylen == 32))) + return 0; + + return -EINVAL; +} + +static int +tls_xform_aead_verify(struct rte_security_tls_record_xform *tls_xform, + struct rte_crypto_sym_xform *crypto_xform) +{ + uint16_t keylen = crypto_xform->aead.key.length; + + if (tls_xform->type == RTE_SECURITY_TLS_SESS_TYPE_WRITE && + crypto_xform->aead.op != RTE_CRYPTO_AEAD_OP_ENCRYPT) + return -EINVAL; + + if (tls_xform->type == RTE_SECURITY_TLS_SESS_TYPE_READ && + crypto_xform->aead.op != RTE_CRYPTO_AEAD_OP_DECRYPT) + return -EINVAL; + + if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) { + if ((keylen == 16) || (keylen == 32)) + return 0; + } + + return -EINVAL; +} + +static int +cnxk_tls_xform_verify(struct rte_security_tls_record_xform *tls_xform, + struct rte_crypto_sym_xform *crypto_xform) +{ + struct rte_crypto_sym_xform *auth_xform, *cipher_xform = NULL; + int ret = 0; + + if ((tls_xform->ver != RTE_SECURITY_VERSION_TLS_1_2) && + (tls_xform->ver != RTE_SECURITY_VERSION_DTLS_1_2)) + return -EINVAL; + + if ((tls_xform->type != RTE_SECURITY_TLS_SESS_TYPE_READ) && + (tls_xform->type != RTE_SECURITY_TLS_SESS_TYPE_WRITE)) + return -EINVAL; + + if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) + return tls_xform_aead_verify(tls_xform, crypto_xform); + + if (tls_xform->type == RTE_SECURITY_TLS_SESS_TYPE_WRITE) { + /* Egress */ + + /* First should be for auth in Egress */ + if (crypto_xform->type != RTE_CRYPTO_SYM_XFORM_AUTH) + return -EINVAL; + + /* Next if present, should be for cipher in Egress */ + if ((crypto_xform->next != NULL) && + (crypto_xform->next->type != RTE_CRYPTO_SYM_XFORM_CIPHER)) + return -EINVAL; + + auth_xform = crypto_xform; + cipher_xform = crypto_xform->next; + } else { + /* Ingress */ + + /* First can be for auth only when next is NULL in Ingress. */ + if ((crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AUTH) && + (crypto_xform->next != NULL)) + return -EINVAL; + else if ((crypto_xform->type != RTE_CRYPTO_SYM_XFORM_CIPHER) || + (crypto_xform->next->type != RTE_CRYPTO_SYM_XFORM_AUTH)) + return -EINVAL; + + cipher_xform = crypto_xform; + auth_xform = crypto_xform->next; + } + + if (cipher_xform) { + if ((tls_xform->type == RTE_SECURITY_TLS_SESS_TYPE_WRITE) && + !(cipher_xform->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT && + auth_xform->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE)) + return -EINVAL; + + if ((tls_xform->type == RTE_SECURITY_TLS_SESS_TYPE_READ) && + !(cipher_xform->cipher.op == RTE_CRYPTO_CIPHER_OP_DECRYPT && + auth_xform->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY)) + return -EINVAL; + } else { + if ((tls_xform->type == RTE_SECURITY_TLS_SESS_TYPE_WRITE) && + (auth_xform->auth.op != RTE_CRYPTO_AUTH_OP_GENERATE)) + return -EINVAL; + + if ((tls_xform->type == RTE_SECURITY_TLS_SESS_TYPE_READ) && + (auth_xform->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY)) + return -EINVAL; + } + + if (cipher_xform) + ret = tls_xform_cipher_verify(cipher_xform); + + if (!ret) + return tls_xform_auth_verify(auth_xform); + + return ret; +} + +static int +tls_write_rlens_get(struct rte_security_tls_record_xform *tls_xfrm, + struct rte_crypto_sym_xform *crypto_xfrm) +{ + enum rte_crypto_cipher_algorithm c_algo = RTE_CRYPTO_CIPHER_NULL; + enum rte_crypto_auth_algorithm a_algo = RTE_CRYPTO_AUTH_NULL; + uint8_t roundup_byte, tls_hdr_len; + uint8_t mac_len, iv_len; + + switch (tls_xfrm->ver) { + case RTE_SECURITY_VERSION_TLS_1_2: + case RTE_SECURITY_VERSION_TLS_1_3: + tls_hdr_len = 5; + break; + case RTE_SECURITY_VERSION_DTLS_1_2: + tls_hdr_len = 13; + break; + default: + tls_hdr_len = 0; + break; + } + + /* Get Cipher and Auth algo */ + if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) + return tls_hdr_len + ROC_CPT_AES_GCM_IV_LEN + ROC_CPT_AES_GCM_MAC_LEN; + + if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + c_algo = crypto_xfrm->cipher.algo; + if (crypto_xfrm->next) + a_algo = crypto_xfrm->next->auth.algo; + } else { + a_algo = crypto_xfrm->auth.algo; + if (crypto_xfrm->next) + c_algo = crypto_xfrm->next->cipher.algo; + } + + switch (c_algo) { + case RTE_CRYPTO_CIPHER_NULL: + roundup_byte = 4; + iv_len = 0; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + roundup_byte = ROC_CPT_DES_BLOCK_LENGTH; + iv_len = ROC_CPT_DES_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_AES_CBC: + roundup_byte = ROC_CPT_AES_BLOCK_LENGTH; + iv_len = ROC_CPT_AES_CBC_IV_LEN; + break; + default: + roundup_byte = 0; + iv_len = 0; + break; + } + + switch (a_algo) { + case RTE_CRYPTO_AUTH_NULL: + mac_len = 0; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + mac_len = 16; + break; + case RTE_CRYPTO_AUTH_SHA1_HMAC: + mac_len = 20; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + mac_len = 32; + break; + default: + mac_len = 0; + break; + } + + return tls_hdr_len + iv_len + mac_len + roundup_byte; +} + +static void +tls_write_sa_init(struct roc_ie_ot_tls_write_sa *sa) +{ + size_t offset; + + memset(sa, 0, sizeof(struct roc_ie_ot_tls_write_sa)); + + offset = offsetof(struct roc_ie_ot_tls_write_sa, w26_rsvd7); + sa->w0.s.hw_ctx_off = offset / ROC_CTX_UNIT_8B; + sa->w0.s.ctx_push_size = sa->w0.s.hw_ctx_off; + sa->w0.s.ctx_size = ROC_IE_OT_TLS_CTX_ILEN; + sa->w0.s.ctx_hdr_size = ROC_IE_OT_TLS_CTX_HDR_SIZE; + sa->w0.s.aop_valid = 1; +} + +static void +tls_read_sa_init(struct roc_ie_ot_tls_read_sa *sa) +{ + size_t offset; + + memset(sa, 0, sizeof(struct roc_ie_ot_tls_read_sa)); + + offset = offsetof(struct roc_ie_ot_tls_read_sa, ctx); + sa->w0.s.hw_ctx_off = offset / ROC_CTX_UNIT_8B; + sa->w0.s.ctx_push_size = sa->w0.s.hw_ctx_off; + sa->w0.s.ctx_size = ROC_IE_OT_TLS_CTX_ILEN; + sa->w0.s.ctx_hdr_size = ROC_IE_OT_TLS_CTX_HDR_SIZE; + sa->w0.s.aop_valid = 1; +} + +static size_t +tls_read_ctx_size(struct roc_ie_ot_tls_read_sa *sa) +{ + size_t size; + + /* Variable based on Anti-replay Window */ + size = offsetof(struct roc_ie_ot_tls_read_sa, ctx) + + offsetof(struct roc_ie_ot_tls_read_ctx_update_reg, ar_winbits); + + if (sa->w0.s.ar_win) + size += (1 << (sa->w0.s.ar_win - 1)) * sizeof(uint64_t); + + return size; +} + +static int +tls_read_sa_fill(struct roc_ie_ot_tls_read_sa *read_sa, + struct rte_security_tls_record_xform *tls_xfrm, + struct rte_crypto_sym_xform *crypto_xfrm) +{ + struct rte_crypto_sym_xform *auth_xfrm, *cipher_xfrm; + const uint8_t *key = NULL; + uint64_t *tmp, *tmp_key; + uint32_t replay_win_sz; + uint8_t *cipher_key; + int i, length = 0; + size_t offset; + + /* Initialize the SA */ + memset(read_sa, 0, sizeof(struct roc_ie_ot_tls_read_sa)); + + cipher_key = read_sa->cipher_key; + + /* Set encryption algorithm */ + if ((crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) && + (crypto_xfrm->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)) { + read_sa->w2.s.cipher_select = ROC_IE_OT_TLS_CIPHER_AES_GCM; + read_sa->w2.s.mac_select = ROC_IE_OT_TLS_MAC_SHA2_256; + + length = crypto_xfrm->aead.key.length; + if (length == 16) + read_sa->w2.s.aes_key_len = ROC_IE_OT_TLS_AES_KEY_LEN_128; + else + read_sa->w2.s.aes_key_len = ROC_IE_OT_TLS_AES_KEY_LEN_256; + + key = crypto_xfrm->aead.key.data; + memcpy(cipher_key, key, length); + + if (tls_xfrm->ver == RTE_SECURITY_VERSION_TLS_1_2) + memcpy(((uint8_t *)cipher_key + 32), &tls_xfrm->tls_1_2.imp_nonce, 4); + else if (tls_xfrm->ver == RTE_SECURITY_VERSION_DTLS_1_2) + memcpy(((uint8_t *)cipher_key + 32), &tls_xfrm->dtls_1_2.imp_nonce, 4); + + goto key_swap; + } + + if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + auth_xfrm = crypto_xfrm; + cipher_xfrm = crypto_xfrm->next; + } else { + cipher_xfrm = crypto_xfrm; + auth_xfrm = crypto_xfrm->next; + } + + if (cipher_xfrm != NULL) { + if (cipher_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC) { + read_sa->w2.s.cipher_select = ROC_IE_OT_TLS_CIPHER_3DES; + length = cipher_xfrm->cipher.key.length; + } else if (cipher_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) { + read_sa->w2.s.cipher_select = ROC_IE_OT_TLS_CIPHER_AES_CBC; + length = cipher_xfrm->cipher.key.length; + if (length == 16) + read_sa->w2.s.aes_key_len = ROC_IE_OT_TLS_AES_KEY_LEN_128; + else if (length == 32) + read_sa->w2.s.aes_key_len = ROC_IE_OT_TLS_AES_KEY_LEN_256; + else + return -EINVAL; + } else { + return -EINVAL; + } + + key = cipher_xfrm->cipher.key.data; + memcpy(cipher_key, key, length); + } + + if (auth_xfrm->auth.algo == RTE_CRYPTO_AUTH_MD5_HMAC) + read_sa->w2.s.mac_select = ROC_IE_OT_TLS_MAC_MD5; + else if (auth_xfrm->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) + read_sa->w2.s.mac_select = ROC_IE_OT_TLS_MAC_SHA1; + else if (auth_xfrm->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC) + read_sa->w2.s.mac_select = ROC_IE_OT_TLS_MAC_SHA2_256; + else + return -EINVAL; + + cnxk_sec_opad_ipad_gen(auth_xfrm, read_sa->opad_ipad, true); + tmp = (uint64_t *)read_sa->opad_ipad; + for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN / sizeof(uint64_t)); i++) + tmp[i] = rte_be_to_cpu_64(tmp[i]); + +key_swap: + tmp_key = (uint64_t *)cipher_key; + for (i = 0; i < (int)(ROC_IE_OT_TLS_CTX_MAX_KEY_IV_LEN / sizeof(uint64_t)); i++) + tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]); + + if (tls_xfrm->ver == RTE_SECURITY_VERSION_DTLS_1_2) { + /* Only support power-of-two window sizes supported */ + replay_win_sz = tls_xfrm->dtls_1_2.ar_win_sz; + if (replay_win_sz) { + if (!rte_is_power_of_2(replay_win_sz) || + replay_win_sz > ROC_IE_OT_TLS_AR_WIN_SIZE_MAX) + return -ENOTSUP; + + read_sa->w0.s.ar_win = rte_log2_u32(replay_win_sz) - 5; + } + } + + read_sa->w0.s.ctx_hdr_size = ROC_IE_OT_TLS_CTX_HDR_SIZE; + read_sa->w0.s.aop_valid = 1; + + offset = offsetof(struct roc_ie_ot_tls_read_sa, ctx); + + /* Word offset for HW managed CTX field */ + read_sa->w0.s.hw_ctx_off = offset / 8; + read_sa->w0.s.ctx_push_size = read_sa->w0.s.hw_ctx_off; + + /* Entire context size in 128B units */ + read_sa->w0.s.ctx_size = (PLT_ALIGN_CEIL(tls_read_ctx_size(read_sa), ROC_CTX_UNIT_128B) / + ROC_CTX_UNIT_128B) - + 1; + + if (tls_xfrm->ver == RTE_SECURITY_VERSION_TLS_1_2) { + read_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_TLS_12; + read_sa->ctx.ar_valid_mask = tls_xfrm->tls_1_2.seq_no - 1; + } else if (tls_xfrm->ver == RTE_SECURITY_VERSION_DTLS_1_2) { + read_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_DTLS_12; + } + + rte_wmb(); + + return 0; +} + +static int +tls_write_sa_fill(struct roc_ie_ot_tls_write_sa *write_sa, + struct rte_security_tls_record_xform *tls_xfrm, + struct rte_crypto_sym_xform *crypto_xfrm) +{ + struct rte_crypto_sym_xform *auth_xfrm, *cipher_xfrm; + const uint8_t *key = NULL; + uint8_t *cipher_key; + uint64_t *tmp_key; + int i, length = 0; + size_t offset; + + cipher_key = write_sa->cipher_key; + + /* Set encryption algorithm */ + if ((crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) && + (crypto_xfrm->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)) { + write_sa->w2.s.cipher_select = ROC_IE_OT_TLS_CIPHER_AES_GCM; + write_sa->w2.s.mac_select = ROC_IE_OT_TLS_MAC_SHA2_256; + + length = crypto_xfrm->aead.key.length; + if (length == 16) + write_sa->w2.s.aes_key_len = ROC_IE_OT_TLS_AES_KEY_LEN_128; + else + write_sa->w2.s.aes_key_len = ROC_IE_OT_TLS_AES_KEY_LEN_256; + + key = crypto_xfrm->aead.key.data; + memcpy(cipher_key, key, length); + + if (tls_xfrm->ver == RTE_SECURITY_VERSION_TLS_1_2) + memcpy(((uint8_t *)cipher_key + 32), &tls_xfrm->tls_1_2.imp_nonce, 4); + else if (tls_xfrm->ver == RTE_SECURITY_VERSION_DTLS_1_2) + memcpy(((uint8_t *)cipher_key + 32), &tls_xfrm->dtls_1_2.imp_nonce, 4); + + goto key_swap; + } + + if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + auth_xfrm = crypto_xfrm; + cipher_xfrm = crypto_xfrm->next; + } else { + cipher_xfrm = crypto_xfrm; + auth_xfrm = crypto_xfrm->next; + } + + if (cipher_xfrm != NULL) { + if (cipher_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC) { + write_sa->w2.s.cipher_select = ROC_IE_OT_TLS_CIPHER_3DES; + length = cipher_xfrm->cipher.key.length; + } else if (cipher_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) { + write_sa->w2.s.cipher_select = ROC_IE_OT_TLS_CIPHER_AES_CBC; + length = cipher_xfrm->cipher.key.length; + if (length == 16) + write_sa->w2.s.aes_key_len = ROC_IE_OT_TLS_AES_KEY_LEN_128; + else if (length == 32) + write_sa->w2.s.aes_key_len = ROC_IE_OT_TLS_AES_KEY_LEN_256; + else + return -EINVAL; + } else { + return -EINVAL; + } + + key = cipher_xfrm->cipher.key.data; + if (key != NULL && length != 0) { + /* Copy encryption key */ + memcpy(cipher_key, key, length); + } + } + + if (auth_xfrm != NULL) { + if (auth_xfrm->auth.algo == RTE_CRYPTO_AUTH_MD5_HMAC) + write_sa->w2.s.mac_select = ROC_IE_OT_TLS_MAC_MD5; + else if (auth_xfrm->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) + write_sa->w2.s.mac_select = ROC_IE_OT_TLS_MAC_SHA1; + else if (auth_xfrm->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC) + write_sa->w2.s.mac_select = ROC_IE_OT_TLS_MAC_SHA2_256; + else + return -EINVAL; + + cnxk_sec_opad_ipad_gen(auth_xfrm, write_sa->opad_ipad, true); + } + + tmp_key = (uint64_t *)write_sa->opad_ipad; + for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN / sizeof(uint64_t)); i++) + tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]); + +key_swap: + tmp_key = (uint64_t *)cipher_key; + for (i = 0; i < (int)(ROC_IE_OT_TLS_CTX_MAX_KEY_IV_LEN / sizeof(uint64_t)); i++) + tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]); + + write_sa->w0.s.ctx_hdr_size = ROC_IE_OT_TLS_CTX_HDR_SIZE; + offset = offsetof(struct roc_ie_ot_tls_write_sa, w26_rsvd7); + + /* Word offset for HW managed CTX field */ + write_sa->w0.s.hw_ctx_off = offset / 8; + write_sa->w0.s.ctx_push_size = write_sa->w0.s.hw_ctx_off; + + /* Entire context size in 128B units */ + write_sa->w0.s.ctx_size = + (PLT_ALIGN_CEIL(sizeof(struct roc_ie_ot_tls_write_sa), ROC_CTX_UNIT_128B) / + ROC_CTX_UNIT_128B) - + 1; + write_sa->w0.s.aop_valid = 1; + + if (tls_xfrm->ver == RTE_SECURITY_VERSION_TLS_1_2) { + write_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_TLS_12; + write_sa->seq_num = tls_xfrm->tls_1_2.seq_no - 1; + } else if (tls_xfrm->ver == RTE_SECURITY_VERSION_DTLS_1_2) { + write_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_DTLS_12; + write_sa->seq_num = ((uint64_t)tls_xfrm->dtls_1_2.epoch << 48) | + (tls_xfrm->dtls_1_2.seq_no & 0x0000ffffffffffff); + write_sa->seq_num -= 1; + } + + write_sa->w2.s.iv_at_cptr = ROC_IE_OT_TLS_IV_SRC_DEFAULT; + +#ifdef LA_IPSEC_DEBUG + if (tls_xfrm->options.iv_gen_disable == 1) + write_sa->w2.s.iv_at_cptr = ROC_IE_OT_TLS_IV_SRC_FROM_SA; +#else + if (tls_xfrm->options.iv_gen_disable) { + plt_err("Application provided IV is not supported"); + return -ENOTSUP; + } +#endif + + rte_wmb(); + + return 0; +} + +static int +cn10k_tls_read_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, + struct rte_security_tls_record_xform *tls_xfrm, + struct rte_crypto_sym_xform *crypto_xfrm, + struct cn10k_sec_session *sec_sess) +{ + struct roc_ie_ot_tls_read_sa *sa_dptr; + struct cn10k_tls_record *tls; + union cpt_inst_w4 inst_w4; + void *read_sa; + int ret = 0; + + tls = &sec_sess->tls_rec; + read_sa = &tls->read_sa; + + /* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */ + sa_dptr = plt_zmalloc(sizeof(struct roc_ie_ot_tls_read_sa), 8); + if (sa_dptr == NULL) { + plt_err("Couldn't allocate memory for SA dptr"); + return -ENOMEM; + } + + /* Translate security parameters to SA */ + ret = tls_read_sa_fill(sa_dptr, tls_xfrm, crypto_xfrm); + if (ret) { + plt_err("Could not fill read session parameters"); + goto sa_dptr_free; + } + if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + sec_sess->iv_offset = crypto_xfrm->aead.iv.offset; + sec_sess->iv_length = crypto_xfrm->aead.iv.length; + } else if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + sec_sess->iv_offset = crypto_xfrm->cipher.iv.offset; + sec_sess->iv_length = crypto_xfrm->cipher.iv.length; + } else { + sec_sess->iv_offset = crypto_xfrm->auth.iv.offset; + sec_sess->iv_length = crypto_xfrm->auth.iv.length; + } + + if (sa_dptr->w2.s.version_select == ROC_IE_OT_TLS_VERSION_DTLS_12) + sec_sess->tls.hdr_len = 13; + else if (sa_dptr->w2.s.version_select == ROC_IE_OT_TLS_VERSION_TLS_12) + sec_sess->tls.hdr_len = 5; + + sec_sess->proto = RTE_SECURITY_PROTOCOL_TLS_RECORD; + + /* Enable mib counters */ + sa_dptr->w0.s.count_mib_bytes = 1; + sa_dptr->w0.s.count_mib_pkts = 1; + + /* pre-populate CPT INST word 4 */ + inst_w4.u64 = 0; + inst_w4.s.opcode_major = ROC_IE_OT_TLS_MAJOR_OP_RECORD_DEC | ROC_IE_OT_INPLACE_BIT; + + sec_sess->inst.w4 = inst_w4.u64; + sec_sess->inst.w7 = cpt_inst_w7_get(roc_cpt, read_sa); + + memset(read_sa, 0, sizeof(struct roc_ie_ot_tls_read_sa)); + + /* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */ + memcpy(read_sa, sa_dptr, 8); + + rte_atomic_thread_fence(rte_memory_order_seq_cst); + + /* Write session using microcode opcode */ + ret = roc_cpt_ctx_write(lf, sa_dptr, read_sa, sizeof(struct roc_ie_ot_tls_read_sa)); + if (ret) { + plt_err("Could not write read session to hardware"); + goto sa_dptr_free; + } + + /* Trigger CTX flush so that data is written back to DRAM */ + roc_cpt_lf_ctx_flush(lf, read_sa, true); + + rte_atomic_thread_fence(rte_memory_order_seq_cst); + +sa_dptr_free: + plt_free(sa_dptr); + + return ret; +} + +static int +cn10k_tls_write_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, + struct rte_security_tls_record_xform *tls_xfrm, + struct rte_crypto_sym_xform *crypto_xfrm, + struct cn10k_sec_session *sec_sess) +{ + struct roc_ie_ot_tls_write_sa *sa_dptr; + struct cn10k_tls_record *tls; + union cpt_inst_w4 inst_w4; + void *write_sa; + int ret = 0; + + tls = &sec_sess->tls_rec; + write_sa = &tls->write_sa; + + /* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */ + sa_dptr = plt_zmalloc(sizeof(struct roc_ie_ot_tls_write_sa), 8); + if (sa_dptr == NULL) { + plt_err("Couldn't allocate memory for SA dptr"); + return -ENOMEM; + } + + /* Translate security parameters to SA */ + ret = tls_write_sa_fill(sa_dptr, tls_xfrm, crypto_xfrm); + if (ret) { + plt_err("Could not fill write session parameters"); + goto sa_dptr_free; + } + + if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + sec_sess->iv_offset = crypto_xfrm->aead.iv.offset; + sec_sess->iv_length = crypto_xfrm->aead.iv.length; + } else if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + sec_sess->iv_offset = crypto_xfrm->cipher.iv.offset; + sec_sess->iv_length = crypto_xfrm->cipher.iv.length; + } else { + sec_sess->iv_offset = crypto_xfrm->next->cipher.iv.offset; + sec_sess->iv_length = crypto_xfrm->next->cipher.iv.length; + } + + sec_sess->tls.is_write = true; + sec_sess->tls.enable_padding = tls_xfrm->options.extra_padding_enable; + sec_sess->max_extended_len = tls_write_rlens_get(tls_xfrm, crypto_xfrm); + sec_sess->proto = RTE_SECURITY_PROTOCOL_TLS_RECORD; + + /* pre-populate CPT INST word 4 */ + inst_w4.u64 = 0; + inst_w4.s.opcode_major = ROC_IE_OT_TLS_MAJOR_OP_RECORD_ENC | ROC_IE_OT_INPLACE_BIT; + + sec_sess->inst.w4 = inst_w4.u64; + sec_sess->inst.w7 = cpt_inst_w7_get(roc_cpt, write_sa); + + memset(write_sa, 0, sizeof(struct roc_ie_ot_tls_write_sa)); + + /* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */ + memcpy(write_sa, sa_dptr, 8); + + rte_atomic_thread_fence(rte_memory_order_seq_cst); + + /* Write session using microcode opcode */ + ret = roc_cpt_ctx_write(lf, sa_dptr, write_sa, sizeof(struct roc_ie_ot_tls_write_sa)); + if (ret) { + plt_err("Could not write tls write session to hardware"); + goto sa_dptr_free; + } + + /* Trigger CTX flush so that data is written back to DRAM */ + roc_cpt_lf_ctx_flush(lf, write_sa, false); + + rte_atomic_thread_fence(rte_memory_order_seq_cst); + +sa_dptr_free: + plt_free(sa_dptr); + + return ret; +} + +int +cn10k_tls_record_session_create(struct cnxk_cpt_vf *vf, struct cnxk_cpt_qp *qp, + struct rte_security_tls_record_xform *tls_xfrm, + struct rte_crypto_sym_xform *crypto_xfrm, + struct rte_security_session *sess) +{ + struct roc_cpt *roc_cpt; + int ret; + + ret = cnxk_tls_xform_verify(tls_xfrm, crypto_xfrm); + if (ret) + return ret; + + roc_cpt = &vf->cpt; + + if (tls_xfrm->type == RTE_SECURITY_TLS_SESS_TYPE_READ) + return cn10k_tls_read_sa_create(roc_cpt, &qp->lf, tls_xfrm, crypto_xfrm, + (struct cn10k_sec_session *)sess); + else + return cn10k_tls_write_sa_create(roc_cpt, &qp->lf, tls_xfrm, crypto_xfrm, + (struct cn10k_sec_session *)sess); +} + +int +cn10k_sec_tls_session_destroy(struct cnxk_cpt_qp *qp, struct cn10k_sec_session *sess) +{ + struct cn10k_tls_record *tls; + struct roc_cpt_lf *lf; + void *sa_dptr = NULL; + int ret; + + lf = &qp->lf; + + tls = &sess->tls_rec; + + /* Trigger CTX flush to write dirty data back to DRAM */ + roc_cpt_lf_ctx_flush(lf, &tls->read_sa, false); + + ret = -1; + + if (sess->tls.is_write) { + sa_dptr = plt_zmalloc(sizeof(struct roc_ie_ot_tls_write_sa), 8); + if (sa_dptr != NULL) { + tls_write_sa_init(sa_dptr); + + ret = roc_cpt_ctx_write(lf, sa_dptr, &tls->write_sa, + sizeof(struct roc_ie_ot_tls_write_sa)); + } + } else { + sa_dptr = plt_zmalloc(sizeof(struct roc_ie_ot_tls_read_sa), 8); + if (sa_dptr != NULL) { + tls_read_sa_init(sa_dptr); + + ret = roc_cpt_ctx_write(lf, sa_dptr, &tls->read_sa, + sizeof(struct roc_ie_ot_tls_read_sa)); + } + } + + plt_free(sa_dptr); + + if (ret) { + /* MC write_ctx failed. Attempt reload of CTX */ + + /* Wait for 1 ms so that flush is complete */ + rte_delay_ms(1); + + rte_atomic_thread_fence(rte_memory_order_seq_cst); + + /* Trigger CTX reload to fetch new data from DRAM */ + roc_cpt_lf_ctx_reload(lf, &tls->read_sa); + } + + return 0; +} diff --git a/drivers/crypto/cnxk/cn10k_tls.h b/drivers/crypto/cnxk/cn10k_tls.h new file mode 100644 index 0000000000..c477d51169 --- /dev/null +++ b/drivers/crypto/cnxk/cn10k_tls.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef __CN10K_TLS_H__ +#define __CN10K_TLS_H__ + +#include +#include + +#include "roc_ie_ot_tls.h" + +#include "cnxk_cryptodev.h" +#include "cnxk_cryptodev_ops.h" + +/* Forward declaration */ +struct cn10k_sec_session; + +struct cn10k_tls_record { + union { + /** Read SA */ + struct roc_ie_ot_tls_read_sa read_sa; + /** Write SA */ + struct roc_ie_ot_tls_write_sa write_sa; + }; +} __rte_aligned(ROC_ALIGN); + +int cn10k_tls_record_session_create(struct cnxk_cpt_vf *vf, struct cnxk_cpt_qp *qp, + struct rte_security_tls_record_xform *tls_xfrm, + struct rte_crypto_sym_xform *crypto_xfrm, + struct rte_security_session *sess); + +int cn10k_sec_tls_session_destroy(struct cnxk_cpt_qp *qp, struct cn10k_sec_session *sess); + +#endif /* __CN10K_TLS_H__ */ diff --git a/drivers/crypto/cnxk/meson.build b/drivers/crypto/cnxk/meson.build index d6fafd43d9..ee0c65e32a 100644 --- a/drivers/crypto/cnxk/meson.build +++ b/drivers/crypto/cnxk/meson.build @@ -16,6 +16,7 @@ sources = files( 'cn10k_cryptodev_ops.c', 'cn10k_cryptodev_sec.c', 'cn10k_ipsec.c', + 'cn10k_tls.c', 'cnxk_cryptodev.c', 'cnxk_cryptodev_capabilities.c', 'cnxk_cryptodev_devargs.c', From patchwork Thu Dec 21 12:35:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135446 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5128F43750; Thu, 21 Dec 2023 13:37:45 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CB10640A81; Thu, 21 Dec 2023 13:36:52 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id C78CB40A7A for ; Thu, 21 Dec 2023 13:36:50 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS30019305 for ; Thu, 21 Dec 2023 04:36:50 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=ooczj5tBdgwBBToxYaYPBuy7UD8WFOqDWIKaa0dCMYY=; b=ZvR wPXnuO+ZMquXZ8b63wZDmkiiBlU2M26WUmf8p5sYzu9fXBX0Q4r0vAUFnQucw2aO Z3+X20ZBgiW5c2RC6AwvvFsu4I1BjMpBp2InAVwbU/oV6V6RaqRGBDqcS52L2loa YSm8vLN3URAccGGZDfVoMvfaHI0tX61fS6sSSFPfsgT2h/28HarW6F4ekJG2Io2v ZFbO7mIVAkQRXSRafIUTMKOvDTPExDxG50xc/rPgb6kgLvtek4gNd2qyj8BIQ+3T Sv+7y9QrNc7+W+xzlQQWbKyhhWILqwyC60NrGq/qvgOtDgoFwqdpjMRzqnGEU5pj XWI++JBAHBKnOirW2oQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0jq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:49 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:32 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:32 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 949BA3F7073; Thu, 21 Dec 2023 04:36:29 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 16/24] crypto/cnxk: add TLS record datapath handling Date: Thu, 21 Dec 2023 18:05:37 +0530 Message-ID: <20231221123545.510-17-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: TrESxIed-PYKHlacPyLLb1-_Y5NMnJgW X-Proofpoint-ORIG-GUID: TrESxIed-PYKHlacPyLLb1-_Y5NMnJgW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Add support for TLS record handling in datapath. Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 57 +++- drivers/crypto/cnxk/cn10k_cryptodev_sec.c | 7 + drivers/crypto/cnxk/cn10k_tls_ops.h | 322 ++++++++++++++++++++++ 3 files changed, 380 insertions(+), 6 deletions(-) create mode 100644 drivers/crypto/cnxk/cn10k_tls_ops.h diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index f105a431f8..c87a8bae1a 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -20,11 +20,14 @@ #include "roc_sso_dp.h" #include "cn10k_cryptodev.h" -#include "cn10k_cryptodev_ops.h" #include "cn10k_cryptodev_event_dp.h" +#include "cn10k_cryptodev_ops.h" +#include "cn10k_cryptodev_sec.h" #include "cn10k_eventdev.h" #include "cn10k_ipsec.h" #include "cn10k_ipsec_la_ops.h" +#include "cn10k_tls.h" +#include "cn10k_tls_ops.h" #include "cnxk_ae.h" #include "cnxk_cryptodev.h" #include "cnxk_cryptodev_ops.h" @@ -101,6 +104,18 @@ cpt_sec_ipsec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, return ret; } +static __rte_always_inline int __rte_hot +cpt_sec_tls_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, + struct cn10k_sec_session *sess, struct cpt_inst_s *inst, + struct cpt_inflight_req *infl_req, const bool is_sg_ver2) +{ + if (sess->tls.is_write) + return process_tls_write(&qp->lf, op, sess, &qp->meta_info, infl_req, inst, + is_sg_ver2); + else + return process_tls_read(op, sess, &qp->meta_info, infl_req, inst, is_sg_ver2); +} + static __rte_always_inline int __rte_hot cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cn10k_sec_session *sess, struct cpt_inst_s *inst, struct cpt_inflight_req *infl_req, const bool is_sg_ver2) @@ -108,6 +123,8 @@ cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cn10k if (sess->proto == RTE_SECURITY_PROTOCOL_IPSEC) return cpt_sec_ipsec_inst_fill(qp, op, sess, &inst[0], infl_req, is_sg_ver2); + else if (sess->proto == RTE_SECURITY_PROTOCOL_TLS_RECORD) + return cpt_sec_tls_inst_fill(qp, op, sess, &inst[0], infl_req, is_sg_ver2); return 0; } @@ -812,7 +829,7 @@ cn10k_cpt_sg_ver2_crypto_adapter_enqueue(void *ws, struct rte_event ev[], uint16 } static inline void -cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *res) +cn10k_cpt_ipsec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *res) { struct rte_mbuf *mbuf = cop->sym->m_src; const uint16_t m_len = res->rlen; @@ -849,10 +866,38 @@ cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *re } static inline void -cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, - struct rte_crypto_op *cop, - struct cpt_inflight_req *infl_req, - struct cpt_cn10k_res_s *res) +cn10k_cpt_tls_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *res) +{ + struct rte_mbuf *mbuf = cop->sym->m_src; + const uint16_t m_len = res->rlen; + + if (!res->uc_compcode) { + if (mbuf->next == NULL) + mbuf->data_len = m_len; + mbuf->pkt_len = m_len; + } else { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + cop->aux_flags = res->uc_compcode; + plt_err("crypto op failed with UC compcode: 0x%x", res->uc_compcode); + } +} + +static inline void +cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *res) +{ + struct rte_crypto_sym_op *sym_op = cop->sym; + struct cn10k_sec_session *sess; + + sess = sym_op->session; + if (sess->proto == RTE_SECURITY_PROTOCOL_IPSEC) + cn10k_cpt_ipsec_post_process(cop, res); + else if (sess->proto == RTE_SECURITY_PROTOCOL_TLS_RECORD) + cn10k_cpt_tls_post_process(cop, res); +} + +static inline void +cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, + struct cpt_inflight_req *infl_req, struct cpt_cn10k_res_s *res) { const uint8_t uc_compcode = res->uc_compcode; const uint8_t compcode = res->compcode; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_sec.c b/drivers/crypto/cnxk/cn10k_cryptodev_sec.c index 0fd0a5b03c..300a8e4f94 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_sec.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_sec.c @@ -32,6 +32,10 @@ cn10k_sec_session_create(void *dev, struct rte_security_session_conf *conf, return cn10k_ipsec_session_create(vf, qp, &conf->ipsec, conf->crypto_xform, sess); } + if (conf->protocol == RTE_SECURITY_PROTOCOL_TLS_RECORD) + return cn10k_tls_record_session_create(vf, qp, &conf->tls_record, + conf->crypto_xform, sess); + return -ENOTSUP; } @@ -54,6 +58,9 @@ cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess) if (cn10k_sec_sess->proto == RTE_SECURITY_PROTOCOL_IPSEC) return cn10k_sec_ipsec_session_destroy(qp, cn10k_sec_sess); + if (cn10k_sec_sess->proto == RTE_SECURITY_PROTOCOL_TLS_RECORD) + return cn10k_sec_tls_session_destroy(qp, cn10k_sec_sess); + return -EINVAL; } diff --git a/drivers/crypto/cnxk/cn10k_tls_ops.h b/drivers/crypto/cnxk/cn10k_tls_ops.h new file mode 100644 index 0000000000..a5d38bacbb --- /dev/null +++ b/drivers/crypto/cnxk/cn10k_tls_ops.h @@ -0,0 +1,322 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef __CN10K_TLS_OPS_H__ +#define __CN10K_TLS_OPS_H__ + +#include +#include + +#include "roc_ie.h" + +#include "cn10k_cryptodev.h" +#include "cn10k_cryptodev_sec.h" +#include "cnxk_cryptodev.h" +#include "cnxk_cryptodev_ops.h" +#include "cnxk_sg.h" + +static __rte_always_inline int +process_tls_write(struct roc_cpt_lf *lf, struct rte_crypto_op *cop, struct cn10k_sec_session *sess, + struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, + struct cpt_inst_s *inst, const bool is_sg_ver2) +{ + struct rte_crypto_sym_op *sym_op = cop->sym; +#ifdef LA_IPSEC_DEBUG + struct roc_ie_ot_tls_write_sa *write_sa; +#endif + struct rte_mbuf *m_src = sym_op->m_src; + struct rte_mbuf *last_seg; + union cpt_inst_w4 w4; + void *m_data = NULL; + uint8_t *in_buffer; + +#ifdef LA_IPSEC_DEBUG + write_sa = &sess->tls_rec.write_sa; + if (write_sa->w2.s.iv_at_cptr == ROC_IE_OT_TLS_IV_SRC_FROM_SA) { + + uint8_t *iv = PLT_PTR_ADD(write_sa->cipher_key, 32); + + if (write_sa->w2.s.cipher_select == ROC_IE_OT_TLS_CIPHER_AES_GCM) { + uint32_t *tmp; + + /* For GCM, the IV and salt format will be like below: + * iv[0-3]: lower bytes of IV in BE format. + * iv[4-7]: salt / nonce. + * iv[12-15]: upper bytes of IV in BE format. + */ + memcpy(iv, rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset), 4); + tmp = (uint32_t *)iv; + *tmp = rte_be_to_cpu_32(*tmp); + + memcpy(iv + 12, + rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset + 4), 4); + tmp = (uint32_t *)(iv + 12); + *tmp = rte_be_to_cpu_32(*tmp); + } else if (write_sa->w2.s.cipher_select == ROC_IE_OT_TLS_CIPHER_AES_CBC) { + uint64_t *tmp; + + memcpy(iv, rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset), 16); + tmp = (uint64_t *)iv; + *tmp = rte_be_to_cpu_64(*tmp); + tmp = (uint64_t *)(iv + 8); + *tmp = rte_be_to_cpu_64(*tmp); + } else if (write_sa->w2.s.cipher_select == ROC_IE_OT_TLS_CIPHER_3DES) { + uint64_t *tmp; + + memcpy(iv, rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset), 8); + tmp = (uint64_t *)iv; + *tmp = rte_be_to_cpu_64(*tmp); + } + + /* Trigger CTX reload to fetch new data from DRAM */ + roc_cpt_lf_ctx_reload(lf, write_sa); + rte_delay_ms(1); + } +#else + RTE_SET_USED(lf); +#endif + /* Single buffer direct mode */ + if (likely(m_src->next == NULL)) { + void *vaddr; + + if (unlikely(rte_pktmbuf_tailroom(m_src) < sess->max_extended_len)) { + plt_dp_err("Not enough tail room"); + return -ENOMEM; + } + + vaddr = rte_pktmbuf_mtod(m_src, void *); + inst->dptr = (uint64_t)vaddr; + inst->rptr = (uint64_t)vaddr; + + w4.u64 = sess->inst.w4; + w4.s.param1 = m_src->data_len; + w4.s.dlen = m_src->data_len; + + w4.s.param2 = cop->param1.tls_record.content_type; + w4.s.opcode_minor = sess->tls.enable_padding * cop->aux_flags * 8; + + inst->w4.u64 = w4.u64; + } else if (is_sg_ver2 == false) { + struct roc_sglist_comp *scatter_comp, *gather_comp; + uint32_t g_size_bytes, s_size_bytes; + uint32_t dlen; + int i; + + last_seg = rte_pktmbuf_lastseg(m_src); + + if (unlikely(rte_pktmbuf_tailroom(last_seg) < sess->max_extended_len)) { + plt_dp_err("Not enough tail room (required: %d, available: %d)", + sess->max_extended_len, rte_pktmbuf_tailroom(last_seg)); + return -ENOMEM; + } + + m_data = alloc_op_meta(NULL, m_info->mlen, m_info->pool, infl_req); + if (unlikely(m_data == NULL)) { + plt_dp_err("Error allocating meta buffer for request"); + return -ENOMEM; + } + + in_buffer = (uint8_t *)m_data; + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* Input Gather List */ + i = 0; + gather_comp = (struct roc_sglist_comp *)((uint8_t *)in_buffer + 8); + + i = fill_sg_comp_from_pkt(gather_comp, i, m_src); + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + + g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + i = 0; + scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); + + i = fill_sg_comp_from_pkt(scatter_comp, i, m_src); + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + + s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + dlen = g_size_bytes + s_size_bytes + ROC_SG_LIST_HDR_SIZE; + + inst->dptr = (uint64_t)in_buffer; + inst->rptr = (uint64_t)in_buffer; + + w4.u64 = sess->inst.w4; + w4.s.dlen = dlen; + w4.s.param1 = rte_pktmbuf_pkt_len(m_src); + w4.s.param2 = cop->param1.tls_record.content_type; + w4.s.opcode_major |= (uint64_t)ROC_DMA_MODE_SG; + w4.s.opcode_minor = sess->tls.enable_padding * cop->aux_flags * 8; + + /* Output Scatter List */ + last_seg->data_len += sess->max_extended_len; + inst->w4.u64 = w4.u64; + } else { + struct roc_sg2list_comp *scatter_comp, *gather_comp; + union cpt_inst_w5 cpt_inst_w5; + union cpt_inst_w6 cpt_inst_w6; + uint32_t g_size_bytes; + int i; + + last_seg = rte_pktmbuf_lastseg(m_src); + + if (unlikely(rte_pktmbuf_tailroom(last_seg) < sess->max_extended_len)) { + plt_dp_err("Not enough tail room (required: %d, available: %d)", + sess->max_extended_len, rte_pktmbuf_tailroom(last_seg)); + return -ENOMEM; + } + + m_data = alloc_op_meta(NULL, m_info->mlen, m_info->pool, infl_req); + if (unlikely(m_data == NULL)) { + plt_dp_err("Error allocating meta buffer for request"); + return -ENOMEM; + } + + in_buffer = (uint8_t *)m_data; + /* Input Gather List */ + i = 0; + gather_comp = (struct roc_sg2list_comp *)((uint8_t *)in_buffer); + i = fill_sg2_comp_from_pkt(gather_comp, i, m_src); + + cpt_inst_w5.s.gather_sz = ((i + 2) / 3); + g_size_bytes = ((i + 2) / 3) * sizeof(struct roc_sg2list_comp); + + i = 0; + scatter_comp = (struct roc_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); + + i = fill_sg2_comp_from_pkt(scatter_comp, i, m_src); + + cpt_inst_w6.s.scatter_sz = ((i + 2) / 3); + + cpt_inst_w5.s.dptr = (uint64_t)gather_comp; + cpt_inst_w6.s.rptr = (uint64_t)scatter_comp; + + inst->w5.u64 = cpt_inst_w5.u64; + inst->w6.u64 = cpt_inst_w6.u64; + w4.u64 = sess->inst.w4; + w4.s.dlen = rte_pktmbuf_pkt_len(m_src); + w4.s.opcode_major &= (~(ROC_IE_OT_INPLACE_BIT)); + w4.s.opcode_minor = sess->tls.enable_padding * cop->aux_flags * 8; + w4.s.param1 = w4.s.dlen; + w4.s.param2 = cop->param1.tls_record.content_type; + /* Output Scatter List */ + last_seg->data_len += sess->max_extended_len; + inst->w4.u64 = w4.u64; + } + + return 0; +} + +static __rte_always_inline int +process_tls_read(struct rte_crypto_op *cop, struct cn10k_sec_session *sess, + struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, + struct cpt_inst_s *inst, const bool is_sg_ver2) +{ + struct rte_crypto_sym_op *sym_op = cop->sym; + struct rte_mbuf *m_src = sym_op->m_src; + union cpt_inst_w4 w4; + uint8_t *in_buffer; + void *m_data; + + if (likely(m_src->next == NULL)) { + void *vaddr; + + vaddr = rte_pktmbuf_mtod(m_src, void *); + + inst->dptr = (uint64_t)vaddr; + inst->rptr = (uint64_t)vaddr; + + w4.u64 = sess->inst.w4; + w4.s.dlen = m_src->data_len; + w4.s.param1 = m_src->data_len; + inst->w4.u64 = w4.u64; + } else if (is_sg_ver2 == false) { + struct roc_sglist_comp *scatter_comp, *gather_comp; + uint32_t g_size_bytes, s_size_bytes; + uint32_t dlen; + int i; + + m_data = alloc_op_meta(NULL, m_info->mlen, m_info->pool, infl_req); + if (unlikely(m_data == NULL)) { + plt_dp_err("Error allocating meta buffer for request"); + return -ENOMEM; + } + + in_buffer = (uint8_t *)m_data; + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* Input Gather List */ + i = 0; + gather_comp = (struct roc_sglist_comp *)((uint8_t *)in_buffer + 8); + + i = fill_sg_comp_from_pkt(gather_comp, i, m_src); + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + + g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + i = 0; + scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); + + i = fill_sg_comp_from_pkt(scatter_comp, i, m_src); + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + + s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + dlen = g_size_bytes + s_size_bytes + ROC_SG_LIST_HDR_SIZE; + + inst->dptr = (uint64_t)in_buffer; + inst->rptr = (uint64_t)in_buffer; + + w4.u64 = sess->inst.w4; + w4.s.dlen = dlen; + w4.s.opcode_major |= (uint64_t)ROC_DMA_MODE_SG; + w4.s.param1 = rte_pktmbuf_pkt_len(m_src); + inst->w4.u64 = w4.u64; + } else { + struct roc_sg2list_comp *scatter_comp, *gather_comp; + union cpt_inst_w5 cpt_inst_w5; + union cpt_inst_w6 cpt_inst_w6; + uint32_t g_size_bytes; + int i; + + m_data = alloc_op_meta(NULL, m_info->mlen, m_info->pool, infl_req); + if (unlikely(m_data == NULL)) { + plt_dp_err("Error allocating meta buffer for request"); + return -ENOMEM; + } + + in_buffer = (uint8_t *)m_data; + /* Input Gather List */ + i = 0; + + gather_comp = (struct roc_sg2list_comp *)((uint8_t *)in_buffer); + i = fill_sg2_comp_from_pkt(gather_comp, i, m_src); + + cpt_inst_w5.s.gather_sz = ((i + 2) / 3); + g_size_bytes = ((i + 2) / 3) * sizeof(struct roc_sg2list_comp); + + i = 0; + scatter_comp = (struct roc_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); + + i = fill_sg2_comp_from_pkt(scatter_comp, i, m_src); + + cpt_inst_w6.s.scatter_sz = ((i + 2) / 3); + + cpt_inst_w5.s.dptr = (uint64_t)gather_comp; + cpt_inst_w6.s.rptr = (uint64_t)scatter_comp; + + inst->w5.u64 = cpt_inst_w5.u64; + inst->w6.u64 = cpt_inst_w6.u64; + w4.u64 = sess->inst.w4; + w4.s.dlen = rte_pktmbuf_pkt_len(m_src); + w4.s.param1 = w4.s.dlen; + w4.s.opcode_major &= (~(ROC_IE_OT_INPLACE_BIT)); + inst->w4.u64 = w4.u64; + } + + return 0; +} +#endif /* __CN10K_TLS_OPS_H__ */ From patchwork Thu Dec 21 12:35:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135449 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1828B43750; Thu, 21 Dec 2023 13:38:09 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 71E6B42ED2; Thu, 21 Dec 2023 13:36:57 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 89FD642EB8 for ; Thu, 21 Dec 2023 13:36:54 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS34019305 for ; Thu, 21 Dec 2023 04:36:53 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=z1gWFZ1xbdY5YLY4AxpQk0tr2V85UDU8suSr7Ho+7Vo=; b=coj XG5fD7TqrE7GBA6qYcwWPVbLMTtGN2oFWi9vGSucUy7/wQw7rb84NpU0l8qL5pDa tl7dd90SpiB2JXBD5qyByVy9SMKgu59EIbWvF56Dv1TYaVPNsRc8XWO3QRQdl44E YaEE3WT8JDkIZEXIrUYBG/+Edy9H3016XSn+reS+Qm2koziktN4yVwTWSyhc7TJp m00L2L+rvatAj5NCvVUeV5fTVUON+ym2YfH4yX+tQ3cD7Os/PA3Rw6wKTUyj7GAr bjuyu4C9BwJDBCM0Jh0GAe9N+j+FV6YtcgM/X2phZ+eH0vSYTbVk3r+QRgFoEztl 1uYSk634og5Aey/RHoA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0k0-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:50 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:35 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:34 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 3E88E3F707D; Thu, 21 Dec 2023 04:36:31 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 17/24] crypto/cnxk: add TLS capability Date: Thu, 21 Dec 2023 18:05:38 +0530 Message-ID: <20231221123545.510-18-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: RadFP5BPgbf8-ojvcF8j0phh_QfQCm7O X-Proofpoint-ORIG-GUID: RadFP5BPgbf8-ojvcF8j0phh_QfQCm7O X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Add TLS 1.2 record read and write capability. Add DTLS 1.2 record read and write capability. Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri --- doc/guides/rel_notes/release_24_03.rst | 2 + drivers/common/cnxk/hw/cpt.h | 3 +- drivers/crypto/cnxk/cnxk_cryptodev.h | 12 +- .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 210 ++++++++++++++++++ 4 files changed, 223 insertions(+), 4 deletions(-) diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst index fa30b46ead..0ebbae9f4e 100644 --- a/doc/guides/rel_notes/release_24_03.rst +++ b/doc/guides/rel_notes/release_24_03.rst @@ -58,6 +58,8 @@ New Features * **Updated Marvell cnxk crypto driver.** * Added support for Rx inject in crypto_cn10k. + * Added support for TLS record processing in crypto_cn10k. Supports TLS 1.2 + and DTLS 1.2. Removed Items ------------- diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h index edab8a5d83..2620965606 100644 --- a/drivers/common/cnxk/hw/cpt.h +++ b/drivers/common/cnxk/hw/cpt.h @@ -80,7 +80,8 @@ union cpt_eng_caps { uint64_t __io sg_ver2 : 1; uint64_t __io sm2 : 1; uint64_t __io pdcp_chain_zuc256 : 1; - uint64_t __io reserved_38_63 : 26; + uint64_t __io tls : 1; + uint64_t __io reserved_39_63 : 25; }; }; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index a5c4365631..8c8c58a76b 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -11,9 +11,11 @@ #include "roc_ae.h" #include "roc_cpt.h" -#define CNXK_CPT_MAX_CAPS 55 -#define CNXK_SEC_IPSEC_CRYPTO_MAX_CAPS 16 -#define CNXK_SEC_MAX_CAPS 9 +#define CNXK_CPT_MAX_CAPS 55 +#define CNXK_SEC_IPSEC_CRYPTO_MAX_CAPS 16 +#define CNXK_SEC_TLS_1_3_CRYPTO_MAX_CAPS 2 +#define CNXK_SEC_TLS_1_2_CRYPTO_MAX_CAPS 6 +#define CNXK_SEC_MAX_CAPS 17 /** * Device private data @@ -25,6 +27,10 @@ struct cnxk_cpt_vf { struct roc_cpt cpt; struct rte_cryptodev_capabilities crypto_caps[CNXK_CPT_MAX_CAPS]; struct rte_cryptodev_capabilities sec_ipsec_crypto_caps[CNXK_SEC_IPSEC_CRYPTO_MAX_CAPS]; + struct rte_cryptodev_capabilities sec_tls_1_3_crypto_caps[CNXK_SEC_TLS_1_3_CRYPTO_MAX_CAPS]; + struct rte_cryptodev_capabilities sec_tls_1_2_crypto_caps[CNXK_SEC_TLS_1_2_CRYPTO_MAX_CAPS]; + struct rte_cryptodev_capabilities + sec_dtls_1_2_crypto_caps[CNXK_SEC_TLS_1_2_CRYPTO_MAX_CAPS]; struct rte_security_capability sec_caps[CNXK_SEC_MAX_CAPS]; uint64_t cnxk_fpm_iova[ROC_AE_EC_ID_PMAX]; struct roc_ae_ec_group *ec_grp[ROC_AE_EC_ID_PMAX]; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index 178f510a63..73100377d9 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -30,6 +30,16 @@ RTE_DIM(sec_ipsec_caps_##name)); \ } while (0) +#define SEC_TLS12_CAPS_ADD(cnxk_caps, cur_pos, hw_caps, name) \ + do { \ + if ((hw_caps[CPT_ENG_TYPE_SE].name) || \ + (hw_caps[CPT_ENG_TYPE_IE].name) || \ + (hw_caps[CPT_ENG_TYPE_AE].name)) \ + sec_tls12_caps_add(cnxk_caps, cur_pos, \ + sec_tls12_caps_##name, \ + RTE_DIM(sec_tls12_caps_##name)); \ + } while (0) + static const struct rte_cryptodev_capabilities caps_mul[] = { { /* RSA */ .op = RTE_CRYPTO_OP_TYPE_ASYMMETRIC, @@ -1502,6 +1512,125 @@ static const struct rte_cryptodev_capabilities sec_ipsec_caps_null[] = { }, }; +static const struct rte_cryptodev_capabilities sec_tls12_caps_aes[] = { + { /* AES GCM */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD, + {.aead = { + .algo = RTE_CRYPTO_AEAD_AES_GCM, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 16 + }, + .digest_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .aad_size = { + .min = 13, + .max = 13, + .increment = 0 + }, + .iv_size = { + .min = 12, + .max = 12, + .increment = 0 + } + }, } + }, } + }, + { /* AES CBC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_CBC, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, +}; + +static const struct rte_cryptodev_capabilities sec_tls12_caps_des[] = { + { /* 3DES CBC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_3DES_CBC, + .block_size = 8, + .key_size = { + .min = 24, + .max = 24, + .increment = 0 + }, + .iv_size = { + .min = 8, + .max = 8, + .increment = 0 + } + }, } + }, } + }, +}; + +static const struct rte_cryptodev_capabilities sec_tls12_caps_sha1_sha2[] = { + { /* SHA1 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA1_HMAC, + .block_size = 64, + .key_size = { + .min = 20, + .max = 20, + .increment = 0 + }, + .digest_size = { + .min = 20, + .max = 20, + .increment = 0 + }, + }, } + }, } + }, + { /* SHA256 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA256_HMAC, + .block_size = 64, + .key_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .digest_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + }, } + }, } + }, +}; + static const struct rte_security_capability sec_caps_templ[] = { { /* IPsec Lookaside Protocol ESP Tunnel Ingress */ .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, @@ -1591,6 +1720,46 @@ static const struct rte_security_capability sec_caps_templ[] = { }, .crypto_capabilities = NULL, }, + { /* TLS 1.2 Record Read */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_TLS_RECORD, + .tls_record = { + .ver = RTE_SECURITY_VERSION_TLS_1_2, + .type = RTE_SECURITY_TLS_SESS_TYPE_READ, + .ar_win_size = 0, + }, + .crypto_capabilities = NULL, + }, + { /* TLS 1.2 Record Write */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_TLS_RECORD, + .tls_record = { + .ver = RTE_SECURITY_VERSION_TLS_1_2, + .type = RTE_SECURITY_TLS_SESS_TYPE_WRITE, + .ar_win_size = 0, + }, + .crypto_capabilities = NULL, + }, + { /* DTLS 1.2 Record Read */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_TLS_RECORD, + .tls_record = { + .ver = RTE_SECURITY_VERSION_DTLS_1_2, + .type = RTE_SECURITY_TLS_SESS_TYPE_READ, + .ar_win_size = 0, + }, + .crypto_capabilities = NULL, + }, + { /* DTLS 1.2 Record Write */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_TLS_RECORD, + .tls_record = { + .ver = RTE_SECURITY_VERSION_DTLS_1_2, + .type = RTE_SECURITY_TLS_SESS_TYPE_WRITE, + .ar_win_size = 0, + }, + .crypto_capabilities = NULL, + }, { .action = RTE_SECURITY_ACTION_TYPE_NONE } @@ -1807,6 +1976,35 @@ cn9k_sec_ipsec_caps_update(struct rte_security_capability *sec_cap) sec_cap->ipsec.options.esn = 1; } +static void +sec_tls12_caps_limit_check(int *cur_pos, int nb_caps) +{ + PLT_VERIFY(*cur_pos + nb_caps <= CNXK_SEC_TLS_1_2_CRYPTO_MAX_CAPS); +} + +static void +sec_tls12_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos, + const struct rte_cryptodev_capabilities *caps, int nb_caps) +{ + sec_tls12_caps_limit_check(cur_pos, nb_caps); + + memcpy(&cnxk_caps[*cur_pos], caps, nb_caps * sizeof(caps[0])); + *cur_pos += nb_caps; +} + +static void +sec_tls12_crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[], + union cpt_eng_caps *hw_caps) +{ + int cur_pos = 0; + + SEC_TLS12_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, aes); + SEC_TLS12_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, des); + SEC_TLS12_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, sha1_sha2); + + sec_tls12_caps_add(cnxk_caps, &cur_pos, caps_end, RTE_DIM(caps_end)); +} + void cnxk_cpt_caps_populate(struct cnxk_cpt_vf *vf) { @@ -1815,6 +2013,11 @@ cnxk_cpt_caps_populate(struct cnxk_cpt_vf *vf) crypto_caps_populate(vf->crypto_caps, vf->cpt.hw_caps); sec_ipsec_crypto_caps_populate(vf->sec_ipsec_crypto_caps, vf->cpt.hw_caps); + if (vf->cpt.hw_caps[CPT_ENG_TYPE_SE].tls) { + sec_tls12_crypto_caps_populate(vf->sec_tls_1_2_crypto_caps, vf->cpt.hw_caps); + sec_tls12_crypto_caps_populate(vf->sec_dtls_1_2_crypto_caps, vf->cpt.hw_caps); + } + PLT_STATIC_ASSERT(RTE_DIM(sec_caps_templ) <= RTE_DIM(vf->sec_caps)); memcpy(vf->sec_caps, sec_caps_templ, sizeof(sec_caps_templ)); @@ -1830,6 +2033,13 @@ cnxk_cpt_caps_populate(struct cnxk_cpt_vf *vf) if (roc_model_is_cn9k()) cn9k_sec_ipsec_caps_update(&vf->sec_caps[i]); + } else if (vf->sec_caps[i].protocol == RTE_SECURITY_PROTOCOL_TLS_RECORD) { + if (vf->sec_caps[i].tls_record.ver == RTE_SECURITY_VERSION_TLS_1_3) + vf->sec_caps[i].crypto_capabilities = vf->sec_tls_1_3_crypto_caps; + else if (vf->sec_caps[i].tls_record.ver == RTE_SECURITY_VERSION_DTLS_1_2) + vf->sec_caps[i].crypto_capabilities = vf->sec_dtls_1_2_crypto_caps; + else + vf->sec_caps[i].crypto_capabilities = vf->sec_tls_1_2_crypto_caps; } } } From patchwork Thu Dec 21 12:35:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135447 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9B76043750; Thu, 21 Dec 2023 13:37:52 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DB01142E7E; Thu, 21 Dec 2023 13:36:53 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 808FD40A7A for ; Thu, 21 Dec 2023 13:36:51 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS32019305 for ; Thu, 21 Dec 2023 04:36:50 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=Y/+YyxLqERrRBb1j1urb7psC80JsJt75UKuuXI6vdfU=; b=KRB TYxF9NlOyzGwo0bhI7eODC/as2/WoYsjPnGaJwqd3X2yH0+fKBeC4nbEP37rU65I fnQL0oDNNcywnYBLm551e6j5GX/2PH1irvN6PYINiJdBZkT/k20YKLoHXaQ+EAEK ipuwBt+nCU6MAv105FzzgCTkOhA9KfK1i+HkOmaGNLZDGrb5WVdj1DH5O7m62OV5 5p00c/LYrJ5yb6Aw0slWoZ0yaUeQwD/47/EkWgFTYGtfQEBqUsnWQWSiVA4oUCA0 7ByQXlfqBU+1pfnKadXQS40zjQMLssDvX3q4tSnlAGmJzI+BbgrNwsoPKwtHT0zR 0KGjvdLaEOrGBFbX8xg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0jq-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:50 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:37 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:37 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id E49B93F7079; Thu, 21 Dec 2023 04:36:34 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Jerin Jacob , Vidya Sagar Velumuri , Tejasree Kondoj , Subject: [PATCH 18/24] crypto/cnxk: add PMD APIs for raw submission to CPT Date: Thu, 21 Dec 2023 18:05:39 +0530 Message-ID: <20231221123545.510-19-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 8E5ppa8AqSI15vs3PAIv3z7dOAt64MJi X-Proofpoint-ORIG-GUID: 8E5ppa8AqSI15vs3PAIv3z7dOAt64MJi X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add PMD APIs to allow applications to directly submit CPT instructions to hardware. Signed-off-by: Anoob Joseph --- doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf.in | 1 + doc/guides/rel_notes/release_24_03.rst | 1 + drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 75 ++++++++--------- drivers/crypto/cnxk/cn10k_cryptodev_ops.h | 3 + drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 56 ------------- drivers/crypto/cnxk/cn9k_cryptodev_ops.h | 62 ++++++++++++++ drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 99 +++++++++++++++++++++++ drivers/crypto/cnxk/meson.build | 2 +- drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h | 46 +++++++++++ 10 files changed, 252 insertions(+), 94 deletions(-) create mode 100644 drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index a6a768bd7c..69f1a54511 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -49,6 +49,7 @@ The public API headers are grouped by topics: [iavf](@ref rte_pmd_iavf.h), [bnxt](@ref rte_pmd_bnxt.h), [cnxk](@ref rte_pmd_cnxk.h), + [cnxk_crypto](@ref rte_pmd_cnxk_crypto.h), [cnxk_eventdev](@ref rte_pmd_cnxk_eventdev.h), [cnxk_mempool](@ref rte_pmd_cnxk_mempool.h), [dpaa](@ref rte_pmd_dpaa.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index e94c9e4e46..6d11de580e 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -6,6 +6,7 @@ PROJECT_NUMBER = @VERSION@ USE_MDFILE_AS_MAINPAGE = @TOPDIR@/doc/api/doxy-api-index.md INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/drivers/bus/vdev \ + @TOPDIR@/drivers/crypto/cnxk \ @TOPDIR@/drivers/crypto/scheduler \ @TOPDIR@/drivers/dma/dpaa2 \ @TOPDIR@/drivers/event/dlb2 \ diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst index 0ebbae9f4e..f5773bab5a 100644 --- a/doc/guides/rel_notes/release_24_03.rst +++ b/doc/guides/rel_notes/release_24_03.rst @@ -60,6 +60,7 @@ New Features * Added support for Rx inject in crypto_cn10k. * Added support for TLS record processing in crypto_cn10k. Supports TLS 1.2 and DTLS 1.2. + * Added PMD API to allow raw submission of instructions to CPT. Removed Items ------------- diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index c87a8bae1a..c350371505 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -34,13 +34,12 @@ #include "cnxk_eventdev.h" #include "cnxk_se.h" -#define PKTS_PER_LOOP 32 -#define PKTS_PER_STEORL 16 +#include "rte_pmd_cnxk_crypto.h" /* Holds information required to send crypto operations in one burst */ struct ops_burst { - struct rte_crypto_op *op[PKTS_PER_LOOP]; - uint64_t w2[PKTS_PER_LOOP]; + struct rte_crypto_op *op[CN10K_PKTS_PER_LOOP]; + uint64_t w2[CN10K_PKTS_PER_LOOP]; struct cn10k_sso_hws *ws; struct cnxk_cpt_qp *qp; uint16_t nb_ops; @@ -252,7 +251,7 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops, goto pend_q_commit; } - for (i = 0; i < RTE_MIN(PKTS_PER_LOOP, nb_ops); i++) { + for (i = 0; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_ops); i++) { infl_req = &pend_q->req_queue[head]; infl_req->op_flags = 0; @@ -267,23 +266,21 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops, pending_queue_advance(&head, pq_mask); } - if (i > PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (PKTS_PER_STEORL - 1) << 12 | + if (i > CN10K_PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | (uint64_t)lmt_id; roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | - (i - PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + PKTS_PER_STEORL); + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); roc_lmt_submit_steorl(lmt_arg, io_addr); } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | - (uint64_t)lmt_id; + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; roc_lmt_submit_steorl(lmt_arg, io_addr); } rte_io_wmb(); - if (nb_ops - i > 0 && i == PKTS_PER_LOOP) { + if (nb_ops - i > 0 && i == CN10K_PKTS_PER_LOOP) { nb_ops -= i; ops += i; count += i; @@ -487,7 +484,7 @@ cn10k_cpt_vec_submit(struct vec_request vec_tbl[], uint16_t vec_tbl_len, struct inst = (struct cpt_inst_s *)lmt_base; again: - burst_size = RTE_MIN(PKTS_PER_STEORL, vec_tbl_len); + burst_size = RTE_MIN(CN10K_PKTS_PER_STEORL, vec_tbl_len); for (i = 0; i < burst_size; i++) cn10k_cpt_vec_inst_fill(&vec_tbl[i], &inst[i * 2], qp, vec_tbl[0].w7); @@ -516,7 +513,7 @@ static inline int ca_lmtst_vec_submit(struct ops_burst *burst, struct vec_request vec_tbl[], uint16_t *vec_tbl_len, const bool is_sg_ver2) { - struct cpt_inflight_req *infl_reqs[PKTS_PER_LOOP]; + struct cpt_inflight_req *infl_reqs[CN10K_PKTS_PER_LOOP]; uint64_t lmt_base, lmt_arg, io_addr; uint16_t lmt_id, len = *vec_tbl_len; struct cpt_inst_s *inst, *inst_base; @@ -618,11 +615,12 @@ next_op:; if (CNXK_TT_FROM_TAG(burst->ws->gw_rdata) == SSO_TT_ORDERED) roc_sso_hws_head_wait(burst->ws->base); - if (i > PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (PKTS_PER_STEORL - 1) << 12 | (uint64_t)lmt_id; + if (i > CN10K_PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)lmt_id; roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + PKTS_PER_STEORL); + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); roc_lmt_submit_steorl(lmt_arg, io_addr); } else { lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; @@ -647,7 +645,7 @@ next_op:; static inline uint16_t ca_lmtst_burst_submit(struct ops_burst *burst, const bool is_sg_ver2) { - struct cpt_inflight_req *infl_reqs[PKTS_PER_LOOP]; + struct cpt_inflight_req *infl_reqs[CN10K_PKTS_PER_LOOP]; uint64_t lmt_base, lmt_arg, io_addr; struct cpt_inst_s *inst, *inst_base; struct cpt_inflight_req *infl_req; @@ -718,11 +716,12 @@ ca_lmtst_burst_submit(struct ops_burst *burst, const bool is_sg_ver2) if (CNXK_TT_FROM_TAG(burst->ws->gw_rdata) == SSO_TT_ORDERED) roc_sso_hws_head_wait(burst->ws->base); - if (i > PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (PKTS_PER_STEORL - 1) << 12 | (uint64_t)lmt_id; + if (i > CN10K_PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)lmt_id; roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + PKTS_PER_STEORL); + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); roc_lmt_submit_steorl(lmt_arg, io_addr); } else { lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; @@ -791,7 +790,7 @@ cn10k_cpt_crypto_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_ev burst.op[burst.nb_ops] = op; /* Max nb_ops per burst check */ - if (++burst.nb_ops == PKTS_PER_LOOP) { + if (++burst.nb_ops == CN10K_PKTS_PER_LOOP) { if (is_vector) submitted = ca_lmtst_vec_submit(&burst, vec_tbl, &vec_tbl_len, is_sg_ver2); @@ -1146,7 +1145,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, again: inst = (struct cpt_inst_s *)lmt_base; - for (i = 0; i < RTE_MIN(PKTS_PER_LOOP, nb_pkts); i++) { + for (i = 0; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_pkts); i++) { m = pkts[i]; sec_sess = (struct cn10k_sec_session *)sess[i]; @@ -1192,11 +1191,12 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, inst += 2; } - if (i > PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (PKTS_PER_STEORL - 1) << 12 | (uint64_t)lmt_id; + if (i > CN10K_PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)lmt_id; roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + PKTS_PER_STEORL); + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); roc_lmt_submit_steorl(lmt_arg, io_addr); } else { lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; @@ -1205,7 +1205,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, rte_io_wmb(); - if (nb_pkts - i > 0 && i == PKTS_PER_LOOP) { + if (nb_pkts - i > 0 && i == CN10K_PKTS_PER_LOOP) { nb_pkts -= i; pkts += i; count += i; @@ -1332,7 +1332,7 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym goto pend_q_commit; } - for (i = 0; i < RTE_MIN(PKTS_PER_LOOP, nb_ops); i++) { + for (i = 0; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_ops); i++) { struct cnxk_iov iov; index = count + i; @@ -1354,11 +1354,12 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym pending_queue_advance(&head, pq_mask); } - if (i > PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (PKTS_PER_STEORL - 1) << 12 | (uint64_t)lmt_id; + if (i > CN10K_PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)lmt_id; roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + PKTS_PER_STEORL); + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); roc_lmt_submit_steorl(lmt_arg, io_addr); } else { lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; @@ -1367,7 +1368,7 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym rte_io_wmb(); - if (nb_ops - i > 0 && i == PKTS_PER_LOOP) { + if (nb_ops - i > 0 && i == CN10K_PKTS_PER_LOOP) { nb_ops -= i; count += i; goto again; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h index 34becede3c..406c4abc7f 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h @@ -12,6 +12,9 @@ #include "cnxk_cryptodev.h" +#define CN10K_PKTS_PER_LOOP 32 +#define CN10K_PKTS_PER_STEORL 16 + extern struct rte_cryptodev_ops cn10k_cpt_ops; void cn10k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev, struct cnxk_cpt_vf *vf); diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index 442cd8e5a9..ac9393eacf 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -122,62 +122,6 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, return ret; } -static inline void -cn9k_cpt_inst_submit(struct cpt_inst_s *inst, uint64_t lmtline, - uint64_t io_addr) -{ - uint64_t lmt_status; - - do { - /* Copy CPT command to LMTLINE */ - roc_lmt_mov64((void *)lmtline, inst); - - /* - * Make sure compiler does not reorder memcpy and ldeor. - * LMTST transactions are always flushed from the write - * buffer immediately, a DMB is not required to push out - * LMTSTs. - */ - rte_io_wmb(); - lmt_status = roc_lmt_submit_ldeor(io_addr); - } while (lmt_status == 0); -} - -static __plt_always_inline void -cn9k_cpt_inst_submit_dual(struct cpt_inst_s *inst, uint64_t lmtline, - uint64_t io_addr) -{ - uint64_t lmt_status; - - do { - /* Copy 2 CPT inst_s to LMTLINE */ -#if defined(RTE_ARCH_ARM64) - uint64_t *s = (uint64_t *)inst; - uint64_t *d = (uint64_t *)lmtline; - - vst1q_u64(&d[0], vld1q_u64(&s[0])); - vst1q_u64(&d[2], vld1q_u64(&s[2])); - vst1q_u64(&d[4], vld1q_u64(&s[4])); - vst1q_u64(&d[6], vld1q_u64(&s[6])); - vst1q_u64(&d[8], vld1q_u64(&s[8])); - vst1q_u64(&d[10], vld1q_u64(&s[10])); - vst1q_u64(&d[12], vld1q_u64(&s[12])); - vst1q_u64(&d[14], vld1q_u64(&s[14])); -#else - roc_lmt_mov_seg((void *)lmtline, inst, 8); -#endif - - /* - * Make sure compiler does not reorder memcpy and ldeor. - * LMTST transactions are always flushed from the write - * buffer immediately, a DMB is not required to push out - * LMTSTs. - */ - rte_io_wmb(); - lmt_status = roc_lmt_submit_ldeor(io_addr); - } while (lmt_status == 0); -} - static uint16_t cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) { diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.h b/drivers/crypto/cnxk/cn9k_cryptodev_ops.h index c6ec96153e..3d667094f3 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.h @@ -8,8 +8,70 @@ #include #include +#include + +#if defined(__aarch64__) +#include "roc_io.h" +#else +#include "roc_io_generic.h" +#endif + extern struct rte_cryptodev_ops cn9k_cpt_ops; +static inline void +cn9k_cpt_inst_submit(struct cpt_inst_s *inst, uint64_t lmtline, uint64_t io_addr) +{ + uint64_t lmt_status; + + do { + /* Copy CPT command to LMTLINE */ + roc_lmt_mov64((void *)lmtline, inst); + + /* + * Make sure compiler does not reorder memcpy and ldeor. + * LMTST transactions are always flushed from the write + * buffer immediately, a DMB is not required to push out + * LMTSTs. + */ + rte_io_wmb(); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __plt_always_inline void +cn9k_cpt_inst_submit_dual(struct cpt_inst_s *inst, uint64_t lmtline, uint64_t io_addr) +{ + uint64_t lmt_status; + + do { + /* Copy 2 CPT inst_s to LMTLINE */ +#if defined(RTE_ARCH_ARM64) + volatile const __uint128_t *src128 = (const __uint128_t *)inst; + volatile __uint128_t *dst128 = (__uint128_t *)lmtline; + + dst128[0] = src128[0]; + dst128[1] = src128[1]; + dst128[2] = src128[2]; + dst128[3] = src128[3]; + dst128[4] = src128[4]; + dst128[5] = src128[5]; + dst128[6] = src128[6]; + dst128[7] = src128[7]; +#else + roc_lmt_mov_seg((void *)lmtline, inst, 8); +#endif + + /* + * Make sure compiler does not reorder memcpy and ldeor. + * LMTST transactions are always flushed from the write + * buffer immediately, a DMB is not required to push out + * LMTSTs. + */ + rte_io_wmb(); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + void cn9k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev); __rte_internal diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index fd44155955..7a37e3e89c 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -12,6 +12,11 @@ #include "roc_errata.h" #include "roc_idev.h" #include "roc_ie_on.h" +#if defined(__aarch64__) +#include "roc_io.h" +#else +#include "roc_io_generic.h" +#endif #include "cnxk_ae.h" #include "cnxk_cryptodev.h" @@ -19,6 +24,11 @@ #include "cnxk_cryptodev_ops.h" #include "cnxk_se.h" +#include "cn10k_cryptodev_ops.h" +#include "cn9k_cryptodev_ops.h" + +#include "rte_pmd_cnxk_crypto.h" + #define CNXK_CPT_MAX_ASYM_OP_NUM_PARAMS 5 #define CNXK_CPT_MAX_ASYM_OP_MOD_LEN 1024 #define CNXK_CPT_META_BUF_MAX_CACHE_SIZE 128 @@ -918,3 +928,92 @@ cnxk_cpt_queue_pair_event_error_query(struct rte_cryptodev *dev, uint16_t qp_id) } return 0; } + +void * +rte_pmd_cnxk_crypto_qptr_get(uint8_t dev_id, uint16_t qp_id) +{ + const struct rte_crypto_fp_ops *fp_ops; + void *qptr; + + fp_ops = &rte_crypto_fp_ops[dev_id]; + qptr = fp_ops->qp.data[qp_id]; + + return qptr; +} + +static inline void +cnxk_crypto_cn10k_submit(void *qptr, void *inst, uint16_t nb_inst) +{ + uint64_t lmt_base, lmt_arg, io_addr; + struct cnxk_cpt_qp *qp = qptr; + uint16_t i, j, lmt_id; + void *lmt_dst; + + lmt_base = qp->lmtline.lmt_base; + io_addr = qp->lmtline.io_addr; + + ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); + +again: + i = RTE_MIN(nb_inst, CN10K_PKTS_PER_LOOP); + lmt_dst = PLT_PTR_CAST(lmt_base); + + for (j = 0; j < i; j++) { + rte_memcpy(lmt_dst, inst, sizeof(struct cpt_inst_s)); + inst = RTE_PTR_ADD(inst, sizeof(struct cpt_inst_s)); + lmt_dst = RTE_PTR_ADD(lmt_dst, 2 * sizeof(struct cpt_inst_s)); + } + + rte_io_wmb(); + + if (i > CN10K_PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, io_addr); + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); + roc_lmt_submit_steorl(lmt_arg, io_addr); + } else { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, io_addr); + } + + rte_io_wmb(); + + if (nb_inst - i > 0) { + nb_inst -= i; + goto again; + } +} + +static inline void +cnxk_crypto_cn9k_submit(void *qptr, void *inst, uint16_t nb_inst) +{ + struct cnxk_cpt_qp *qp = qptr; + + const uint64_t lmt_base = qp->lf.lmt_base; + const uint64_t io_addr = qp->lf.io_addr; + + if (unlikely(nb_inst & 1)) { + cn9k_cpt_inst_submit(inst, lmt_base, io_addr); + inst = RTE_PTR_ADD(inst, sizeof(struct cpt_inst_s)); + nb_inst -= 1; + } + + while (nb_inst > 0) { + cn9k_cpt_inst_submit_dual(inst, lmt_base, io_addr); + inst = RTE_PTR_ADD(inst, 2 * sizeof(struct cpt_inst_s)); + nb_inst -= 2; + } +} + +void +rte_pmd_cnxk_crypto_submit(void *qptr, void *inst, uint16_t nb_inst) +{ + if (roc_model_is_cn10k()) + return cnxk_crypto_cn10k_submit(qptr, inst, nb_inst); + else if (roc_model_is_cn9k()) + return cnxk_crypto_cn9k_submit(qptr, inst, nb_inst); + + plt_err("Invalid cnxk model"); +} diff --git a/drivers/crypto/cnxk/meson.build b/drivers/crypto/cnxk/meson.build index ee0c65e32a..aa840fb7bb 100644 --- a/drivers/crypto/cnxk/meson.build +++ b/drivers/crypto/cnxk/meson.build @@ -24,8 +24,8 @@ sources = files( 'cnxk_cryptodev_sec.c', ) +headers = files('rte_pmd_cnxk_crypto.h') deps += ['bus_pci', 'common_cnxk', 'security', 'eventdev'] - includes += include_directories('../../../lib/net', '../../event/cnxk') if get_option('buildtype').contains('debug') diff --git a/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h b/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h new file mode 100644 index 0000000000..64978a008b --- /dev/null +++ b/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +/** + * @file rte_pmd_cnxk_crypto.h + * Marvell CNXK Crypto PMD specific functions. + * + **/ + +#ifndef _PMD_CNXK_CRYPTO_H_ +#define _PMD_CNXK_CRYPTO_H_ + +#include + +/** + * Get queue pointer of a specific queue in a cryptodev. + * + * @param dev_id + * Device identifier of cryptodev device. + * @param qp_id + * Index of the queue pair. + * @return + * Pointer to queue pair structure that would be the input to submit APIs. + */ +void *rte_pmd_cnxk_crypto_qptr_get(uint8_t dev_id, uint16_t qp_id); + +/** + * Submit CPT instruction (cpt_inst_s) to hardware (CPT). + * + * The ``qp`` is a pointer obtained from ``rte_pmd_cnxk_crypto_qp_get``. Application should make + * sure it doesn't overflow the internal hardware queues. It may do so by making sure the inflight + * packets are not more than the number of descriptors configured. + * + * This API may be called only after the cryptodev and queue pair is configured and is started. + * + * @param qptr + * Pointer obtained with ``rte_pmd_cnxk_crypto_qptr_get``. + * @param inst + * Pointer to an array of instructions prepared by application. + * @param nb_inst + * Number of instructions. + */ +void rte_pmd_cnxk_crypto_submit(void *qptr, void *inst, uint16_t nb_inst); + +#endif /* _PMD_CNXK_CRYPTO_H_ */ From patchwork Thu Dec 21 12:35:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135448 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1599343750; Thu, 21 Dec 2023 13:38:01 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5D04542ECC; Thu, 21 Dec 2023 13:36:55 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 068C640A81 for ; Thu, 21 Dec 2023 13:36:51 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS33019305 for ; Thu, 21 Dec 2023 04:36:51 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=aSvQTe7WBotnt/6UMdsXoi/VeGt3AGV8Tm7AVNbZOBI=; b=Nvo EABwpyKMUTUr6n0VkXMuhSnL9eD2C/5myHbCN/BGjS0iLIQGKCjgmc/t2mzrd7iV nq1J6MvhrKxQtrs+mSbdozMm9USphO5y8fHINBn4JjvinDFwGd6uvzB7ad/0mhtR zMYBKKqB1s/saGfmw8ymHWzFUAp3DjQRhDo7LecckaiJKU3K7ZaXatdwQ3EVdA5o QS7v/QwbEj0Gm1wz+sISltVTfsl1akQLdv9kAD7TPXIu57Sg92nRCKsUyXeMHr9t co6FRdI2FtgkQd9m0pLU/vOkAf29SxlAp5xtBu7x5r6Ruzit+03kLuA12S4C/1/D UFx0sfYySUgaPEWxrUw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0jq-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:50 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:40 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:40 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 9214E3F7073; Thu, 21 Dec 2023 04:36:37 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Tejasree Kondoj , Jerin Jacob , Vidya Sagar Velumuri , Subject: [PATCH 19/24] crypto/cnxk: replace PDCP with PDCP chain opcode Date: Thu, 21 Dec 2023 18:05:40 +0530 Message-ID: <20231221123545.510-20-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: yeqoZOeDtkbW3YFvi-CRTMawQzhIFlds X-Proofpoint-ORIG-GUID: yeqoZOeDtkbW3YFvi-CRTMawQzhIFlds X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Tejasree Kondoj Replacing PDCP opcode with PDCP chain opcode. Signed-off-by: Tejasree Kondoj --- drivers/common/cnxk/roc_se.c | 331 +++++++++------------------------- drivers/common/cnxk/roc_se.h | 18 +- drivers/crypto/cnxk/cnxk_se.h | 96 +++++----- 3 files changed, 135 insertions(+), 310 deletions(-) diff --git a/drivers/common/cnxk/roc_se.c b/drivers/common/cnxk/roc_se.c index 6ced4ef789..4e00268149 100644 --- a/drivers/common/cnxk/roc_se.c +++ b/drivers/common/cnxk/roc_se.c @@ -88,13 +88,20 @@ cpt_ciph_type_set(roc_se_cipher_type type, struct roc_se_ctx *ctx, uint16_t key_ fc_type = ROC_SE_FC_GEN; break; case ROC_SE_ZUC_EEA3: - if (chained_op) { - if (unlikely(key_len != 16)) + if (unlikely(key_len != 16)) { + /* + * ZUC 256 is not supported with older microcode + * where pdcp_iv_offset is 16 + */ + if (chained_op || (ctx->pdcp_iv_offset == 16)) { + plt_err("ZUC 256 is not supported with chained operations"); return -1; + } + } + if (chained_op) fc_type = ROC_SE_PDCP_CHAIN; - } else { + else fc_type = ROC_SE_PDCP; - } break; case ROC_SE_SNOW3G_UEA2: if (unlikely(key_len != 16)) @@ -197,33 +204,6 @@ cpt_hmac_opad_ipad_gen(roc_se_auth_type auth_type, const uint8_t *key, uint16_t } } -static int -cpt_pdcp_key_type_set(struct roc_se_zuc_snow3g_ctx *zs_ctx, uint16_t key_len) -{ - roc_se_aes_type key_type = 0; - - if (roc_model_is_cn9k()) { - if (key_len != 16) { - plt_err("Only key len 16 is supported on cn9k"); - return -ENOTSUP; - } - } - - switch (key_len) { - case 16: - key_type = ROC_SE_AES_128_BIT; - break; - case 32: - key_type = ROC_SE_AES_256_BIT; - break; - default: - plt_err("Invalid AES key len"); - return -ENOTSUP; - } - zs_ctx->zuc.otk_ctx.w0.s.key_len = key_type; - return 0; -} - static int cpt_pdcp_chain_key_type_get(uint16_t key_len) { @@ -247,36 +227,6 @@ cpt_pdcp_chain_key_type_get(uint16_t key_len) return key_type; } -static int -cpt_pdcp_mac_len_set(struct roc_se_zuc_snow3g_ctx *zs_ctx, uint16_t mac_len) -{ - roc_se_pdcp_mac_len_type mac_type = 0; - - if (roc_model_is_cn9k()) { - if (mac_len != 4) { - plt_err("Only mac len 4 is supported on cn9k"); - return -ENOTSUP; - } - } - - switch (mac_len) { - case 4: - mac_type = ROC_SE_PDCP_MAC_LEN_32_BIT; - break; - case 8: - mac_type = ROC_SE_PDCP_MAC_LEN_64_BIT; - break; - case 16: - mac_type = ROC_SE_PDCP_MAC_LEN_128_BIT; - break; - default: - plt_err("Invalid ZUC MAC len"); - return -ENOTSUP; - } - zs_ctx->zuc.otk_ctx.w0.s.mac_len = mac_type; - return 0; -} - static void cpt_zuc_const_update(uint8_t *zuc_const, int key_len, int mac_len) { @@ -300,32 +250,27 @@ cpt_zuc_const_update(uint8_t *zuc_const, int key_len, int mac_len) } int -roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, - const uint8_t *key, uint16_t key_len, uint16_t mac_len) +roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, const uint8_t *key, + uint16_t key_len, uint16_t mac_len) { - struct roc_se_zuc_snow3g_chain_ctx *zs_ch_ctx; - struct roc_se_zuc_snow3g_ctx *zs_ctx; struct roc_se_kasumi_ctx *k_ctx; + struct roc_se_pdcp_ctx *pctx; struct roc_se_context *fctx; uint8_t opcode_minor; - uint8_t pdcp_alg; bool chained_op; - int ret; if (se_ctx == NULL) return -1; - zs_ctx = &se_ctx->se_ctx.zs_ctx; - zs_ch_ctx = &se_ctx->se_ctx.zs_ch_ctx; + pctx = &se_ctx->se_ctx.pctx; k_ctx = &se_ctx->se_ctx.k_ctx; fctx = &se_ctx->se_ctx.fctx; chained_op = se_ctx->ciph_then_auth || se_ctx->auth_then_ciph; if ((type >= ROC_SE_ZUC_EIA3) && (type <= ROC_SE_KASUMI_F9_ECB)) { - uint8_t *zuc_const; uint32_t keyx[4]; - uint8_t *ci_key; + int key_type; if (!key_len) return -1; @@ -335,98 +280,64 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, return -1; } - if (roc_model_is_cn9k()) { - ci_key = zs_ctx->zuc.onk_ctx.ci_key; - zuc_const = zs_ctx->zuc.onk_ctx.zuc_const; - } else { - ci_key = zs_ctx->zuc.otk_ctx.ci_key; - zuc_const = zs_ctx->zuc.otk_ctx.zuc_const; - } - /* For ZUC/SNOW3G/Kasumi */ switch (type) { case ROC_SE_SNOW3G_UIA2: - if (chained_op) { - struct roc_se_onk_zuc_chain_ctx *ctx = - &zs_ch_ctx->zuc.onk_ctx; - zs_ch_ctx->zuc.onk_ctx.w0.s.state_conf = - ROC_SE_PDCP_CHAIN_CTX_KEY_IV; - ctx->w0.s.auth_type = - ROC_SE_PDCP_CHAIN_ALG_TYPE_SNOW3G; - ctx->w0.s.mac_len = mac_len; - ctx->w0.s.auth_key_len = key_len; - se_ctx->fc_type = ROC_SE_PDCP_CHAIN; - cpt_snow3g_key_gen(key, keyx); - memcpy(ctx->st.auth_key, keyx, key_len); - } else { - zs_ctx->zuc.otk_ctx.w0.s.alg_type = - ROC_SE_PDCP_ALG_TYPE_SNOW3G; - zs_ctx->zuc.otk_ctx.w0.s.mac_len = - ROC_SE_PDCP_MAC_LEN_32_BIT; - cpt_snow3g_key_gen(key, keyx); - memcpy(ci_key, keyx, key_len); + pctx->w0.s.state_conf = ROC_SE_PDCP_CHAIN_CTX_KEY_IV; + pctx->w0.s.auth_type = ROC_SE_PDCP_CHAIN_ALG_TYPE_SNOW3G; + pctx->w0.s.mac_len = mac_len; + pctx->w0.s.auth_key_len = key_len; + se_ctx->fc_type = ROC_SE_PDCP_CHAIN; + cpt_snow3g_key_gen(key, keyx); + memcpy(pctx->st.auth_key, keyx, key_len); + + if (!chained_op) se_ctx->fc_type = ROC_SE_PDCP; - } se_ctx->pdcp_auth_alg = ROC_SE_PDCP_ALG_TYPE_SNOW3G; se_ctx->zsk_flags = 0x1; break; case ROC_SE_ZUC_EIA3: - if (chained_op) { - struct roc_se_onk_zuc_chain_ctx *ctx = - &zs_ch_ctx->zuc.onk_ctx; - ctx->w0.s.state_conf = - ROC_SE_PDCP_CHAIN_CTX_KEY_IV; - ctx->w0.s.auth_type = - ROC_SE_PDCP_CHAIN_ALG_TYPE_ZUC; - ctx->w0.s.mac_len = mac_len; - ctx->w0.s.auth_key_len = key_len; - memcpy(ctx->st.auth_key, key, key_len); - cpt_zuc_const_update(ctx->st.auth_zuc_const, - key_len, mac_len); - se_ctx->fc_type = ROC_SE_PDCP_CHAIN; - } else { - zs_ctx->zuc.otk_ctx.w0.s.alg_type = - ROC_SE_PDCP_ALG_TYPE_ZUC; - ret = cpt_pdcp_key_type_set(zs_ctx, key_len); - if (ret) - return ret; - ret = cpt_pdcp_mac_len_set(zs_ctx, mac_len); - if (ret) - return ret; - memcpy(ci_key, key, key_len); - if (key_len == 32) - roc_se_zuc_bytes_swap(ci_key, key_len); - cpt_zuc_const_update(zuc_const, key_len, - mac_len); - se_ctx->fc_type = ROC_SE_PDCP; + if (unlikely(key_len != 16)) { + /* + * ZUC 256 is not supported with older microcode + * where pdcp_iv_offset is 16 + */ + if (chained_op || (se_ctx->pdcp_iv_offset == 16)) { + plt_err("ZUC 256 is not supported with chained operations"); + return -1; + } } + key_type = cpt_pdcp_chain_key_type_get(key_len); + if (key_type < 0) + return key_type; + pctx->w0.s.auth_key_len = key_type; + pctx->w0.s.state_conf = ROC_SE_PDCP_CHAIN_CTX_KEY_IV; + pctx->w0.s.auth_type = ROC_SE_PDCP_CHAIN_ALG_TYPE_ZUC; + pctx->w0.s.mac_len = mac_len; + memcpy(pctx->st.auth_key, key, key_len); + if (key_len == 32) + roc_se_zuc_bytes_swap(pctx->st.auth_key, key_len); + cpt_zuc_const_update(pctx->st.auth_zuc_const, key_len, mac_len); + se_ctx->fc_type = ROC_SE_PDCP_CHAIN; + + if (!chained_op) + se_ctx->fc_type = ROC_SE_PDCP; se_ctx->pdcp_auth_alg = ROC_SE_PDCP_ALG_TYPE_ZUC; se_ctx->zsk_flags = 0x1; break; case ROC_SE_AES_CMAC_EIA2: - if (chained_op) { - struct roc_se_onk_zuc_chain_ctx *ctx = - &zs_ch_ctx->zuc.onk_ctx; - int key_type; - key_type = cpt_pdcp_chain_key_type_get(key_len); - if (key_type < 0) - return key_type; - ctx->w0.s.auth_key_len = key_type; - ctx->w0.s.state_conf = - ROC_SE_PDCP_CHAIN_CTX_KEY_IV; - ctx->w0.s.auth_type = - ROC_SE_PDCP_ALG_TYPE_AES_CTR; - ctx->w0.s.mac_len = mac_len; - memcpy(ctx->st.auth_key, key, key_len); - se_ctx->fc_type = ROC_SE_PDCP_CHAIN; - } else { - zs_ctx->zuc.otk_ctx.w0.s.alg_type = - ROC_SE_PDCP_ALG_TYPE_AES_CTR; - zs_ctx->zuc.otk_ctx.w0.s.mac_len = - ROC_SE_PDCP_MAC_LEN_32_BIT; - memcpy(ci_key, key, key_len); + key_type = cpt_pdcp_chain_key_type_get(key_len); + if (key_type < 0) + return key_type; + pctx->w0.s.auth_key_len = key_type; + pctx->w0.s.state_conf = ROC_SE_PDCP_CHAIN_CTX_KEY_IV; + pctx->w0.s.auth_type = ROC_SE_PDCP_ALG_TYPE_AES_CTR; + pctx->w0.s.mac_len = mac_len; + memcpy(pctx->st.auth_key, key, key_len); + se_ctx->fc_type = ROC_SE_PDCP_CHAIN; + + if (!chained_op) se_ctx->fc_type = ROC_SE_PDCP; - } se_ctx->pdcp_auth_alg = ROC_SE_PDCP_ALG_TYPE_AES_CMAC; se_ctx->eia2 = 1; se_ctx->zsk_flags = 0x1; @@ -454,11 +365,8 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, se_ctx->mac_len = mac_len; se_ctx->hash_type = type; - pdcp_alg = zs_ctx->zuc.otk_ctx.w0.s.alg_type; if (chained_op) opcode_minor = se_ctx->ciph_then_auth ? 2 : 3; - else if (roc_model_is_cn9k()) - opcode_minor = ((1 << 7) | (pdcp_alg << 5) | 1); else opcode_minor = ((1 << 4) | 1); @@ -513,29 +421,18 @@ int roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, const uint8_t *key, uint16_t key_len) { - bool chained_op = se_ctx->ciph_then_auth || se_ctx->auth_then_ciph; - struct roc_se_zuc_snow3g_ctx *zs_ctx = &se_ctx->se_ctx.zs_ctx; struct roc_se_context *fctx = &se_ctx->se_ctx.fctx; - struct roc_se_zuc_snow3g_chain_ctx *zs_ch_ctx; + struct roc_se_pdcp_ctx *pctx; uint8_t opcode_minor = 0; - uint8_t *zuc_const; uint32_t keyx[4]; - uint8_t *ci_key; + int key_type; int i, ret; /* For NULL cipher, no processing required. */ if (type == ROC_SE_PASSTHROUGH) return 0; - zs_ch_ctx = &se_ctx->se_ctx.zs_ch_ctx; - - if (roc_model_is_cn9k()) { - ci_key = zs_ctx->zuc.onk_ctx.ci_key; - zuc_const = zs_ctx->zuc.onk_ctx.zuc_const; - } else { - ci_key = zs_ctx->zuc.otk_ctx.ci_key; - zuc_const = zs_ctx->zuc.otk_ctx.zuc_const; - } + pctx = &se_ctx->se_ctx.pctx; if ((type == ROC_SE_AES_GCM) || (type == ROC_SE_AES_CCM)) se_ctx->template_w4.s.opcode_minor = BIT(5); @@ -615,72 +512,38 @@ roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, const ui fctx->enc.enc_cipher = ROC_SE_DES3_CBC; goto success; case ROC_SE_SNOW3G_UEA2: - if (chained_op == true) { - struct roc_se_onk_zuc_chain_ctx *ctx = - &zs_ch_ctx->zuc.onk_ctx; - zs_ch_ctx->zuc.onk_ctx.w0.s.state_conf = - ROC_SE_PDCP_CHAIN_CTX_KEY_IV; - zs_ch_ctx->zuc.onk_ctx.w0.s.cipher_type = - ROC_SE_PDCP_CHAIN_ALG_TYPE_SNOW3G; - zs_ch_ctx->zuc.onk_ctx.w0.s.ci_key_len = key_len; - cpt_snow3g_key_gen(key, keyx); - memcpy(ctx->st.ci_key, keyx, key_len); - } else { - zs_ctx->zuc.otk_ctx.w0.s.key_len = ROC_SE_AES_128_BIT; - zs_ctx->zuc.otk_ctx.w0.s.alg_type = - ROC_SE_PDCP_ALG_TYPE_SNOW3G; - cpt_snow3g_key_gen(key, keyx); - memcpy(ci_key, keyx, key_len); - } + pctx->w0.s.state_conf = ROC_SE_PDCP_CHAIN_CTX_KEY_IV; + pctx->w0.s.cipher_type = ROC_SE_PDCP_CHAIN_ALG_TYPE_SNOW3G; + pctx->w0.s.ci_key_len = key_len; + cpt_snow3g_key_gen(key, keyx); + memcpy(pctx->st.ci_key, keyx, key_len); se_ctx->pdcp_ci_alg = ROC_SE_PDCP_ALG_TYPE_SNOW3G; se_ctx->zsk_flags = 0; goto success; case ROC_SE_ZUC_EEA3: - if (chained_op == true) { - struct roc_se_onk_zuc_chain_ctx *ctx = - &zs_ch_ctx->zuc.onk_ctx; - zs_ch_ctx->zuc.onk_ctx.w0.s.state_conf = - ROC_SE_PDCP_CHAIN_CTX_KEY_IV; - zs_ch_ctx->zuc.onk_ctx.w0.s.cipher_type = - ROC_SE_PDCP_CHAIN_ALG_TYPE_ZUC; - memcpy(ctx->st.ci_key, key, key_len); - memcpy(ctx->st.ci_zuc_const, zuc_key128, 32); - zs_ch_ctx->zuc.onk_ctx.w0.s.ci_key_len = key_len; - } else { - ret = cpt_pdcp_key_type_set(zs_ctx, key_len); - if (ret) - return ret; - zs_ctx->zuc.otk_ctx.w0.s.alg_type = - ROC_SE_PDCP_ALG_TYPE_ZUC; - memcpy(ci_key, key, key_len); - if (key_len == 32) { - roc_se_zuc_bytes_swap(ci_key, key_len); - memcpy(zuc_const, zuc_key256, 16); - } else - memcpy(zuc_const, zuc_key128, 32); - } - + key_type = cpt_pdcp_chain_key_type_get(key_len); + if (key_type < 0) + return key_type; + pctx->w0.s.ci_key_len = key_type; + pctx->w0.s.state_conf = ROC_SE_PDCP_CHAIN_CTX_KEY_IV; + pctx->w0.s.cipher_type = ROC_SE_PDCP_CHAIN_ALG_TYPE_ZUC; + memcpy(pctx->st.ci_key, key, key_len); + if (key_len == 32) { + roc_se_zuc_bytes_swap(pctx->st.ci_key, key_len); + memcpy(pctx->st.ci_zuc_const, zuc_key256, 16); + } else + memcpy(pctx->st.ci_zuc_const, zuc_key128, 32); se_ctx->pdcp_ci_alg = ROC_SE_PDCP_ALG_TYPE_ZUC; se_ctx->zsk_flags = 0; goto success; case ROC_SE_AES_CTR_EEA2: - if (chained_op == true) { - struct roc_se_onk_zuc_chain_ctx *ctx = - &zs_ch_ctx->zuc.onk_ctx; - int key_type; - key_type = cpt_pdcp_chain_key_type_get(key_len); - if (key_type < 0) - return key_type; - ctx->w0.s.ci_key_len = key_type; - ctx->w0.s.state_conf = ROC_SE_PDCP_CHAIN_CTX_KEY_IV; - ctx->w0.s.cipher_type = ROC_SE_PDCP_ALG_TYPE_AES_CTR; - memcpy(ctx->st.ci_key, key, key_len); - } else { - zs_ctx->zuc.otk_ctx.w0.s.key_len = ROC_SE_AES_128_BIT; - zs_ctx->zuc.otk_ctx.w0.s.alg_type = - ROC_SE_PDCP_ALG_TYPE_AES_CTR; - memcpy(ci_key, key, key_len); - } + key_type = cpt_pdcp_chain_key_type_get(key_len); + if (key_type < 0) + return key_type; + pctx->w0.s.ci_key_len = key_type; + pctx->w0.s.state_conf = ROC_SE_PDCP_CHAIN_CTX_KEY_IV; + pctx->w0.s.cipher_type = ROC_SE_PDCP_ALG_TYPE_AES_CTR; + memcpy(pctx->st.ci_key, key, key_len); se_ctx->pdcp_ci_alg = ROC_SE_PDCP_ALG_TYPE_AES_CTR; se_ctx->zsk_flags = 0; goto success; @@ -720,20 +583,6 @@ roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, const ui return 0; } -void -roc_se_ctx_swap(struct roc_se_ctx *se_ctx) -{ - struct roc_se_zuc_snow3g_ctx *zs_ctx = &se_ctx->se_ctx.zs_ctx; - - if (roc_model_is_cn9k()) - return; - - if (se_ctx->fc_type == ROC_SE_PDCP_CHAIN) - return; - - zs_ctx->zuc.otk_ctx.w0.u64 = htobe64(zs_ctx->zuc.otk_ctx.w0.u64); -} - void roc_se_ctx_init(struct roc_se_ctx *roc_se_ctx) { @@ -745,15 +594,13 @@ roc_se_ctx_init(struct roc_se_ctx *roc_se_ctx) case ROC_SE_FC_GEN: ctx_len = sizeof(struct roc_se_context); break; + case ROC_SE_PDCP_CHAIN: case ROC_SE_PDCP: - ctx_len = sizeof(struct roc_se_zuc_snow3g_ctx); + ctx_len = sizeof(struct roc_se_pdcp_ctx); break; case ROC_SE_KASUMI: ctx_len = sizeof(struct roc_se_kasumi_ctx); break; - case ROC_SE_PDCP_CHAIN: - ctx_len = sizeof(struct roc_se_zuc_snow3g_chain_ctx); - break; case ROC_SE_SM: ctx_len = sizeof(struct roc_se_sm_context); break; diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h index abb8c6a149..d62c40b310 100644 --- a/drivers/common/cnxk/roc_se.h +++ b/drivers/common/cnxk/roc_se.h @@ -246,7 +246,7 @@ struct roc_se_onk_zuc_ctx { uint8_t zuc_const[32]; }; -struct roc_se_onk_zuc_chain_ctx { +struct roc_se_pdcp_ctx { union { uint64_t u64; struct { @@ -278,19 +278,6 @@ struct roc_se_onk_zuc_chain_ctx { } st; }; -struct roc_se_zuc_snow3g_chain_ctx { - union { - struct roc_se_onk_zuc_chain_ctx onk_ctx; - } zuc; -}; - -struct roc_se_zuc_snow3g_ctx { - union { - struct roc_se_onk_zuc_ctx onk_ctx; - struct roc_se_otk_zuc_ctx otk_ctx; - } zuc; -}; - struct roc_se_kasumi_ctx { uint8_t reg_A[8]; uint8_t ci_key[16]; @@ -356,8 +343,7 @@ struct roc_se_ctx { } w0; union { struct roc_se_context fctx; - struct roc_se_zuc_snow3g_ctx zs_ctx; - struct roc_se_zuc_snow3g_chain_ctx zs_ch_ctx; + struct roc_se_pdcp_ctx pctx; struct roc_se_kasumi_ctx k_ctx; struct roc_se_sm_context sm_ctx; }; diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 1aec7dea9f..8193e96a92 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -298,8 +298,13 @@ sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t iv_d = ((uint8_t *)offset_vaddr + ROC_SE_OFF_CTRL_LEN); if (pdcp_flag) { - if (likely(iv_len)) - pdcp_iv_copy(iv_d, iv_s, pdcp_alg_type, pack_iv); + if (likely(iv_len)) { + if (zsk_flags == 0x1) + pdcp_iv_copy(iv_d + params->pdcp_iv_offset, iv_s, pdcp_alg_type, + pack_iv); + else + pdcp_iv_copy(iv_d, iv_s, pdcp_alg_type, pack_iv); + } } else { if (likely(iv_len)) memcpy(iv_d, iv_s, iv_len); @@ -375,7 +380,7 @@ sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t i = 0; scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); - if (zsk_flags == 0x1) { + if ((zsk_flags == 0x1) && (se_ctx->fc_type == ROC_SE_KASUMI)) { /* IV in SLIST only for EEA3 & UEA2 or for F8 */ iv_len = 0; } @@ -492,8 +497,13 @@ sg2_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t iv_d = ((uint8_t *)offset_vaddr + ROC_SE_OFF_CTRL_LEN); if (pdcp_flag) { - if (likely(iv_len)) - pdcp_iv_copy(iv_d, iv_s, pdcp_alg_type, pack_iv); + if (likely(iv_len)) { + if (zsk_flags == 0x1) + pdcp_iv_copy(iv_d + params->pdcp_iv_offset, iv_s, pdcp_alg_type, + pack_iv); + else + pdcp_iv_copy(iv_d, iv_s, pdcp_alg_type, pack_iv); + } } else { if (likely(iv_len)) memcpy(iv_d, iv_s, iv_len); @@ -567,7 +577,7 @@ sg2_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t i = 0; scatter_comp = (struct roc_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); - if (zsk_flags == 0x1) { + if ((zsk_flags == 0x1) && (se_ctx->fc_type == ROC_SE_KASUMI)) { /* IV in SLIST only for EEA3 & UEA2 or for F8 */ iv_len = 0; } @@ -1617,28 +1627,34 @@ static __rte_always_inline int cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, struct roc_se_fc_params *params, struct cpt_inst_s *inst, const bool is_sg_ver2) { + /* + * pdcp_iv_offset is auth_iv_offset wrt cipher_iv_offset which is + * 16 with old microcode without ZUC 256 support + * whereas it is 24 with new microcode which has ZUC 256. + * So iv_len reserved is 32B for cipher and auth IVs with old microcode + * and 48B with new microcode. + */ + const int iv_len = params->pdcp_iv_offset * 2; + struct roc_se_ctx *se_ctx = params->ctx; uint32_t encr_data_len, auth_data_len; + const int flags = se_ctx->zsk_flags; uint32_t encr_offset, auth_offset; union cpt_inst_w4 cpt_inst_w4; int32_t inputlen, outputlen; - struct roc_se_ctx *se_ctx; uint64_t *offset_vaddr; uint8_t pdcp_alg_type; uint32_t mac_len = 0; const uint8_t *iv_s; uint8_t pack_iv = 0; uint64_t offset_ctrl; - int flags, iv_len; int ret; - se_ctx = params->ctx; - flags = se_ctx->zsk_flags; mac_len = se_ctx->mac_len; cpt_inst_w4.u64 = se_ctx->template_w4.u64; - cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_PDCP; if (flags == 0x1) { + cpt_inst_w4.s.opcode_minor = 1; iv_s = params->auth_iv_buf; /* @@ -1650,47 +1666,32 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, pdcp_alg_type = se_ctx->pdcp_auth_alg; if (pdcp_alg_type != ROC_SE_PDCP_ALG_TYPE_AES_CMAC) { - iv_len = params->auth_iv_len; - if (iv_len == 25) { - iv_len -= 2; + if (params->auth_iv_len == 25) pack_iv = 1; - } auth_offset = auth_offset / 8; - - /* consider iv len */ - auth_offset += iv_len; - - inputlen = - auth_offset + (RTE_ALIGN(auth_data_len, 8) / 8); - } else { - iv_len = 16; - - /* consider iv len */ - auth_offset += iv_len; - - inputlen = auth_offset + auth_data_len; - - /* length should be in bits */ - auth_data_len *= 8; + auth_data_len = RTE_ALIGN(auth_data_len, 8) / 8; } - outputlen = mac_len; + /* consider iv len */ + auth_offset += iv_len; + + inputlen = auth_offset + auth_data_len; + outputlen = iv_len + mac_len; offset_ctrl = rte_cpu_to_be_64((uint64_t)auth_offset); + cpt_inst_w4.s.param1 = auth_data_len; encr_data_len = 0; encr_offset = 0; } else { + cpt_inst_w4.s.opcode_minor = 0; iv_s = params->iv_buf; - iv_len = params->cipher_iv_len; pdcp_alg_type = se_ctx->pdcp_ci_alg; - if (iv_len == 25) { - iv_len -= 2; + if (params->cipher_iv_len == 25) pack_iv = 1; - } /* * Microcode expects offsets in bytes @@ -1700,6 +1701,7 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, encr_offset = ROC_SE_ENCR_OFFSET(d_offs); encr_offset = encr_offset / 8; + /* consider iv len */ encr_offset += iv_len; @@ -1707,10 +1709,11 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, outputlen = inputlen; /* iv offset is 0 */ - offset_ctrl = rte_cpu_to_be_64((uint64_t)encr_offset << 16); + offset_ctrl = rte_cpu_to_be_64((uint64_t)encr_offset); auth_data_len = 0; auth_offset = 0; + cpt_inst_w4.s.param1 = (RTE_ALIGN(encr_data_len, 8) / 8); } if (unlikely((encr_offset >> 16) || (auth_offset >> 8))) { @@ -1720,12 +1723,6 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, return -1; } - /* - * Lengths are expected in bits. - */ - cpt_inst_w4.s.param1 = encr_data_len; - cpt_inst_w4.s.param2 = auth_data_len; - /* * In cn9k, cn10k since we have a limitation of * IV & Offset control word not part of instruction @@ -1738,6 +1735,7 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, /* Use Direct mode */ + cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_PDCP_CHAIN; offset_vaddr = (uint64_t *)((uint8_t *)dm_vaddr - ROC_SE_OFF_CTRL_LEN - iv_len); /* DPTR */ @@ -1753,6 +1751,7 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, *offset_vaddr = offset_ctrl; inst->w4.u64 = cpt_inst_w4.u64; } else { + cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_PDCP_CHAIN | ROC_DMA_MODE_SG; inst->w4.u64 = cpt_inst_w4.u64; if (is_sg_ver2) ret = sg2_inst_prep(params, inst, offset_ctrl, iv_s, iv_len, pack_iv, @@ -2243,8 +2242,6 @@ fill_sess_cipher(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) c_form->key.length))) return -1; - if ((enc_type >= ROC_SE_ZUC_EEA3) && (enc_type <= ROC_SE_AES_CTR_EEA2)) - roc_se_ctx_swap(&sess->roc_se_ctx); return 0; } @@ -2403,15 +2400,10 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) sess->auth_iv_offset = a_form->iv.offset; sess->auth_iv_length = a_form->iv.length; } - if (unlikely(roc_se_auth_key_set(&sess->roc_se_ctx, auth_type, - a_form->key.data, a_form->key.length, - a_form->digest_length))) + if (unlikely(roc_se_auth_key_set(&sess->roc_se_ctx, auth_type, a_form->key.data, + a_form->key.length, a_form->digest_length))) return -1; - if ((auth_type >= ROC_SE_ZUC_EIA3) && - (auth_type <= ROC_SE_AES_CMAC_EIA2)) - roc_se_ctx_swap(&sess->roc_se_ctx); - return 0; } From patchwork Thu Dec 21 12:35:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135451 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E03343750; Thu, 21 Dec 2023 13:38:24 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 06FE942EDC; Thu, 21 Dec 2023 13:37:00 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id E968B42EC0 for ; Thu, 21 Dec 2023 13:36:54 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS36019305 for ; Thu, 21 Dec 2023 04:36:54 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=V0dw9fLVl1kgJ+yshs1QvhfC3BLADhZvIZwB3ucRZvc=; b=ejq noH2JwEmkOSWZyHrxIFUzBI9K/0zPcOe5kIPAoHLc25dzfSdZLkQ9pAWICE9aIOf v4SIDOU+TagMXk/C+5/XIntNwc83ENvzzvwb1liQqEl4E0a+aDmh31sSIJcihabg uESu8Ya9dG8eQgwdcYFgA+zIdHZebLS8s0VIhEJvkROTwnuA/LQXS+ftNVpHmGue 1ijnOZ4oVpJ2piYKyVHQppEVVSW25XTiodcq0EmJvLCJM7aSuNhkQrUNp0DGIsyG RrVzcQlr3MyY4XbqMerkkzvxafxg/tQxRAwQinhNc/LGKddTFrDDamlDd2CFeyRC yVGpQoIlzt+nZIRPaOQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0k0-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:54 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:43 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:43 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 47D8A3F7079; Thu, 21 Dec 2023 04:36:39 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 20/24] crypto/cnxk: validate the combinations supported in TLS Date: Thu, 21 Dec 2023 18:05:41 +0530 Message-ID: <20231221123545.510-21-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: p-yeIsguZo7BB-KPI78fRYOXo3R6CrWN X-Proofpoint-ORIG-GUID: p-yeIsguZo7BB-KPI78fRYOXo3R6CrWN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Validate the cipher and auth combination to allow only the ones supported by hardware. Signed-off-by: Vidya Sagar Velumuri --- drivers/crypto/cnxk/cn10k_tls.c | 35 ++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/cnxk/cn10k_tls.c b/drivers/crypto/cnxk/cn10k_tls.c index e1ed65b06a..fa3ce3e758 100644 --- a/drivers/crypto/cnxk/cn10k_tls.c +++ b/drivers/crypto/cnxk/cn10k_tls.c @@ -17,6 +17,36 @@ #include "cnxk_cryptodev_ops.h" #include "cnxk_security.h" +static int +tls_xform_cipher_auth_verify(struct rte_crypto_sym_xform *cipher_xform, + struct rte_crypto_sym_xform *auth_xform) +{ + enum rte_crypto_cipher_algorithm c_algo = cipher_xform->cipher.algo; + enum rte_crypto_auth_algorithm a_algo = auth_xform->auth.algo; + int ret = -ENOTSUP; + + switch (c_algo) { + case RTE_CRYPTO_CIPHER_NULL: + if ((a_algo == RTE_CRYPTO_AUTH_MD5_HMAC) || (a_algo == RTE_CRYPTO_AUTH_SHA1_HMAC) || + (a_algo == RTE_CRYPTO_AUTH_SHA256_HMAC)) + ret = 0; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + if (a_algo == RTE_CRYPTO_AUTH_SHA1_HMAC) + ret = 0; + break; + case RTE_CRYPTO_CIPHER_AES_CBC: + if ((a_algo == RTE_CRYPTO_AUTH_SHA1_HMAC) || + (a_algo == RTE_CRYPTO_AUTH_SHA256_HMAC)) + ret = 0; + break; + default: + break; + } + + return ret; +} + static int tls_xform_cipher_verify(struct rte_crypto_sym_xform *crypto_xform) { @@ -138,7 +168,10 @@ cnxk_tls_xform_verify(struct rte_security_tls_record_xform *tls_xform, ret = tls_xform_cipher_verify(cipher_xform); if (!ret) - return tls_xform_auth_verify(auth_xform); + ret = tls_xform_auth_verify(auth_xform); + + if (cipher_xform && !ret) + return tls_xform_cipher_auth_verify(cipher_xform, auth_xform); return ret; } From patchwork Thu Dec 21 12:35:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135453 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5BA0043750; Thu, 21 Dec 2023 13:38:39 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 453DF42EEA; Thu, 21 Dec 2023 13:37:02 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 9C0F242EBC for ; Thu, 21 Dec 2023 13:36:55 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS39019305 for ; Thu, 21 Dec 2023 04:36:54 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=XopYekZuDL9WlxgM+U/L8z58g2Uyw0c9/nU2RwVloRw=; b=VhK rA8q/WqwVMwAnWZntRiLQL8dXirWiL13cfYjJ6BkoV06Yf9SmZiBeG/wgiOZB939 TesqpAS5ONRH+dJl9tB7sO6xKHFYvMeI09wF9OIQM/wUn+1kveFH9jHcGt6SJv6F ff4TDup18JCOD69PGMopuZ6yny7J/ia7+fff6ID7Ysav8dU1Sodt80sS7w5fRiDn KUS9HuU9d0RkMkntA9Aq5B4q7p3Rkymg6RAJ3o9u8uUXiI/ubN9wO4bCrXTvQOkz wijVrt3kuVVgPvasKynNejrFHW6O/swTzVhtOGVNbFGHl8JfqmQZ/IAFj4SH1laN 4lD0IBcYlM4cdWr0AbA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0k0-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:54 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:45 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:45 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 0135D3F7073; Thu, 21 Dec 2023 04:36:42 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 21/24] crypto/cnxk: use a single function for opad ipad Date: Thu, 21 Dec 2023 18:05:42 +0530 Message-ID: <20231221123545.510-22-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: KfltbpXu-F_RZ9B7-7YcIsOwQmcs-f3B X-Proofpoint-ORIG-GUID: KfltbpXu-F_RZ9B7-7YcIsOwQmcs-f3B X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Use a single function for opad and ipad generation for IPsec, TLS and flexi crypto. Signed-off-by: Vidya Sagar Velumuri --- drivers/common/cnxk/cnxk_security.c | 65 ++++++----------------------- drivers/common/cnxk/cnxk_security.h | 5 --- drivers/common/cnxk/roc_se.c | 48 ++++++++++++++------- drivers/common/cnxk/roc_se.h | 9 ++++ drivers/common/cnxk/version.map | 2 +- drivers/crypto/cnxk/cn10k_tls.c | 8 +++- 6 files changed, 61 insertions(+), 76 deletions(-) diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c index bdb04fe142..64c901a57a 100644 --- a/drivers/common/cnxk/cnxk_security.c +++ b/drivers/common/cnxk/cnxk_security.c @@ -8,55 +8,9 @@ #include "roc_api.h" -void -cnxk_sec_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, uint8_t *hmac_opad_ipad, - bool is_tls) -{ - const uint8_t *key = auth_xform->auth.key.data; - uint32_t length = auth_xform->auth.key.length; - uint8_t opad[128] = {[0 ... 127] = 0x5c}; - uint8_t ipad[128] = {[0 ... 127] = 0x36}; - uint32_t i; - - /* HMAC OPAD and IPAD */ - for (i = 0; i < 128 && i < length; i++) { - opad[i] = opad[i] ^ key[i]; - ipad[i] = ipad[i] ^ key[i]; - } - - /* Precompute hash of HMAC OPAD and IPAD to avoid - * per packet computation - */ - switch (auth_xform->auth.algo) { - case RTE_CRYPTO_AUTH_MD5_HMAC: - roc_hash_md5_gen(opad, (uint32_t *)&hmac_opad_ipad[0]); - roc_hash_md5_gen(ipad, (uint32_t *)&hmac_opad_ipad[is_tls ? 64 : 24]); - break; - case RTE_CRYPTO_AUTH_SHA1_HMAC: - roc_hash_sha1_gen(opad, (uint32_t *)&hmac_opad_ipad[0]); - roc_hash_sha1_gen(ipad, (uint32_t *)&hmac_opad_ipad[is_tls ? 64 : 24]); - break; - case RTE_CRYPTO_AUTH_SHA256_HMAC: - roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0], 256); - roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64], 256); - break; - case RTE_CRYPTO_AUTH_SHA384_HMAC: - roc_hash_sha512_gen(opad, (uint64_t *)&hmac_opad_ipad[0], 384); - roc_hash_sha512_gen(ipad, (uint64_t *)&hmac_opad_ipad[64], 384); - break; - case RTE_CRYPTO_AUTH_SHA512_HMAC: - roc_hash_sha512_gen(opad, (uint64_t *)&hmac_opad_ipad[0], 512); - roc_hash_sha512_gen(ipad, (uint64_t *)&hmac_opad_ipad[64], 512); - break; - default: - break; - } -} - static int -ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, - uint8_t *cipher_key, uint8_t *salt_key, - uint8_t *hmac_opad_ipad, +ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, uint8_t *cipher_key, + uint8_t *salt_key, uint8_t *hmac_opad_ipad, struct rte_security_ipsec_xform *ipsec_xfrm, struct rte_crypto_sym_xform *crypto_xfrm) { @@ -192,7 +146,9 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, const uint8_t *auth_key = auth_xfrm->auth.key.data; roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad); } else { - cnxk_sec_opad_ipad_gen(auth_xfrm, hmac_opad_ipad, false); + roc_se_hmac_opad_ipad_gen(w2->s.auth_type, auth_xfrm->auth.key.data, + auth_xfrm->auth.key.length, &hmac_opad_ipad[0], + ROC_SE_IPSEC); } tmp_key = (uint64_t *)hmac_opad_ipad; @@ -741,7 +697,8 @@ onf_ipsec_sa_common_param_fill(struct roc_ie_onf_sa_ctl *ctl, uint8_t *salt, key = cipher_xfrm->cipher.key.data; length = cipher_xfrm->cipher.key.length; - cnxk_sec_opad_ipad_gen(auth_xfrm, hmac_opad_ipad, false); + roc_se_hmac_opad_ipad_gen(ctl->auth_type, auth_xfrm->auth.key.data, + auth_xfrm->auth.key.length, hmac_opad_ipad, ROC_SE_IPSEC); } switch (length) { @@ -1374,7 +1331,9 @@ cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipsec, roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad); } else if (auth_xform->auth.algo != RTE_CRYPTO_AUTH_NULL) { - cnxk_sec_opad_ipad_gen(auth_xform, hmac_opad_ipad, false); + roc_se_hmac_opad_ipad_gen( + out_sa->common_sa.ctl.auth_type, auth_xform->auth.key.data, + auth_xform->auth.key.length, &hmac_opad_ipad[0], ROC_SE_IPSEC); } } @@ -1441,7 +1400,9 @@ cnxk_on_ipsec_inb_sa_create(struct rte_security_ipsec_xform *ipsec, roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad); } else if (auth_xform->auth.algo != RTE_CRYPTO_AUTH_NULL) { - cnxk_sec_opad_ipad_gen(auth_xform, hmac_opad_ipad, false); + roc_se_hmac_opad_ipad_gen( + in_sa->common_sa.ctl.auth_type, auth_xform->auth.key.data, + auth_xform->auth.key.length, &hmac_opad_ipad[0], ROC_SE_IPSEC); } } diff --git a/drivers/common/cnxk/cnxk_security.h b/drivers/common/cnxk/cnxk_security.h index 86ec657cb0..b323b8b757 100644 --- a/drivers/common/cnxk/cnxk_security.h +++ b/drivers/common/cnxk/cnxk_security.h @@ -68,9 +68,4 @@ int __roc_api cnxk_on_ipsec_inb_sa_create(struct rte_security_ipsec_xform *ipsec int __roc_api cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipsec, struct rte_crypto_sym_xform *crypto_xform, struct roc_ie_on_outb_sa *out_sa); - -__rte_internal -void cnxk_sec_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, uint8_t *hmac_opad_ipad, - bool is_tls); - #endif /* _CNXK_SECURITY_H__ */ diff --git a/drivers/common/cnxk/roc_se.c b/drivers/common/cnxk/roc_se.c index 4e00268149..5a3ed0b647 100644 --- a/drivers/common/cnxk/roc_se.c +++ b/drivers/common/cnxk/roc_se.c @@ -157,14 +157,29 @@ cpt_ciph_aes_key_type_set(struct roc_se_context *fctx, uint16_t key_len) fctx->enc.aes_key = aes_key_type; } -static void -cpt_hmac_opad_ipad_gen(roc_se_auth_type auth_type, const uint8_t *key, uint16_t length, - struct roc_se_hmac_context *hmac) +void +roc_se_hmac_opad_ipad_gen(roc_se_auth_type auth_type, const uint8_t *key, uint16_t length, + uint8_t *opad_ipad, roc_se_op_type op_type) { uint8_t opad[128] = {[0 ... 127] = 0x5c}; uint8_t ipad[128] = {[0 ... 127] = 0x36}; + uint8_t ipad_offset, opad_offset; uint32_t i; + if (op_type == ROC_SE_IPSEC) { + if ((auth_type == ROC_SE_MD5_TYPE) || (auth_type == ROC_SE_SHA1_TYPE)) + ipad_offset = 24; + else + ipad_offset = 64; + opad_offset = 0; + } else if (op_type == ROC_SE_TLS) { + ipad_offset = 64; + opad_offset = 0; + } else { + ipad_offset = 0; + opad_offset = 64; + } + /* HMAC OPAD and IPAD */ for (i = 0; i < 128 && i < length; i++) { opad[i] = opad[i] ^ key[i]; @@ -176,28 +191,28 @@ cpt_hmac_opad_ipad_gen(roc_se_auth_type auth_type, const uint8_t *key, uint16_t */ switch (auth_type) { case ROC_SE_MD5_TYPE: - roc_hash_md5_gen(opad, (uint32_t *)hmac->opad); - roc_hash_md5_gen(ipad, (uint32_t *)hmac->ipad); + roc_hash_md5_gen(opad, (uint32_t *)&opad_ipad[opad_offset]); + roc_hash_md5_gen(ipad, (uint32_t *)&opad_ipad[ipad_offset]); break; case ROC_SE_SHA1_TYPE: - roc_hash_sha1_gen(opad, (uint32_t *)hmac->opad); - roc_hash_sha1_gen(ipad, (uint32_t *)hmac->ipad); + roc_hash_sha1_gen(opad, (uint32_t *)&opad_ipad[opad_offset]); + roc_hash_sha1_gen(ipad, (uint32_t *)&opad_ipad[ipad_offset]); break; case ROC_SE_SHA2_SHA224: - roc_hash_sha256_gen(opad, (uint32_t *)hmac->opad, 224); - roc_hash_sha256_gen(ipad, (uint32_t *)hmac->ipad, 224); + roc_hash_sha256_gen(opad, (uint32_t *)&opad_ipad[opad_offset], 224); + roc_hash_sha256_gen(ipad, (uint32_t *)&opad_ipad[ipad_offset], 224); break; case ROC_SE_SHA2_SHA256: - roc_hash_sha256_gen(opad, (uint32_t *)hmac->opad, 256); - roc_hash_sha256_gen(ipad, (uint32_t *)hmac->ipad, 256); + roc_hash_sha256_gen(opad, (uint32_t *)&opad_ipad[opad_offset], 256); + roc_hash_sha256_gen(ipad, (uint32_t *)&opad_ipad[ipad_offset], 256); break; case ROC_SE_SHA2_SHA384: - roc_hash_sha512_gen(opad, (uint64_t *)hmac->opad, 384); - roc_hash_sha512_gen(ipad, (uint64_t *)hmac->ipad, 384); + roc_hash_sha512_gen(opad, (uint64_t *)&opad_ipad[opad_offset], 384); + roc_hash_sha512_gen(ipad, (uint64_t *)&opad_ipad[ipad_offset], 384); break; case ROC_SE_SHA2_SHA512: - roc_hash_sha512_gen(opad, (uint64_t *)hmac->opad, 512); - roc_hash_sha512_gen(ipad, (uint64_t *)hmac->ipad, 512); + roc_hash_sha512_gen(opad, (uint64_t *)&opad_ipad[opad_offset], 512); + roc_hash_sha512_gen(ipad, (uint64_t *)&opad_ipad[ipad_offset], 512); break; default: break; @@ -401,7 +416,8 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, const uint if (chained_op) { memset(fctx->hmac.ipad, 0, sizeof(fctx->hmac.ipad)); memset(fctx->hmac.opad, 0, sizeof(fctx->hmac.opad)); - cpt_hmac_opad_ipad_gen(type, key, key_len, &fctx->hmac); + roc_se_hmac_opad_ipad_gen(type, key, key_len, &fctx->hmac.ipad[0], + ROC_SE_FC); fctx->enc.auth_input_type = 0; } else { se_ctx->hmac = 1; diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h index d62c40b310..ddcf6bdb44 100644 --- a/drivers/common/cnxk/roc_se.h +++ b/drivers/common/cnxk/roc_se.h @@ -191,6 +191,12 @@ typedef enum { ROC_SE_PDCP_MAC_LEN_128_BIT = 0x3 } roc_se_pdcp_mac_len_type; +typedef enum { + ROC_SE_IPSEC = 0x0, + ROC_SE_TLS = 0x1, + ROC_SE_FC = 0x2, +} roc_se_op_type; + struct roc_se_enc_context { uint64_t iv_source : 1; uint64_t aes_key : 2; @@ -401,4 +407,7 @@ int __roc_api roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type void __roc_api roc_se_ctx_swap(struct roc_se_ctx *se_ctx); void __roc_api roc_se_ctx_init(struct roc_se_ctx *se_ctx); +void __roc_api roc_se_hmac_opad_ipad_gen(roc_se_auth_type auth_type, const uint8_t *key, + uint16_t length, uint8_t *opad_ipad, + roc_se_op_type op_type); #endif /* __ROC_SE_H__ */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 15fd5710d2..b8b0478848 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -1,7 +1,6 @@ INTERNAL { global: - cnxk_sec_opad_ipad_gen; cnxk_ipsec_icvlen_get; cnxk_ipsec_ivlen_get; cnxk_ipsec_outb_rlens_get; @@ -472,6 +471,7 @@ INTERNAL { roc_plt_init; roc_plt_init_cb_register; roc_plt_lmt_validate; + roc_se_hmac_opad_ipad_gen; roc_sso_dev_fini; roc_sso_dev_init; roc_sso_dump; diff --git a/drivers/crypto/cnxk/cn10k_tls.c b/drivers/crypto/cnxk/cn10k_tls.c index fa3ce3e758..5baea181e8 100644 --- a/drivers/crypto/cnxk/cn10k_tls.c +++ b/drivers/crypto/cnxk/cn10k_tls.c @@ -376,7 +376,9 @@ tls_read_sa_fill(struct roc_ie_ot_tls_read_sa *read_sa, else return -EINVAL; - cnxk_sec_opad_ipad_gen(auth_xfrm, read_sa->opad_ipad, true); + roc_se_hmac_opad_ipad_gen(read_sa->w2.s.mac_select, auth_xfrm->auth.key.data, + auth_xfrm->auth.key.length, read_sa->opad_ipad, ROC_SE_TLS); + tmp = (uint64_t *)read_sa->opad_ipad; for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN / sizeof(uint64_t)); i++) tmp[i] = rte_be_to_cpu_64(tmp[i]); @@ -503,7 +505,9 @@ tls_write_sa_fill(struct roc_ie_ot_tls_write_sa *write_sa, else return -EINVAL; - cnxk_sec_opad_ipad_gen(auth_xfrm, write_sa->opad_ipad, true); + roc_se_hmac_opad_ipad_gen(write_sa->w2.s.mac_select, auth_xfrm->auth.key.data, + auth_xfrm->auth.key.length, write_sa->opad_ipad, + ROC_SE_TLS); } tmp_key = (uint64_t *)write_sa->opad_ipad; From patchwork Thu Dec 21 12:35:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135450 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 575DB43750; Thu, 21 Dec 2023 13:38:15 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A0DF442ED4; Thu, 21 Dec 2023 13:36:58 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id D3BAE42EBC for ; Thu, 21 Dec 2023 13:36:54 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS35019305 for ; Thu, 21 Dec 2023 04:36:54 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=okzcIATGKpMyRsxls3E45oIpbYuEo1zV64R0UrHXcUc=; b=V6A His9iZ0vb/SOvDl5iM1uC9ZE5PjMU71Y0DguoFVCPmsKAl4LV4CZNVrOPCoU1VYn tQt+24qRMh1DmWqQ/SJrl04aTBQY0ZFs2x8PGS8fqGxZMJPBUixEff9Jw+DekvKi L+2+lU9iSJFBlVmI3eR2J+sg3YTpnKECsR9b72OfRQL9GmbSJ5HWcdl8TOAV6xUD MSocyKrE/tIUFUnWiIca9vuotQpyWzXz0/r0ORIqLpJgB/54iQAl24C0rx8ZcCqR 6GqWjfqd7RMWVpysOdlzBbz9M9KDr047kvZJ2RC0PkYM1qmUpzav1t/AfczN+Ie3 TsuXCXfl6tPjVVJ37nw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0jq-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:53 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:48 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:48 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id A32A33F7079; Thu, 21 Dec 2023 04:36:45 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 22/24] crypto/cnxk: add support for TLS 1.3 Date: Thu, 21 Dec 2023 18:05:43 +0530 Message-ID: <20231221123545.510-23-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: TYZzIO9Yroa5uJfcudkMyjbvOAP_chWt X-Proofpoint-ORIG-GUID: TYZzIO9Yroa5uJfcudkMyjbvOAP_chWt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Add support for TLS-1.3. Signed-off-by: Vidya Sagar Velumuri --- drivers/common/cnxk/roc_ie_ot_tls.h | 50 +++++-- drivers/crypto/cnxk/cn10k_cryptodev_sec.h | 3 +- drivers/crypto/cnxk/cn10k_tls.c | 159 +++++++++++++--------- 3 files changed, 136 insertions(+), 76 deletions(-) diff --git a/drivers/common/cnxk/roc_ie_ot_tls.h b/drivers/common/cnxk/roc_ie_ot_tls.h index 61955ef4d1..91ddb25f7a 100644 --- a/drivers/common/cnxk/roc_ie_ot_tls.h +++ b/drivers/common/cnxk/roc_ie_ot_tls.h @@ -17,8 +17,10 @@ (PLT_ALIGN_CEIL(ROC_IE_OT_TLS_AR_WIN_SIZE_MAX, BITS_PER_LONG_LONG) / BITS_PER_LONG_LONG) /* CN10K TLS opcodes */ -#define ROC_IE_OT_TLS_MAJOR_OP_RECORD_ENC 0x16UL -#define ROC_IE_OT_TLS_MAJOR_OP_RECORD_DEC 0x17UL +#define ROC_IE_OT_TLS_MAJOR_OP_RECORD_ENC 0x16UL +#define ROC_IE_OT_TLS_MAJOR_OP_RECORD_DEC 0x17UL +#define ROC_IE_OT_TLS13_MAJOR_OP_RECORD_ENC 0x18UL +#define ROC_IE_OT_TLS13_MAJOR_OP_RECORD_DEC 0x19UL #define ROC_IE_OT_TLS_CTX_MAX_OPAD_IPAD_LEN 128 #define ROC_IE_OT_TLS_CTX_MAX_KEY_IV_LEN 48 @@ -42,6 +44,7 @@ enum roc_ie_ot_tls_cipher_type { enum roc_ie_ot_tls_ver { ROC_IE_OT_TLS_VERSION_TLS_12 = 1, ROC_IE_OT_TLS_VERSION_DTLS_12 = 2, + ROC_IE_OT_TLS_VERSION_TLS_13 = 3, }; enum roc_ie_ot_tls_aes_key_len { @@ -131,11 +134,23 @@ struct roc_ie_ot_tls_read_sa { /* Word4 - Word9 */ uint8_t cipher_key[ROC_IE_OT_TLS_CTX_MAX_KEY_IV_LEN]; - /* Word10 - Word25 */ - uint8_t opad_ipad[ROC_IE_OT_TLS_CTX_MAX_OPAD_IPAD_LEN]; + union { + struct { + /* Word10 */ + uint64_t w10_rsvd6; + + /* Word11 - Word25 */ + struct roc_ie_ot_tls_read_ctx_update_reg ctx; + } tls_13; + + struct { + /* Word10 - Word25 */ + uint8_t opad_ipad[ROC_IE_OT_TLS_CTX_MAX_OPAD_IPAD_LEN]; - /* Word26 - Word32 */ - struct roc_ie_ot_tls_read_ctx_update_reg ctx; + /* Word26 - Word95 */ + struct roc_ie_ot_tls_read_ctx_update_reg ctx; + } tls_12; + }; }; struct roc_ie_ot_tls_write_sa { @@ -187,13 +202,24 @@ struct roc_ie_ot_tls_write_sa { /* Word4 - Word9 */ uint8_t cipher_key[ROC_IE_OT_TLS_CTX_MAX_KEY_IV_LEN]; - /* Word10 - Word25 */ - uint8_t opad_ipad[ROC_IE_OT_TLS_CTX_MAX_OPAD_IPAD_LEN]; + union { + struct { + /* Word10 */ + uint64_t w10_rsvd7; + + uint64_t seq_num; + } tls_13; + + struct { + /* Word10 - Word25 */ + uint8_t opad_ipad[ROC_IE_OT_TLS_CTX_MAX_OPAD_IPAD_LEN]; - /* Word26 */ - uint64_t w26_rsvd7; + /* Word26 */ + uint64_t w26_rsvd7; - /* Word27 */ - uint64_t seq_num; + /* Word27 */ + uint64_t seq_num; + } tls_12; + }; }; #endif /* __ROC_IE_OT_TLS_H__ */ diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_sec.h b/drivers/crypto/cnxk/cn10k_cryptodev_sec.h index 33fd3aa398..1e117051cc 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_sec.h +++ b/drivers/crypto/cnxk/cn10k_cryptodev_sec.h @@ -31,8 +31,7 @@ struct cn10k_sec_session { } ipsec; struct { uint8_t enable_padding : 1; - uint8_t hdr_len : 4; - uint8_t rvsd : 3; + uint8_t rvsd : 7; bool is_write; } tls; }; diff --git a/drivers/crypto/cnxk/cn10k_tls.c b/drivers/crypto/cnxk/cn10k_tls.c index 5baea181e8..ce253e3eba 100644 --- a/drivers/crypto/cnxk/cn10k_tls.c +++ b/drivers/crypto/cnxk/cn10k_tls.c @@ -105,7 +105,8 @@ cnxk_tls_xform_verify(struct rte_security_tls_record_xform *tls_xform, int ret = 0; if ((tls_xform->ver != RTE_SECURITY_VERSION_TLS_1_2) && - (tls_xform->ver != RTE_SECURITY_VERSION_DTLS_1_2)) + (tls_xform->ver != RTE_SECURITY_VERSION_DTLS_1_2) && + (tls_xform->ver != RTE_SECURITY_VERSION_TLS_1_3)) return -EINVAL; if ((tls_xform->type != RTE_SECURITY_TLS_SESS_TYPE_READ) && @@ -115,6 +116,12 @@ cnxk_tls_xform_verify(struct rte_security_tls_record_xform *tls_xform, if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) return tls_xform_aead_verify(tls_xform, crypto_xform); + /* TLS-1.3 only support AEAD. + * Control should not reach here for TLS-1.3 + */ + if (tls_xform->ver == RTE_SECURITY_VERSION_TLS_1_3) + return -EINVAL; + if (tls_xform->type == RTE_SECURITY_TLS_SESS_TYPE_WRITE) { /* Egress */ @@ -259,7 +266,7 @@ tls_write_sa_init(struct roc_ie_ot_tls_write_sa *sa) memset(sa, 0, sizeof(struct roc_ie_ot_tls_write_sa)); - offset = offsetof(struct roc_ie_ot_tls_write_sa, w26_rsvd7); + offset = offsetof(struct roc_ie_ot_tls_write_sa, tls_12.w26_rsvd7); sa->w0.s.hw_ctx_off = offset / ROC_CTX_UNIT_8B; sa->w0.s.ctx_push_size = sa->w0.s.hw_ctx_off; sa->w0.s.ctx_size = ROC_IE_OT_TLS_CTX_ILEN; @@ -274,7 +281,7 @@ tls_read_sa_init(struct roc_ie_ot_tls_read_sa *sa) memset(sa, 0, sizeof(struct roc_ie_ot_tls_read_sa)); - offset = offsetof(struct roc_ie_ot_tls_read_sa, ctx); + offset = offsetof(struct roc_ie_ot_tls_read_sa, tls_12.ctx); sa->w0.s.hw_ctx_off = offset / ROC_CTX_UNIT_8B; sa->w0.s.ctx_push_size = sa->w0.s.hw_ctx_off; sa->w0.s.ctx_size = ROC_IE_OT_TLS_CTX_ILEN; @@ -283,13 +290,18 @@ tls_read_sa_init(struct roc_ie_ot_tls_read_sa *sa) } static size_t -tls_read_ctx_size(struct roc_ie_ot_tls_read_sa *sa) +tls_read_ctx_size(struct roc_ie_ot_tls_read_sa *sa, enum rte_security_tls_version tls_ver) { size_t size; /* Variable based on Anti-replay Window */ - size = offsetof(struct roc_ie_ot_tls_read_sa, ctx) + - offsetof(struct roc_ie_ot_tls_read_ctx_update_reg, ar_winbits); + if (tls_ver == RTE_SECURITY_VERSION_TLS_1_3) { + size = offsetof(struct roc_ie_ot_tls_read_sa, tls_13.ctx) + + offsetof(struct roc_ie_ot_tls_read_ctx_update_reg, ar_winbits); + } else { + size = offsetof(struct roc_ie_ot_tls_read_sa, tls_12.ctx) + + offsetof(struct roc_ie_ot_tls_read_ctx_update_reg, ar_winbits); + } if (sa->w0.s.ar_win) size += (1 << (sa->w0.s.ar_win - 1)) * sizeof(uint64_t); @@ -302,6 +314,7 @@ tls_read_sa_fill(struct roc_ie_ot_tls_read_sa *read_sa, struct rte_security_tls_record_xform *tls_xfrm, struct rte_crypto_sym_xform *crypto_xfrm) { + enum rte_security_tls_version tls_ver = tls_xfrm->ver; struct rte_crypto_sym_xform *auth_xfrm, *cipher_xfrm; const uint8_t *key = NULL; uint64_t *tmp, *tmp_key; @@ -313,13 +326,22 @@ tls_read_sa_fill(struct roc_ie_ot_tls_read_sa *read_sa, /* Initialize the SA */ memset(read_sa, 0, sizeof(struct roc_ie_ot_tls_read_sa)); + if (tls_ver == RTE_SECURITY_VERSION_TLS_1_2) { + read_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_TLS_12; + read_sa->tls_12.ctx.ar_valid_mask = tls_xfrm->tls_1_2.seq_no - 1; + } else if (tls_ver == RTE_SECURITY_VERSION_DTLS_1_2) { + read_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_DTLS_12; + } else if (tls_ver == RTE_SECURITY_VERSION_TLS_1_3) { + read_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_TLS_13; + read_sa->tls_13.ctx.ar_valid_mask = tls_xfrm->tls_1_3.seq_no - 1; + } + cipher_key = read_sa->cipher_key; /* Set encryption algorithm */ if ((crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) && (crypto_xfrm->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)) { read_sa->w2.s.cipher_select = ROC_IE_OT_TLS_CIPHER_AES_GCM; - read_sa->w2.s.mac_select = ROC_IE_OT_TLS_MAC_SHA2_256; length = crypto_xfrm->aead.key.length; if (length == 16) @@ -330,10 +352,12 @@ tls_read_sa_fill(struct roc_ie_ot_tls_read_sa *read_sa, key = crypto_xfrm->aead.key.data; memcpy(cipher_key, key, length); - if (tls_xfrm->ver == RTE_SECURITY_VERSION_TLS_1_2) + if (tls_ver == RTE_SECURITY_VERSION_TLS_1_2) memcpy(((uint8_t *)cipher_key + 32), &tls_xfrm->tls_1_2.imp_nonce, 4); - else if (tls_xfrm->ver == RTE_SECURITY_VERSION_DTLS_1_2) + else if (tls_ver == RTE_SECURITY_VERSION_DTLS_1_2) memcpy(((uint8_t *)cipher_key + 32), &tls_xfrm->dtls_1_2.imp_nonce, 4); + else if (tls_ver == RTE_SECURITY_VERSION_TLS_1_3) + memcpy(((uint8_t *)cipher_key + 32), &tls_xfrm->tls_1_3.imp_nonce, 12); goto key_swap; } @@ -377,9 +401,10 @@ tls_read_sa_fill(struct roc_ie_ot_tls_read_sa *read_sa, return -EINVAL; roc_se_hmac_opad_ipad_gen(read_sa->w2.s.mac_select, auth_xfrm->auth.key.data, - auth_xfrm->auth.key.length, read_sa->opad_ipad, ROC_SE_TLS); + auth_xfrm->auth.key.length, read_sa->tls_12.opad_ipad, + ROC_SE_TLS); - tmp = (uint64_t *)read_sa->opad_ipad; + tmp = (uint64_t *)read_sa->tls_12.opad_ipad; for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN / sizeof(uint64_t)); i++) tmp[i] = rte_be_to_cpu_64(tmp[i]); @@ -403,24 +428,20 @@ tls_read_sa_fill(struct roc_ie_ot_tls_read_sa *read_sa, read_sa->w0.s.ctx_hdr_size = ROC_IE_OT_TLS_CTX_HDR_SIZE; read_sa->w0.s.aop_valid = 1; - offset = offsetof(struct roc_ie_ot_tls_read_sa, ctx); + offset = offsetof(struct roc_ie_ot_tls_read_sa, tls_12.ctx); + if (tls_ver == RTE_SECURITY_VERSION_TLS_1_3) + offset = offsetof(struct roc_ie_ot_tls_read_sa, tls_13.ctx); + + /* Entire context size in 128B units */ + read_sa->w0.s.ctx_size = + (PLT_ALIGN_CEIL(tls_read_ctx_size(read_sa, tls_ver), ROC_CTX_UNIT_128B) / + ROC_CTX_UNIT_128B) - + 1; /* Word offset for HW managed CTX field */ read_sa->w0.s.hw_ctx_off = offset / 8; read_sa->w0.s.ctx_push_size = read_sa->w0.s.hw_ctx_off; - /* Entire context size in 128B units */ - read_sa->w0.s.ctx_size = (PLT_ALIGN_CEIL(tls_read_ctx_size(read_sa), ROC_CTX_UNIT_128B) / - ROC_CTX_UNIT_128B) - - 1; - - if (tls_xfrm->ver == RTE_SECURITY_VERSION_TLS_1_2) { - read_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_TLS_12; - read_sa->ctx.ar_valid_mask = tls_xfrm->tls_1_2.seq_no - 1; - } else if (tls_xfrm->ver == RTE_SECURITY_VERSION_DTLS_1_2) { - read_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_DTLS_12; - } - rte_wmb(); return 0; @@ -431,6 +452,7 @@ tls_write_sa_fill(struct roc_ie_ot_tls_write_sa *write_sa, struct rte_security_tls_record_xform *tls_xfrm, struct rte_crypto_sym_xform *crypto_xfrm) { + enum rte_security_tls_version tls_ver = tls_xfrm->ver; struct rte_crypto_sym_xform *auth_xfrm, *cipher_xfrm; const uint8_t *key = NULL; uint8_t *cipher_key; @@ -438,13 +460,25 @@ tls_write_sa_fill(struct roc_ie_ot_tls_write_sa *write_sa, int i, length = 0; size_t offset; + if (tls_ver == RTE_SECURITY_VERSION_TLS_1_2) { + write_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_TLS_12; + write_sa->tls_12.seq_num = tls_xfrm->tls_1_2.seq_no - 1; + } else if (tls_ver == RTE_SECURITY_VERSION_DTLS_1_2) { + write_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_DTLS_12; + write_sa->tls_12.seq_num = ((uint64_t)tls_xfrm->dtls_1_2.epoch << 48) | + (tls_xfrm->dtls_1_2.seq_no & 0x0000ffffffffffff); + write_sa->tls_12.seq_num -= 1; + } else if (tls_ver == RTE_SECURITY_VERSION_TLS_1_3) { + write_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_TLS_13; + write_sa->tls_13.seq_num = tls_xfrm->tls_1_3.seq_no - 1; + } + cipher_key = write_sa->cipher_key; /* Set encryption algorithm */ if ((crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) && (crypto_xfrm->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)) { write_sa->w2.s.cipher_select = ROC_IE_OT_TLS_CIPHER_AES_GCM; - write_sa->w2.s.mac_select = ROC_IE_OT_TLS_MAC_SHA2_256; length = crypto_xfrm->aead.key.length; if (length == 16) @@ -455,10 +489,12 @@ tls_write_sa_fill(struct roc_ie_ot_tls_write_sa *write_sa, key = crypto_xfrm->aead.key.data; memcpy(cipher_key, key, length); - if (tls_xfrm->ver == RTE_SECURITY_VERSION_TLS_1_2) + if (tls_ver == RTE_SECURITY_VERSION_TLS_1_2) memcpy(((uint8_t *)cipher_key + 32), &tls_xfrm->tls_1_2.imp_nonce, 4); - else if (tls_xfrm->ver == RTE_SECURITY_VERSION_DTLS_1_2) + else if (tls_ver == RTE_SECURITY_VERSION_DTLS_1_2) memcpy(((uint8_t *)cipher_key + 32), &tls_xfrm->dtls_1_2.imp_nonce, 4); + else if (tls_ver == RTE_SECURITY_VERSION_TLS_1_3) + memcpy(((uint8_t *)cipher_key + 32), &tls_xfrm->tls_1_3.imp_nonce, 12); goto key_swap; } @@ -506,11 +542,11 @@ tls_write_sa_fill(struct roc_ie_ot_tls_write_sa *write_sa, return -EINVAL; roc_se_hmac_opad_ipad_gen(write_sa->w2.s.mac_select, auth_xfrm->auth.key.data, - auth_xfrm->auth.key.length, write_sa->opad_ipad, + auth_xfrm->auth.key.length, write_sa->tls_12.opad_ipad, ROC_SE_TLS); } - tmp_key = (uint64_t *)write_sa->opad_ipad; + tmp_key = (uint64_t *)write_sa->tls_12.opad_ipad; for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN / sizeof(uint64_t)); i++) tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]); @@ -520,40 +556,37 @@ tls_write_sa_fill(struct roc_ie_ot_tls_write_sa *write_sa, tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]); write_sa->w0.s.ctx_hdr_size = ROC_IE_OT_TLS_CTX_HDR_SIZE; - offset = offsetof(struct roc_ie_ot_tls_write_sa, w26_rsvd7); - - /* Word offset for HW managed CTX field */ - write_sa->w0.s.hw_ctx_off = offset / 8; - write_sa->w0.s.ctx_push_size = write_sa->w0.s.hw_ctx_off; - /* Entire context size in 128B units */ write_sa->w0.s.ctx_size = (PLT_ALIGN_CEIL(sizeof(struct roc_ie_ot_tls_write_sa), ROC_CTX_UNIT_128B) / ROC_CTX_UNIT_128B) - 1; - write_sa->w0.s.aop_valid = 1; + offset = offsetof(struct roc_ie_ot_tls_write_sa, tls_12.w26_rsvd7); - if (tls_xfrm->ver == RTE_SECURITY_VERSION_TLS_1_2) { - write_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_TLS_12; - write_sa->seq_num = tls_xfrm->tls_1_2.seq_no - 1; - } else if (tls_xfrm->ver == RTE_SECURITY_VERSION_DTLS_1_2) { - write_sa->w2.s.version_select = ROC_IE_OT_TLS_VERSION_DTLS_12; - write_sa->seq_num = ((uint64_t)tls_xfrm->dtls_1_2.epoch << 48) | - (tls_xfrm->dtls_1_2.seq_no & 0x0000ffffffffffff); - write_sa->seq_num -= 1; + if (tls_ver == RTE_SECURITY_VERSION_TLS_1_3) { + offset = offsetof(struct roc_ie_ot_tls_write_sa, tls_13.w10_rsvd7); + write_sa->w0.s.ctx_size -= 1; } + /* Word offset for HW managed CTX field */ + write_sa->w0.s.hw_ctx_off = offset / 8; + write_sa->w0.s.ctx_push_size = write_sa->w0.s.hw_ctx_off; + + write_sa->w0.s.aop_valid = 1; + write_sa->w2.s.iv_at_cptr = ROC_IE_OT_TLS_IV_SRC_DEFAULT; + if (write_sa->w2.s.version_select != ROC_IE_OT_TLS_VERSION_TLS_13) { #ifdef LA_IPSEC_DEBUG - if (tls_xfrm->options.iv_gen_disable == 1) - write_sa->w2.s.iv_at_cptr = ROC_IE_OT_TLS_IV_SRC_FROM_SA; + if (tls_xfrm->options.iv_gen_disable == 1) + write_sa->w2.s.iv_at_cptr = ROC_IE_OT_TLS_IV_SRC_FROM_SA; #else - if (tls_xfrm->options.iv_gen_disable) { - plt_err("Application provided IV is not supported"); - return -ENOTSUP; - } + if (tls_xfrm->options.iv_gen_disable) { + plt_err("Application provided IV is not supported"); + return -ENOTSUP; + } #endif + } rte_wmb(); @@ -599,20 +632,17 @@ cn10k_tls_read_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, sec_sess->iv_length = crypto_xfrm->auth.iv.length; } - if (sa_dptr->w2.s.version_select == ROC_IE_OT_TLS_VERSION_DTLS_12) - sec_sess->tls.hdr_len = 13; - else if (sa_dptr->w2.s.version_select == ROC_IE_OT_TLS_VERSION_TLS_12) - sec_sess->tls.hdr_len = 5; - sec_sess->proto = RTE_SECURITY_PROTOCOL_TLS_RECORD; - /* Enable mib counters */ - sa_dptr->w0.s.count_mib_bytes = 1; - sa_dptr->w0.s.count_mib_pkts = 1; - /* pre-populate CPT INST word 4 */ inst_w4.u64 = 0; - inst_w4.s.opcode_major = ROC_IE_OT_TLS_MAJOR_OP_RECORD_DEC | ROC_IE_OT_INPLACE_BIT; + if ((sa_dptr->w2.s.version_select == ROC_IE_OT_TLS_VERSION_TLS_12) || + (sa_dptr->w2.s.version_select == ROC_IE_OT_TLS_VERSION_DTLS_12)) { + inst_w4.s.opcode_major = ROC_IE_OT_TLS_MAJOR_OP_RECORD_DEC | ROC_IE_OT_INPLACE_BIT; + } else if (sa_dptr->w2.s.version_select == ROC_IE_OT_TLS_VERSION_TLS_13) { + inst_w4.s.opcode_major = + ROC_IE_OT_TLS13_MAJOR_OP_RECORD_DEC | ROC_IE_OT_INPLACE_BIT; + } sec_sess->inst.w4 = inst_w4.u64; sec_sess->inst.w7 = cpt_inst_w7_get(roc_cpt, read_sa); @@ -689,8 +719,13 @@ cn10k_tls_write_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, /* pre-populate CPT INST word 4 */ inst_w4.u64 = 0; - inst_w4.s.opcode_major = ROC_IE_OT_TLS_MAJOR_OP_RECORD_ENC | ROC_IE_OT_INPLACE_BIT; - + if ((sa_dptr->w2.s.version_select == ROC_IE_OT_TLS_VERSION_TLS_12) || + (sa_dptr->w2.s.version_select == ROC_IE_OT_TLS_VERSION_DTLS_12)) { + inst_w4.s.opcode_major = ROC_IE_OT_TLS_MAJOR_OP_RECORD_ENC | ROC_IE_OT_INPLACE_BIT; + } else if (sa_dptr->w2.s.version_select == ROC_IE_OT_TLS_VERSION_TLS_13) { + inst_w4.s.opcode_major = + ROC_IE_OT_TLS13_MAJOR_OP_RECORD_ENC | ROC_IE_OT_INPLACE_BIT; + } sec_sess->inst.w4 = inst_w4.u64; sec_sess->inst.w7 = cpt_inst_w7_get(roc_cpt, write_sa); From patchwork Thu Dec 21 12:35:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135452 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0561543750; Thu, 21 Dec 2023 13:38:32 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2913B42EE0; Thu, 21 Dec 2023 13:37:01 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 8433640A6F for ; Thu, 21 Dec 2023 13:36:55 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS38019305 for ; Thu, 21 Dec 2023 04:36:54 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=fQNGS+1vGqqlQDu1szKnqnSreAYUGYcKKcI3zYLgEHY=; b=cz5 J3g3K5WQ58wydmd4k/1PzBKW1OHKJEwdggVCE5p2PBVxhLHQHBPiSgoCW0o4twNW HxKKX6udXPrrITIRLn2rn31c7Eq+wfNIURwmcnwTygOlw2biwQZxZp4kd9yGMhSE 1xmGRO5ijQdkJUzMMi1bma/2Zs+Uhx2IRkvmudotloBNW1sIM0M8+4hxuwcn4d68 121cEQ522Cqa0W9qqS6wPKwVG2TqMHQ0wQeLHeIzSs1AvQZN2Eqke6+rzZ7lPPSe hNk20dBCUoROHx0DijsiY+4W7/KBSCMI2kJpsqH6DSZ3/zrm9i1TEGoN//4Hr26x tm6ZflPWq0+EjrLSJfA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0jq-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:54 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:51 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:51 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 521AD3F7073; Thu, 21 Dec 2023 04:36:48 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 23/24] crypto/cnxk: add TLS 1.3 capability Date: Thu, 21 Dec 2023 18:05:44 +0530 Message-ID: <20231221123545.510-24-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: Np7HFyx1qyh6JtiTb-z6CxbQ8bhK2Vf- X-Proofpoint-ORIG-GUID: Np7HFyx1qyh6JtiTb-z6CxbQ8bhK2Vf- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Add TLS 1.3 record read and write capability Signed-off-by: Vidya Sagar Velumuri --- doc/guides/rel_notes/release_24_03.rst | 4 +- .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 92 +++++++++++++++++++ 2 files changed, 94 insertions(+), 2 deletions(-) diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst index f5773bab5a..89110e0650 100644 --- a/doc/guides/rel_notes/release_24_03.rst +++ b/doc/guides/rel_notes/release_24_03.rst @@ -58,8 +58,8 @@ New Features * **Updated Marvell cnxk crypto driver.** * Added support for Rx inject in crypto_cn10k. - * Added support for TLS record processing in crypto_cn10k. Supports TLS 1.2 - and DTLS 1.2. + * Added support for TLS record processing in crypto_cn10k. Supports TLS 1.2, + DTLS 1.2 and TLS 1.3. * Added PMD API to allow raw submission of instructions to CPT. Removed Items diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index 73100377d9..db50de5d58 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -40,6 +40,16 @@ RTE_DIM(sec_tls12_caps_##name)); \ } while (0) +#define SEC_TLS13_CAPS_ADD(cnxk_caps, cur_pos, hw_caps, name) \ + do { \ + if ((hw_caps[CPT_ENG_TYPE_SE].name) || \ + (hw_caps[CPT_ENG_TYPE_IE].name) || \ + (hw_caps[CPT_ENG_TYPE_AE].name)) \ + sec_tls13_caps_add(cnxk_caps, cur_pos, \ + sec_tls13_caps_##name, \ + RTE_DIM(sec_tls13_caps_##name)); \ + } while (0) + static const struct rte_cryptodev_capabilities caps_mul[] = { { /* RSA */ .op = RTE_CRYPTO_OP_TYPE_ASYMMETRIC, @@ -1631,6 +1641,40 @@ static const struct rte_cryptodev_capabilities sec_tls12_caps_sha1_sha2[] = { }, }; +static const struct rte_cryptodev_capabilities sec_tls13_caps_aes[] = { + { /* AES GCM */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD, + {.aead = { + .algo = RTE_CRYPTO_AEAD_AES_GCM, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 16 + }, + .digest_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .aad_size = { + .min = 5, + .max = 5, + .increment = 0 + }, + .iv_size = { + .min = 0, + .max = 0, + .increment = 0 + } + }, } + }, } + }, +}; + + static const struct rte_security_capability sec_caps_templ[] = { { /* IPsec Lookaside Protocol ESP Tunnel Ingress */ .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, @@ -1760,6 +1804,26 @@ static const struct rte_security_capability sec_caps_templ[] = { }, .crypto_capabilities = NULL, }, + { /* TLS 1.3 Record Read */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_TLS_RECORD, + .tls_record = { + .ver = RTE_SECURITY_VERSION_TLS_1_3, + .type = RTE_SECURITY_TLS_SESS_TYPE_READ, + .ar_win_size = 0, + }, + .crypto_capabilities = NULL, + }, + { /* TLS 1.3 Record Write */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_TLS_RECORD, + .tls_record = { + .ver = RTE_SECURITY_VERSION_TLS_1_3, + .type = RTE_SECURITY_TLS_SESS_TYPE_WRITE, + .ar_win_size = 0, + }, + .crypto_capabilities = NULL, + }, { .action = RTE_SECURITY_ACTION_TYPE_NONE } @@ -2005,6 +2069,33 @@ sec_tls12_crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[], sec_tls12_caps_add(cnxk_caps, &cur_pos, caps_end, RTE_DIM(caps_end)); } +static void +sec_tls13_caps_limit_check(int *cur_pos, int nb_caps) +{ + PLT_VERIFY(*cur_pos + nb_caps <= CNXK_SEC_TLS_1_3_CRYPTO_MAX_CAPS); +} + +static void +sec_tls13_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos, + const struct rte_cryptodev_capabilities *caps, int nb_caps) +{ + sec_tls13_caps_limit_check(cur_pos, nb_caps); + + memcpy(&cnxk_caps[*cur_pos], caps, nb_caps * sizeof(caps[0])); + *cur_pos += nb_caps; +} + +static void +sec_tls13_crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[], + union cpt_eng_caps *hw_caps) +{ + int cur_pos = 0; + + SEC_TLS13_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, aes); + + sec_tls13_caps_add(cnxk_caps, &cur_pos, caps_end, RTE_DIM(caps_end)); +} + void cnxk_cpt_caps_populate(struct cnxk_cpt_vf *vf) { @@ -2016,6 +2107,7 @@ cnxk_cpt_caps_populate(struct cnxk_cpt_vf *vf) if (vf->cpt.hw_caps[CPT_ENG_TYPE_SE].tls) { sec_tls12_crypto_caps_populate(vf->sec_tls_1_2_crypto_caps, vf->cpt.hw_caps); sec_tls12_crypto_caps_populate(vf->sec_dtls_1_2_crypto_caps, vf->cpt.hw_caps); + sec_tls13_crypto_caps_populate(vf->sec_tls_1_3_crypto_caps, vf->cpt.hw_caps); } PLT_STATIC_ASSERT(RTE_DIM(sec_caps_templ) <= RTE_DIM(vf->sec_caps)); From patchwork Thu Dec 21 12:35:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 135454 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 09D3343750; Thu, 21 Dec 2023 13:38:47 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 87B3542EE4; Thu, 21 Dec 2023 13:37:03 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 001E942EC0 for ; Thu, 21 Dec 2023 13:36:55 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS3A019305 for ; Thu, 21 Dec 2023 04:36:55 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=xSH6zysELKxU8qenaBD+enUz6xsWOZ2Km2Fe7L9VYMw=; b=hpc l2FcEgUUuMPfHTSRoZ5FR25+m0B1M+70xSD3mNzPvYZgIeO4TnaGimfdDbYXhr9I HR7mNorLaYaDbKCsceUjG8F/lCKzBNmM594EPZr5N02I+CBaa4KCdcPA0V4x0sos NILKVldGkx/CrAyvLoA6GXSowcMGvPm0F5o1Hq5uHh6SCCySdhB55SUPU0rxeZjN ZcO+oarHaFqx5F4s3RpywsYpIXqJgdAMpyY3W+iuuWxruOECzl/E45UdMTcf6gza Zlcxh4M2nfNTuRPZ559SZQtUp2as725QIoPPyGgIQrJuJgN31KW9kKhJ+skSFDOQ I0TEpvaZzN0RrbCVPpg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0k0-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:55 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:53 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:53 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 08C993F7079; Thu, 21 Dec 2023 04:36:50 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Tejasree Kondoj , Jerin Jacob , Vidya Sagar Velumuri , Subject: [PATCH 24/24] crypto/cnxk: add CPT SG mode debug Date: Thu, 21 Dec 2023 18:05:45 +0530 Message-ID: <20231221123545.510-25-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: c239U0MNB_YgsRS6XT-1gLqAuvlyqbyi X-Proofpoint-ORIG-GUID: c239U0MNB_YgsRS6XT-1gLqAuvlyqbyi X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Tejasree Kondoj Adding CPT SG mode debug dump. Signed-off-by: Tejasree Kondoj --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 135 +++++++++++++++++++++- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 7 ++ 2 files changed, 141 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index c350371505..6cfcbafdcc 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -2,9 +2,10 @@ * Copyright(C) 2021 Marvell. */ -#include #include +#include #include +#include #include #include @@ -103,6 +104,104 @@ cpt_sec_ipsec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, return ret; } +#ifdef CPT_INST_DEBUG_ENABLE +static inline void +cpt_request_data_sgv2_mode_dump(uint8_t *in_buffer, bool glist, uint16_t components) +{ + struct roc_se_buf_ptr list_ptr[ROC_MAX_SG_CNT]; + const char *list = glist ? "glist" : "slist"; + struct roc_sg2list_comp *sg_ptr = NULL; + uint16_t list_cnt = 0; + char suffix[64]; + int i, j; + + sg_ptr = (void *)in_buffer; + for (i = 0; i < components; i++) { + for (j = 0; j < sg_ptr->u.s.valid_segs; j++) { + list_ptr[i * 3 + j].size = sg_ptr->u.s.len[j]; + list_ptr[i * 3 + j].vaddr = (void *)sg_ptr->ptr[j]; + list_ptr[i * 3 + j].vaddr = list_ptr[i * 3 + j].vaddr; + list_cnt++; + } + sg_ptr++; + } + + printf("Current %s: %u\n", list, list_cnt); + + for (i = 0; i < list_cnt; i++) { + snprintf(suffix, sizeof(suffix), "%s[%d]: vaddr 0x%" PRIx64 ", vaddr %p len %u", + list, i, (uint64_t)list_ptr[i].vaddr, list_ptr[i].vaddr, list_ptr[i].size); + rte_hexdump(stdout, suffix, list_ptr[i].vaddr, list_ptr[i].size); + } +} + +static inline void +cpt_request_data_sg_mode_dump(uint8_t *in_buffer, bool glist) +{ + struct roc_se_buf_ptr list_ptr[ROC_MAX_SG_CNT]; + const char *list = glist ? "glist" : "slist"; + struct roc_sglist_comp *sg_ptr = NULL; + uint16_t list_cnt, components; + char suffix[64]; + int i; + + sg_ptr = (void *)(in_buffer + 8); + list_cnt = rte_be_to_cpu_16((((uint16_t *)in_buffer)[2])); + if (!glist) { + components = list_cnt / 4; + if (list_cnt % 4) + components++; + sg_ptr += components; + list_cnt = rte_be_to_cpu_16((((uint16_t *)in_buffer)[3])); + } + + printf("Current %s: %u\n", list, list_cnt); + components = list_cnt / 4; + for (i = 0; i < components; i++) { + list_ptr[i * 4 + 0].size = rte_be_to_cpu_16(sg_ptr->u.s.len[0]); + list_ptr[i * 4 + 1].size = rte_be_to_cpu_16(sg_ptr->u.s.len[1]); + list_ptr[i * 4 + 2].size = rte_be_to_cpu_16(sg_ptr->u.s.len[2]); + list_ptr[i * 4 + 3].size = rte_be_to_cpu_16(sg_ptr->u.s.len[3]); + list_ptr[i * 4 + 0].vaddr = (void *)rte_be_to_cpu_64(sg_ptr->ptr[0]); + list_ptr[i * 4 + 1].vaddr = (void *)rte_be_to_cpu_64(sg_ptr->ptr[1]); + list_ptr[i * 4 + 2].vaddr = (void *)rte_be_to_cpu_64(sg_ptr->ptr[2]); + list_ptr[i * 4 + 3].vaddr = (void *)rte_be_to_cpu_64(sg_ptr->ptr[3]); + list_ptr[i * 4 + 0].vaddr = list_ptr[i * 4 + 0].vaddr; + list_ptr[i * 4 + 1].vaddr = list_ptr[i * 4 + 1].vaddr; + list_ptr[i * 4 + 2].vaddr = list_ptr[i * 4 + 2].vaddr; + list_ptr[i * 4 + 3].vaddr = list_ptr[i * 4 + 3].vaddr; + sg_ptr++; + } + + components = list_cnt % 4; + switch (components) { + case 3: + list_ptr[i * 4 + 2].size = rte_be_to_cpu_16(sg_ptr->u.s.len[2]); + list_ptr[i * 4 + 2].vaddr = (void *)rte_be_to_cpu_64(sg_ptr->ptr[2]); + list_ptr[i * 4 + 2].vaddr = list_ptr[i * 4 + 2].vaddr; + /* FALLTHROUGH */ + case 2: + list_ptr[i * 4 + 1].size = rte_be_to_cpu_16(sg_ptr->u.s.len[1]); + list_ptr[i * 4 + 1].vaddr = (void *)rte_be_to_cpu_64(sg_ptr->ptr[1]); + list_ptr[i * 4 + 1].vaddr = list_ptr[i * 4 + 1].vaddr; + /* FALLTHROUGH */ + case 1: + list_ptr[i * 4 + 0].size = rte_be_to_cpu_16(sg_ptr->u.s.len[0]); + list_ptr[i * 4 + 0].vaddr = (void *)rte_be_to_cpu_64(sg_ptr->ptr[0]); + list_ptr[i * 4 + 0].vaddr = list_ptr[i * 4 + 0].vaddr; + break; + default: + break; + } + + for (i = 0; i < list_cnt; i++) { + snprintf(suffix, sizeof(suffix), "%s[%d]: vaddr 0x%" PRIx64 ", vaddr %p len %u", + list, i, (uint64_t)list_ptr[i].vaddr, list_ptr[i].vaddr, list_ptr[i].size); + rte_hexdump(stdout, suffix, list_ptr[i].vaddr, list_ptr[i].size); + } +} +#endif + static __rte_always_inline int __rte_hot cpt_sec_tls_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cn10k_sec_session *sess, struct cpt_inst_s *inst, @@ -205,6 +304,31 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], struct inst[0].w7.u64 = w7; +#ifdef CPT_INST_DEBUG_ENABLE + infl_req->dptr = (uint8_t *)inst[0].dptr; + infl_req->rptr = (uint8_t *)inst[0].rptr; + infl_req->is_sg_ver2 = is_sg_ver2; + infl_req->scatter_sz = inst[0].w6.s.scatter_sz; + infl_req->opcode_major = inst[0].w4.s.opcode_major; + + rte_hexdump(stdout, "cptr", (void *)(uint64_t)inst[0].w7.s.cptr, 128); + printf("major opcode:%d\n", inst[0].w4.s.opcode_major); + printf("minor opcode:%d\n", inst[0].w4.s.opcode_minor); + printf("param1:%d\n", inst[0].w4.s.param1); + printf("param2:%d\n", inst[0].w4.s.param2); + printf("dlen:%d\n", inst[0].w4.s.dlen); + + if (is_sg_ver2) { + cpt_request_data_sgv2_mode_dump((void *)inst[0].dptr, 1, inst[0].w5.s.gather_sz); + cpt_request_data_sgv2_mode_dump((void *)inst[0].rptr, 0, inst[0].w6.s.scatter_sz); + } else { + if (infl_req->opcode_major >> 7) { + cpt_request_data_sg_mode_dump((void *)inst[0].dptr, 1); + cpt_request_data_sg_mode_dump((void *)inst[0].dptr, 0); + } + } +#endif + return 1; } @@ -935,6 +1059,15 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop } if (likely(compcode == CPT_COMP_GOOD)) { +#ifdef CPT_INST_DEBUG_ENABLE + if (infl_req->is_sg_ver2) + cpt_request_data_sgv2_mode_dump(infl_req->rptr, 0, infl_req->scatter_sz); + else { + if (infl_req->opcode_major >> 7) + cpt_request_data_sg_mode_dump(infl_req->dptr, 0); + } +#endif + if (unlikely(uc_compcode)) { if (uc_compcode == ROC_SE_ERR_GC_ICV_MISCOMPARE) cop->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h index c6bb8023ea..e7bba25cb8 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h @@ -51,6 +51,13 @@ struct cpt_inflight_req { }; void *mdata; uint8_t op_flags; +#ifdef CPT_INST_DEBUG_ENABLE + uint8_t scatter_sz; + uint8_t opcode_major; + uint8_t is_sg_ver2; + uint8_t *dptr; + uint8_t *rptr; +#endif void *qp; } __rte_aligned(ROC_ALIGN);