From patchwork Mon Jun 24 06:23:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141512 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3C517454A7; Mon, 24 Jun 2024 08:31:44 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 14C2540EE3; Mon, 24 Jun 2024 08:31:34 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 4B8CF4027B for ; Mon, 24 Jun 2024 08:24:10 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NMe8IS025560 for ; Sun, 23 Jun 2024 23:24:09 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=4 S4ksnydXUEsJuuAdvroPu1vA+FAxc9EZ2KxLR9G0eU=; b=VF3A2m3SNrmDBjURy +Lz8iWaCIxp53c+ckn0Wml4L3qfw1pJZK2X3qSbPnPNfoPBrHceLU4vxraIAqq74 6oO/ZxMsyvBIlFbulCprzEGyJkrgRE+bUDd88egtCuIscgOcmVjbJVRDq2DH8hij oJFTwgO7Ra+K1wU2PYlbWzVmDx2SDwIeNdr42VBHD84lURspSoXXO+F6AU4ZNNpD 9ia1Y8FnfufOmME4MHEzzAbhFXvep/VBFUL5LK+n4szORP9Zi7pMFf21/LEgTujF Nd5hV3jVr1MnFqXO8YP6nHsNwso05LycaK/8Ci5f5U/CqSE8CVrqlCJq8HkTC9/1 pAWmw== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ywx4gbp6w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:09 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:08 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:08 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 05A853F7040; Sun, 23 Jun 2024 23:24:04 -0700 (PDT) From: Aakash Sasidharan To: Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Harman Kalra CC: , , , , , Subject: [PATCH v2 01/12] common/cnxk: add comments to denote skipped entries Date: Mon, 24 Jun 2024 11:53:50 +0530 Message-ID: <20240624062401.4143606-2-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: vNIAcSMriJEQBwwVWoNo4rFFHSrDu0_b X-Proofpoint-ORIG-GUID: vNIAcSMriJEQBwwVWoNo4rFFHSrDu0_b X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Add comments to denote unused table entries. Signed-off-by: Anoob Joseph --- drivers/common/cnxk/roc_ae.c | 6 +++--- drivers/common/cnxk/roc_ae_fpm_tables.c | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/common/cnxk/roc_ae.c b/drivers/common/cnxk/roc_ae.c index e6a013d7c4..7ef0efe2b3 100644 --- a/drivers/common/cnxk/roc_ae.c +++ b/drivers/common/cnxk/roc_ae.c @@ -151,9 +151,9 @@ const struct roc_ae_ec_group ae_ec_grp[ROC_AE_EC_ID_PMAX] = { 0x3F, 0x00}, .length = 66}, }, - {}, - {}, - {}, + { /* ROC_AE_EC_ID_P160 */ }, + { /* ROC_AE_EC_ID_P320 */ }, + { /* ROC_AE_EC_ID_P512 */ }, { .prime = {.data = {0xFF, 0xFF, 0xFF, 0xFE, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, diff --git a/drivers/common/cnxk/roc_ae_fpm_tables.c b/drivers/common/cnxk/roc_ae_fpm_tables.c index ead3128e7f..942657b56a 100644 --- a/drivers/common/cnxk/roc_ae_fpm_tables.c +++ b/drivers/common/cnxk/roc_ae_fpm_tables.c @@ -1261,9 +1261,9 @@ const struct ae_fpm_entry ae_fpm_tbl_scalar[ROC_AE_EC_ID_PMAX] = { .data = ae_fpm_tbl_p521, .len = sizeof(ae_fpm_tbl_p521) }, - {}, - {}, - {}, + { /* ROC_AE_EC_ID_P160 */ }, + { /* ROC_AE_EC_ID_P320 */ }, + { /* ROC_AE_EC_ID_P512 */ }, { .data = ae_fpm_tbl_p256_sm2, .len = sizeof(ae_fpm_tbl_p256_sm2) From patchwork Mon Jun 24 06:23:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141513 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BA036454A7; Mon, 24 Jun 2024 08:31:50 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C84ED4068A; Mon, 24 Jun 2024 08:31:35 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 584184027B for ; Mon, 24 Jun 2024 08:24:13 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NMe8IT025560 for ; Sun, 23 Jun 2024 23:24:12 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=q l71vZWZrf4/CUjtOHgSPn0dXYV2jm6KNWrMqMtwvqg=; b=hJE1p4mFcSo3l8CmV 6V/AubjtTOEqzValy/1cTabkW+Ztp0O6CPr3GRU/Wf5pKurTxfbyqGi7KkPp1Nzp T5HNwCqLXbHc0g9Fsn0xGCMGdz1xS2KoGM9QVLhpglIjNqRPkKy0keeOMddhYOyu 5AMF8XofEyE2lFcJwLiPDtlZ359gO6xFdx61EucTYVuWH+/uPohiufaRhvggO68L M/09NaDYGmMDHSND+GLidIeNXjFMXZ9GfpSDJKy+WdvDiDISVSAFDRLcxdxjftNI 0f2ZUnWa8AFAvqQPbHDZiX4qgb4cAycxS+x0txoGqEyGFZeyRkcaO64/iSpUjdkE oUxsw== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ywx4gbp70-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:12 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:11 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:11 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id DCCA93F7040; Sun, 23 Jun 2024 23:24:08 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v2 02/12] crypto/cnxk: update version map file with PMD APIs Date: Mon, 24 Jun 2024 11:53:51 +0530 Message-ID: <20240624062401.4143606-3-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: DQVZheLSIJv5E19r-QVK_vYL0TQ1tG45 X-Proofpoint-ORIG-GUID: DQVZheLSIJv5E19r-QVK_vYL0TQ1tG45 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Update version map with details of PMD APIs added. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h | 2 ++ drivers/crypto/cnxk/version.map | 8 ++++++++ 2 files changed, 10 insertions(+) diff --git a/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h b/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h index 8b0a5ba0f2..eab1243065 100644 --- a/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h +++ b/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h @@ -23,6 +23,7 @@ * @return * Pointer to queue pair structure that would be the input to submit APIs. */ +__rte_experimental void *rte_pmd_cnxk_crypto_qptr_get(uint8_t dev_id, uint16_t qp_id); /** @@ -41,6 +42,7 @@ void *rte_pmd_cnxk_crypto_qptr_get(uint8_t dev_id, uint16_t qp_id); * @param nb_inst * Number of instructions. */ +__rte_experimental void rte_pmd_cnxk_crypto_submit(void *qptr, void *inst, uint16_t nb_inst); #endif /* _PMD_CNXK_CRYPTO_H_ */ diff --git a/drivers/crypto/cnxk/version.map b/drivers/crypto/cnxk/version.map index 5789a6bfc9..7a77607774 100644 --- a/drivers/crypto/cnxk/version.map +++ b/drivers/crypto/cnxk/version.map @@ -1,3 +1,11 @@ +EXPERIMENTAL { + global: + + # added in 24.03 + rte_pmd_cnxk_crypto_submit; + rte_pmd_cnxk_crypto_qptr_get; +}; + INTERNAL { global: From patchwork Mon Jun 24 06:23:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141514 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 80088454A7; Mon, 24 Jun 2024 08:32:00 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 28FF14042F; Mon, 24 Jun 2024 08:31:37 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 406CD4027B for ; Mon, 24 Jun 2024 08:24:17 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NMe8IU025560 for ; Sun, 23 Jun 2024 23:24:16 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=F 9Qc9z2CHV/xfwfzvr+jja5tzDz5xLn+Og8k8z9Zrfo=; b=Atxa0HF5mjnaudMCk YHonpfZms5+GkTV8YUBkSJ6aE4ZN14mDcK5euuQoi9VFar2gC2GyEHo6tx2QnMkP U42YNdqV2WkfsPBu4srNMXvNsoumB71Jx4Psn4g4rFxY9RZFCU6C9DLTS/b59aa2 L6KRUMu03UcSx/QEEn0B3Nr+JmEk8evenTLSqUx5wN4+CnNye3ET4zFHsgQBCTUc ZJGqN+2/mBtRVXCBHv6zk4b1H6DKSaR5jqI42Ap3M0ad2/owMHRoJTpueVqLbzDw oM3WEu+9mKimISVQWEpzvBdOmTQUQZEoX0WLMkvZ2Z1xE/0o5hCatJTemeb8ZZGs m4Xyg== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ywx4gbp74-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:16 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:15 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:15 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 0B01B3F7040; Sun, 23 Jun 2024 23:24:11 -0700 (PDT) From: Aakash Sasidharan To: Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Harman Kalra CC: , , , , , Subject: [PATCH v2 03/12] common/cnxk: make inline dev PF func get as idev API Date: Mon, 24 Jun 2024 11:53:52 +0530 Message-ID: <20240624062401.4143606-4-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: x9xwWd5IzIghug9nX9F4xqeiU6bdjcM0 X-Proofpoint-ORIG-GUID: x9xwWd5IzIghug9nX9F4xqeiU6bdjcM0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Inline PF FUNC would be required to set SSO_PF_FUNC in the instruction for cryptodev Rx inject. Move the API to idev to allow usage of the same. Signed-off-by: Anoob Joseph --- drivers/common/cnxk/roc_idev.c | 6 ++++++ drivers/common/cnxk/roc_idev.h | 2 ++ drivers/common/cnxk/roc_nix_inl.h | 1 - drivers/common/cnxk/roc_nix_inl_dev.c | 6 ------ drivers/common/cnxk/version.map | 2 +- drivers/net/cnxk/cn10k_ethdev_sec.c | 2 +- drivers/net/cnxk/cnxk_ethdev_telemetry.c | 3 +-- 7 files changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/common/cnxk/roc_idev.c b/drivers/common/cnxk/roc_idev.c index d0307c666c..0778d51d1e 100644 --- a/drivers/common/cnxk/roc_idev.c +++ b/drivers/common/cnxk/roc_idev.c @@ -374,3 +374,9 @@ roc_idev_nix_rx_chan_set(uint16_t port, uint16_t chan) if (idev != NULL && port < PLT_MAX_ETHPORTS) __atomic_store_n(&idev->inl_rx_inj_cfg.chan[port], chan, __ATOMIC_RELEASE); } + +uint16_t +roc_idev_nix_inl_dev_pffunc_get(void) +{ + return nix_inl_dev_pffunc_get(); +} diff --git a/drivers/common/cnxk/roc_idev.h b/drivers/common/cnxk/roc_idev.h index 00664eaed6..fc0f7db54e 100644 --- a/drivers/common/cnxk/roc_idev.h +++ b/drivers/common/cnxk/roc_idev.h @@ -27,4 +27,6 @@ uint8_t __roc_api roc_idev_nix_rx_inject_get(uint16_t port); void __roc_api roc_idev_nix_rx_inject_set(uint16_t port, uint8_t enable); uint16_t *__roc_api roc_idev_nix_rx_chan_base_get(void); void __roc_api roc_idev_nix_rx_chan_set(uint16_t port, uint16_t chan); + +uint16_t __roc_api roc_idev_nix_inl_dev_pffunc_get(void); #endif /* _ROC_IDEV_H_ */ diff --git a/drivers/common/cnxk/roc_nix_inl.h b/drivers/common/cnxk/roc_nix_inl.h index ab0965e512..1a4bf8808c 100644 --- a/drivers/common/cnxk/roc_nix_inl.h +++ b/drivers/common/cnxk/roc_nix_inl.h @@ -112,7 +112,6 @@ void __roc_api roc_nix_inl_dev_lock(void); void __roc_api roc_nix_inl_dev_unlock(void); int __roc_api roc_nix_inl_dev_xaq_realloc(uint64_t aura_handle); int __roc_api roc_nix_inl_dev_stats_get(struct roc_nix_stats *stats); -uint16_t __roc_api roc_nix_inl_dev_pffunc_get(void); int __roc_api roc_nix_inl_dev_cpt_setup(bool use_inl_dev_sso); int __roc_api roc_nix_inl_dev_cpt_release(void); bool __roc_api roc_nix_inl_dev_is_multi_channel(void); diff --git a/drivers/common/cnxk/roc_nix_inl_dev.c b/drivers/common/cnxk/roc_nix_inl_dev.c index 60e6a43033..e2bbe3a67b 100644 --- a/drivers/common/cnxk/roc_nix_inl_dev.c +++ b/drivers/common/cnxk/roc_nix_inl_dev.c @@ -34,12 +34,6 @@ nix_inl_dev_pffunc_get(void) return 0; } -uint16_t -roc_nix_inl_dev_pffunc_get(void) -{ - return nix_inl_dev_pffunc_get(); -} - static void nix_inl_selftest_work_cb(uint64_t *gw, void *args, uint32_t soft_exp_event) { diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index eac2ea9ff8..f98738d07e 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -112,6 +112,7 @@ INTERNAL { roc_idev_npa_nix_get; roc_idev_num_lmtlines_get; roc_idev_nix_inl_meta_aura_get; + roc_idev_nix_inl_dev_pffunc_get; roc_idev_nix_list_get; roc_idev_nix_rx_chan_base_get; roc_idev_nix_rx_chan_set; @@ -244,7 +245,6 @@ INTERNAL { roc_nix_inl_dev_is_probed; roc_nix_inl_dev_stats_get; roc_nix_inl_dev_lock; - roc_nix_inl_dev_pffunc_get; roc_nix_inl_dev_rq; roc_nix_inl_dev_rq_get; roc_nix_inl_dev_rq_put; diff --git a/drivers/net/cnxk/cn10k_ethdev_sec.c b/drivers/net/cnxk/cn10k_ethdev_sec.c index b8b0da5ea9..5e509e97d4 100644 --- a/drivers/net/cnxk/cn10k_ethdev_sec.c +++ b/drivers/net/cnxk/cn10k_ethdev_sec.c @@ -1360,7 +1360,7 @@ cn10k_eth_sec_rx_inject_config(void *device, uint16_t port_id, bool enable) inj_cfg->io_addr = inl_lf->io_addr; inj_cfg->lmt_base = nix->lmt_base; channel = roc_nix_get_base_chan(nix); - pf_func = roc_nix_inl_dev_pffunc_get(); + pf_func = roc_idev_nix_inl_dev_pffunc_get(); inj_cfg->cmd_w0 = pf_func << 48 | inj_match_id << 32 | channel << 4; return 0; diff --git a/drivers/net/cnxk/cnxk_ethdev_telemetry.c b/drivers/net/cnxk/cnxk_ethdev_telemetry.c index 3027ca4735..a1958185f2 100644 --- a/drivers/net/cnxk/cnxk_ethdev_telemetry.c +++ b/drivers/net/cnxk/cnxk_ethdev_telemetry.c @@ -65,8 +65,7 @@ ethdev_tel_handle_info(const char *cmd __rte_unused, info = ð_info.info; dev = cnxk_eth_pmd_priv(eth_dev); if (dev) { - info->inl_dev_pf_func = - roc_nix_inl_dev_pffunc_get(); + info->inl_dev_pf_func = roc_idev_nix_inl_dev_pffunc_get(); info->pf_func = roc_nix_get_pf_func(&dev->nix); info->max_mac_entries = dev->max_mac_entries; info->dmac_filter_ena = dev->dmac_filter_enable; From patchwork Mon Jun 24 06:23:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141515 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0C5CC454A7; Mon, 24 Jun 2024 08:32:09 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5AC0640676; Mon, 24 Jun 2024 08:31:38 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 6A6364027B for ; Mon, 24 Jun 2024 08:24:20 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NMoZ8Y009335 for ; Sun, 23 Jun 2024 23:24:19 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=d sJSLjMQaW8kwSznLyZHSS2x7yGQ1Lku7HA7iDX/374=; b=StrrXoKdY3Idb63af ViKXVqyaRRf81nqln6Fnu/XffJA64LX8LYWm/43H1LqY4P7stcaYCUDef/w3ifLK P8urWmSRRLSKeU6cKQx2e0aOqiKc2O0TiM1VmSZqwrY/DtVaCeMcPmna4VjZ+1Pv 5t/5GroBP7Exs1ko6n53mRldGPsEYB4dF4ZJNl4pHVqpbtWH5oYB01m9XFBhWbb2 5hGAfC+VRC+LUor5nSyu42nwXBfcUHuwopiienPlp1Uiq6NJ6YUE3+Mu1o1+mApO TmHB76H9usc5dWCZPd8+HJVGfUODhEEFkkhQLdWEamPhekLLagyXtFgRGbe5/gRn lRF5Q== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ywx4gbp76-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:19 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:18 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:18 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id D60673F704A; Sun, 23 Jun 2024 23:24:15 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v2 04/12] crypto/cnxk: add flow control in Rx inject path Date: Mon, 24 Jun 2024 11:53:53 +0530 Message-ID: <20240624062401.4143606-5-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: B8nkDgMtSco-6z1vq6L49wUSmjzjIoSW X-Proofpoint-ORIG-GUID: B8nkDgMtSco-6z1vq6L49wUSmjzjIoSW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Add flow control in Rx inject path to avoid over submission to CPT. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 720b756001..9f1c074925 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -1400,8 +1400,10 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, struct rte_cryptodev *cdev = dev; union cpt_res_s *hw_res = NULL; struct cpt_inst_s *inst; + union cpt_fc_write_s fc; struct cnxk_cpt_vf *vf; struct rte_mbuf *m; + uint64_t *fc_addr; uint64_t dptr; int i; @@ -1413,13 +1415,24 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, lmt_base = vf->rx_inj_lmtline.lmt_base; io_addr = vf->rx_inj_lmtline.io_addr; + fc_addr = vf->rx_inj_lmtline.fc_addr; ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); pf_func = vf->rx_inj_pf_func; + const uint32_t fc_thresh = vf->rx_inj_lmtline.fc_thresh; + again: + fc.u64[0] = + rte_atomic_load_explicit((RTE_ATOMIC(uint64_t) *)fc_addr, rte_memory_order_relaxed); inst = (struct cpt_inst_s *)lmt_base; - for (i = 0; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_pkts); i++) { + + i = 0; + + if (unlikely(fc.s.qsize > fc_thresh)) + goto exit; + + for (; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_pkts); i++) { m = pkts[i]; sec_sess = (struct cn10k_sec_session *)sess[i]; @@ -1487,6 +1500,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, goto again; } +exit: return count + i; } From patchwork Mon Jun 24 06:23:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141516 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DDB31454A7; Mon, 24 Jun 2024 08:32:17 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6F3E240691; Mon, 24 Jun 2024 08:31:39 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 88ADC4027B for ; Mon, 24 Jun 2024 08:24:23 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NMnWjR008017 for ; Sun, 23 Jun 2024 23:24:23 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=a TTQcuQo2SoOp91Df3kBi0eMhRJCylEaSuEIs4wUckc=; b=UHP6RQycXK3q84h22 +pIk6BgLF1XmABsIrPO0IBCZxHoRla51E9670I61TZS1myX/vVrKMfQqo29ZSiZc tUk6sJ0BsDQ+vnfgbfluLA7x9Rv4VhDA1HLr1JlcPtenlcKYQn8iAthbHDM2yZRR KUWbOjpDIPNJVAHPDmj2NRd6MkGL0kaStF4oUTx0Zt0JeDZWewnng3f/l3/B7A5h dzWmwsxS77XQnf+AbbnmeEonNaS80i6mw9ZOLsZGBn/C9OVzk+DZ1dJZ0LwxqMlY Ks828qCN6/qR5Kp05vMILJJniASu21HNRvL4WEnBDpPq/NUqf/q1GWN9i8FURuNs VbQyw== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ywx4gbp78-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:22 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:21 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:21 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id F3BA53F7040; Sun, 23 Jun 2024 23:24:18 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v2 05/12] crypto/cnxk: use SSO PF func of inline device in inst Date: Mon, 24 Jun 2024 11:53:54 +0530 Message-ID: <20240624062401.4143606-6-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: _FKqBnwDl2UwrxsFjgsHLAHsNwgh1-Sh X-Proofpoint-ORIG-GUID: _FKqBnwDl2UwrxsFjgsHLAHsNwgh1-Sh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph RVU PF FUNC of the CPT LF need not be set as the hardware would determine that. Instead SSO PF FUNC need to be set as inline device so that critical errors would reach inline device. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 2 +- drivers/crypto/cnxk/cnxk_cryptodev.h | 2 +- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 9f1c074925..f2980399c5 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -1418,7 +1418,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, fc_addr = vf->rx_inj_lmtline.fc_addr; ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); - pf_func = vf->rx_inj_pf_func; + pf_func = vf->rx_inj_sso_pf_func; const uint32_t fc_thresh = vf->rx_inj_lmtline.fc_thresh; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index fffc4a47b4..4000e84a7e 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -22,7 +22,7 @@ */ struct cnxk_cpt_vf { struct roc_cpt_lmtline rx_inj_lmtline; - uint16_t rx_inj_pf_func; + uint16_t rx_inj_sso_pf_func; uint16_t *rx_chan_base; struct roc_cpt cpt; struct rte_cryptodev_capabilities crypto_caps[CNXK_CPT_MAX_CAPS]; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index d7f5780637..51369309c5 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -483,7 +483,7 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, goto exit; } - vf->rx_inj_pf_func = qp->lf.pf_func; + vf->rx_inj_sso_pf_func = roc_idev_nix_inl_dev_pffunc_get(); /* Block the queue for other submissions */ qp->pend_q.pq_mask = 0; From patchwork Mon Jun 24 06:23:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141517 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C01A4454A7; Mon, 24 Jun 2024 08:32:25 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9B1AD40A87; Mon, 24 Jun 2024 08:31:40 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id C34B54027B for ; Mon, 24 Jun 2024 08:24:26 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NMe8IW025560 for ; Sun, 23 Jun 2024 23:24:26 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=7 wf5vZFMdC7Jnn+O4xZg8c9VDxVNddLoGsazSIkGqyg=; b=XLWS3kiIg1WWPWNqE Pl1p9amyiaxG4gRZI6Ibv0LamFafDEp9q0MxryWGknXnp16VR2QX2yqBkRuJv6R3 mkVya12YIFBGtV/J7VFznzPKpjcegUPZ6kdTHRvhGrr13Nq0LWSNdielZeeK0ARz EqxByt6CJbxGYZZ3fDMVRs2O+2w1EoQehV0rnzB5lOBXt+5t9HliSOnkdJOHpmLA pZDxwZUPCn7DH/wVKqOPmBbqp2iWnu8oqmA/d7WV4K3edw6MeyCw8zgCWBn/rhT3 OdmOkVfVjM5gkGdFfuc+ekaCWuAbsNAW/jud/6LuR7PIKrVX2HCIaYzQqMKb2tOQ twq1w== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ywx4gbp7g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:26 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:24 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:24 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 19BB23F7040; Sun, 23 Jun 2024 23:24:21 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v2 06/12] crypto/cnxk: use NEON for Rx inject inst preparation Date: Mon, 24 Jun 2024 11:53:55 +0530 Message-ID: <20240624062401.4143606-7-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: SUtQIhRHTzP81GKq2GVrEZTymrs94YR0 X-Proofpoint-ORIG-GUID: SUtQIhRHTzP81GKq2GVrEZTymrs94YR0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Use NEON instructions for Rx inject instruction preparation. Signed-off-by: Anoob Joseph Signed-off-by: Aakash Sasidharan --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 62 +++++++++++++++++------ 1 file changed, 46 insertions(+), 16 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index f2980399c5..446a3c3fd8 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -7,6 +7,7 @@ #include #include #include +#include #include @@ -1390,15 +1391,17 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return i; } +#if defined(RTE_ARCH_ARM64) uint16_t __rte_hot cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, struct rte_security_session **sess, uint16_t nb_pkts) { - uint16_t l2_len, pf_func, lmt_id, count = 0; - uint64_t lmt_base, lmt_arg, io_addr; + uint64_t lmt_base, lmt_arg, io_addr, u64_0, u64_1, l2_len, pf_func; + uint64x2_t inst_01, inst_23, inst_45, inst_67; struct cn10k_sec_session *sec_sess; struct rte_cryptodev *cdev = dev; union cpt_res_s *hw_res = NULL; + uint16_t lmt_id, count = 0; struct cpt_inst_s *inst; union cpt_fc_write_s fc; struct cnxk_cpt_vf *vf; @@ -1456,26 +1459,41 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, hw_res = RTE_PTR_ALIGN_CEIL(hw_res, 16); /* Prepare CPT instruction */ - inst->w0.u64 = 0; - inst->w2.u64 = 0; - inst->w2.s.rvu_pf_func = pf_func; - inst->w3.u64 = (((uint64_t)m + sizeof(struct rte_mbuf)) >> 3) << 3 | 1; - inst->w4.u64 = sec_sess->inst.w4 | (rte_pktmbuf_pkt_len(m)); + /* Word 0 and 1 */ + inst_01 = vdupq_n_u64(0); + u64_0 = pf_func << 48 | *(vf->rx_chan_base + m->port) << 4 | (l2_len - 2) << 24 | + l2_len << 16; + inst_01 = vsetq_lane_u64(u64_0, inst_01, 0); + inst_01 = vsetq_lane_u64((uint64_t)hw_res, inst_01, 1); + vst1q_u64(&inst->w0.u64, inst_01); + + /* Word 2 and 3 */ + inst_23 = vdupq_n_u64(0); + u64_1 = (((uint64_t)m + sizeof(struct rte_mbuf)) >> 3) << 3 | 1; + inst_23 = vsetq_lane_u64(u64_1, inst_23, 1); + vst1q_u64(&inst->w2.u64, inst_23); + + /* Word 4 and 5 */ + inst_45 = vdupq_n_u64(0); + u64_0 = sec_sess->inst.w4 | (rte_pktmbuf_pkt_len(m)); + inst_45 = vsetq_lane_u64(u64_0, inst_45, 0); dptr = (uint64_t)rte_pktmbuf_iova(m); - inst->dptr = dptr; - inst->rptr = dptr; - - inst->w0.hw_s.chan = *(vf->rx_chan_base + m->port); - inst->w0.hw_s.l2_len = l2_len; - inst->w0.hw_s.et_offset = l2_len - 2; + u64_1 = dptr; + inst_45 = vsetq_lane_u64(u64_1, inst_45, 1); + vst1q_u64(&inst->w4.u64, inst_45); + + /* Word 6 and 7 */ + inst_67 = vdupq_n_u64(0); + u64_0 = dptr; + u64_1 = sec_sess->inst.w7; + inst_67 = vsetq_lane_u64(u64_0, inst_67, 0); + inst_67 = vsetq_lane_u64(u64_1, inst_67, 1); + vst1q_u64(&inst->w6.u64, inst_67); - inst->res_addr = (uint64_t)hw_res; rte_atomic_store_explicit((unsigned long __rte_atomic *)&hw_res->u64[0], res.u64[0], rte_memory_order_relaxed); - inst->w7.u64 = sec_sess->inst.w7; - inst += 2; } @@ -1503,6 +1521,18 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, exit: return count + i; } +#else +uint16_t __rte_hot +cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, + struct rte_security_session **sess, uint16_t nb_pkts) +{ + RTE_SET_USED(dev); + RTE_SET_USED(pkts); + RTE_SET_USED(sess); + RTE_SET_USED(nb_pkts); + return 0; +} +#endif void cn10k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev, struct cnxk_cpt_vf *vf) From patchwork Mon Jun 24 06:23:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141518 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6FF58454A7; Mon, 24 Jun 2024 08:32:32 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D252E40DD5; Mon, 24 Jun 2024 08:31:41 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 7CC364027B for ; Mon, 24 Jun 2024 08:24:30 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NLvlf6014517 for ; Sun, 23 Jun 2024 23:24:29 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=i xeEVv3Ky6YI65jSVwf5D7AEFjowlkSuAcmvb3RtuF4=; b=ZEkWQjhjBruV48YAi 6IIEWDleMq8v46877siw/jdsa2CtZtJBXkB4LbIR8MbTBXJH8QiHoY7TRjnLGneN FIUtVSduduPFF7YHgczfdUUJAbicYPYnt17MCHxuUV7fdv3Btc7qfR+hLS/9/OEH TZd8c7wfu/v1H6/QxAcQ2iImbks+akaJ5t+V42UkJD5kOvsuQCNm2pyquN5WPa79 51k9JAUvXig9MJmXVp3DTftlX75TUbnnk+fQhFXLfsKl7f0W61mj6NcqBPq7vTUA oidbzjNR+Owciu7TPYTfSq8oWoNnabVF0XTDl1esnO4ln9yOWO6/5lUrsrFVeGKT QyJhQ== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3yxcgtjbqh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:29 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:28 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:28 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 4ECF33F7040; Sun, 23 Jun 2024 23:24:25 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v2 07/12] crypto/cnxk: remove init of CPT result field in packet Date: Mon, 24 Jun 2024 11:53:56 +0530 Message-ID: <20240624062401.4143606-8-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 1QnnwhNnVQWkLu3NZu3fAcofjuN0CEV4 X-Proofpoint-ORIG-GUID: 1QnnwhNnVQWkLu3NZu3fAcofjuN0CEV4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph The packet would be posted to CPT only when there is a valid result. Skip setting of the same. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 446a3c3fd8..90ca9eec27 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -1410,10 +1410,6 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, uint64_t dptr; int i; - const union cpt_res_s res = { - .cn10k.compcode = CPT_COMP_NOT_DONE, - }; - vf = cdev->data->dev_private; lmt_base = vf->rx_inj_lmtline.lmt_base; @@ -1491,9 +1487,6 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, inst_67 = vsetq_lane_u64(u64_1, inst_67, 1); vst1q_u64(&inst->w6.u64, inst_67); - rte_atomic_store_explicit((unsigned long __rte_atomic *)&hw_res->u64[0], res.u64[0], - rte_memory_order_relaxed); - inst += 2; } From patchwork Mon Jun 24 06:23:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141519 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B41DC454A7; Mon, 24 Jun 2024 08:32:38 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F2F7E40DF6; Mon, 24 Jun 2024 08:31:42 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 57B374027B for ; Mon, 24 Jun 2024 08:24:34 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NLvu61026887 for ; Sun, 23 Jun 2024 23:24:33 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=8 e8ObmXQWboziiL9CPbn9HehxyNIbDji9FqikHB/iQU=; b=J6PRkYBmgiVxQV/f5 52jyYn4yrhoHEbEImeTIuxF5DeDi5GB3MAm+FQIp/fCHS+a3jXaxV7+jSW7AIfZ1 8nyTAbJK4p1mHDqVCMZts5vdrjfu5b2R1GIb3+plN64YWC/pUOoezOg/kEw3CKHE vq9UumAHyckp8US+egEi4p1/ejtRiNqNSfWsLB7eJFpzN6yBiF+e59RH/FIQ9iTp annu3Z/v+RR7zeRHPn33ea36u7Rr9OU4ec9nJndNQclzHXzbqcwF51yETFSJgQVJ jwb0ra4XxuNo+YKub3l+xpBHDvoxkEsnFkAkzvQZlvehNh2Poy4wP4nCaZTCwnSL wUFrw== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ywx4gbp7n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:33 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:32 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:32 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 7F2353F7040; Sun, 23 Jun 2024 23:24:28 -0700 (PDT) From: Aakash Sasidharan To: Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Harman Kalra , Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v2 08/12] crypto/cnxk: add dual submission in Rx inject Date: Mon, 24 Jun 2024 11:53:57 +0530 Message-ID: <20240624062401.4143606-9-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: fKwVcECZKf_3uo2KE0ceoAUluX79lrlD X-Proofpoint-ORIG-GUID: fKwVcECZKf_3uo2KE0ceoAUluX79lrlD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Add dual submission to CPT in Rx inject path. Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri --- drivers/common/cnxk/roc_cpt.h | 43 +++++++++----- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 70 +++++++++++++++++------ drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 9 +++ 3 files changed, 90 insertions(+), 32 deletions(-) diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h index 3721fa08c0..8ef9062ae0 100644 --- a/drivers/common/cnxk/roc_cpt.h +++ b/drivers/common/cnxk/roc_cpt.h @@ -30,23 +30,36 @@ /* Vector of sizes in the burst of 16 CPT inst except first in 63:19 of * APT_LMT_ARG_S */ -#define ROC_CN10K_CPT_LMT_ARG \ - (ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 0) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 1) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 2) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 3) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 4) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 5) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 6) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 7) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 8) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 9) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 10) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 11) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 12) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 13) | \ +#define ROC_CN10K_CPT_LMT_ARG \ + (ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 0) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 1) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 2) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 3) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 4) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 5) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 6) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 7) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 8) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 9) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 10) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 11) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 12) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 13) | \ ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 14)) +/* Vector of sizes in the burst of 2 * 16 CPT inst except first in 63:19 of + * APT_LMT_ARG_S + */ +#define ROC_CN10K_DUAL_CPT_LMT_ARG \ + (ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 0) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 1) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 2) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 3) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 4) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 5) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 6) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 7) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 8) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 9) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 10) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 11) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 12) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 13) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 14)) + /* CPT helper macros */ #define ROC_CPT_AH_HDR_LEN 12 #define ROC_CPT_AES_GCM_IV_LEN 8 diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 90ca9eec27..a3a13c032e 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -55,6 +55,54 @@ struct vec_request { uint64_t w2; }; +static __rte_always_inline void __rte_hot +cn10k_cpt_lmtst_dual_submit(uint64_t *io_addr, const uint16_t lmt_id, int *i) +{ + uint64_t lmt_arg; + + /* Check if the total number of instructions is odd or even. */ + const int flag_odd = *i & 0x1; + + /* Reduce i by 1 when odd number of instructions.*/ + *i -= flag_odd; + + if (*i > 2 * CN10K_PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, *io_addr); + lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | (*i / 2 - CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + if (flag_odd) { + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_CPT_INST_DW_M1 << 4); + lmt_arg = (uint64_t)(lmt_id + *i / 2); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); + *i += 1; + } + } else { + if (*i != 0) { + lmt_arg = + ROC_CN10K_DUAL_CPT_LMT_ARG | (*i / 2 - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, *io_addr); + } + + if (flag_odd) { + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_CPT_INST_DW_M1 << 4); + lmt_arg = (uint64_t)(lmt_id + *i / 2); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); + *i += 1; + } + } + + rte_io_wmb(); +} + static inline struct cnxk_se_sess * cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op) { @@ -1396,7 +1444,7 @@ uint16_t __rte_hot cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, struct rte_security_session **sess, uint16_t nb_pkts) { - uint64_t lmt_base, lmt_arg, io_addr, u64_0, u64_1, l2_len, pf_func; + uint64_t lmt_base, io_addr, u64_0, u64_1, l2_len, pf_func; uint64x2_t inst_01, inst_23, inst_45, inst_67; struct cn10k_sec_session *sec_sess; struct rte_cryptodev *cdev = dev; @@ -1431,7 +1479,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, if (unlikely(fc.s.qsize > fc_thresh)) goto exit; - for (; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_pkts); i++) { + for (; i < RTE_MIN(2 * CN10K_PKTS_PER_LOOP, nb_pkts); i++) { m = pkts[i]; sec_sess = (struct cn10k_sec_session *)sess[i]; @@ -1487,24 +1535,12 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, inst_67 = vsetq_lane_u64(u64_1, inst_67, 1); vst1q_u64(&inst->w6.u64, inst_67); - inst += 2; - } - - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); + inst++; } - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); - if (nb_pkts - i > 0 && i == CN10K_PKTS_PER_LOOP) { + if (nb_pkts - i > 0 && i == 2 * CN10K_PKTS_PER_LOOP) { nb_pkts -= i; pkts += i; count += i; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index 51369309c5..6acaa4413b 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -431,6 +431,7 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, struct rte_pci_device *pci_dev; struct cnxk_cpt_qp *qp; uint32_t nb_desc; + uint64_t io_addr; int ret; if (dev->data->queue_pairs[qp_id] != NULL) @@ -485,6 +486,14 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, vf->rx_inj_sso_pf_func = roc_idev_nix_inl_dev_pffunc_get(); + /* Update IO addr to enable dual submission */ + io_addr = vf->rx_inj_lmtline.io_addr; + io_addr = (io_addr & ~(uint64_t)(0x7 << 4)) | ROC_CN10K_TWO_CPT_INST_DW_M1 << 4; + vf->rx_inj_lmtline.io_addr = io_addr; + + /* Update FC threshold to reflect dual submission */ + vf->rx_inj_lmtline.fc_thresh -= 32; + /* Block the queue for other submissions */ qp->pend_q.pq_mask = 0; } From patchwork Mon Jun 24 06:23:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141520 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CD6A4454A7; Mon, 24 Jun 2024 08:32:44 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 372B640E19; Mon, 24 Jun 2024 08:31:44 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id F18694027B for ; Mon, 24 Jun 2024 08:24:37 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NM04HT029847 for ; Sun, 23 Jun 2024 23:24:37 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=i WsZa4iTDKKcLOq8Shg4aYZyv7gy1i9kJjjJ8c7eOxk=; b=Xq5Qu8BVo2HpQZ+ZB xDSCXzazPbISwHufNf5Tf21hGli40TJCivS1brjaCw8BGWKgTGY5VaXP8Lr90btg NLLiKN1zRVNpLNDvj8LQzvdGAzK6FvuUCJdo1orLRuqSpx3TFXIMyQ66/vKTJ+Hl nIEjBInIV0c5Tho/+x1RTgXscR3ZcBIo8z8IgZGVO+oYAzRVg1gcBK/twanGiCZ8 VwXliYvjznHBjNvytYNLyDg3TVf+XmYKHMv0YbkA+uUVNmU81Vm9gXKxhtHYJcC+ IKqiG6spvQWUfxOz44Ixcg4tOalWbHLVgeRn+mJ2xPwFfy3EZSm1aToXnvwk4gom +MOxw== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ywx4gbp7q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:36 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:35 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:35 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id D313B3F7040; Sun, 23 Jun 2024 23:24:32 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v2 09/12] crypto/cnxk: update sess pointer for next iteration Date: Mon, 24 Jun 2024 11:53:58 +0530 Message-ID: <20240624062401.4143606-10-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: jM43u270IBOZqBxnyQNqYu_mEDtBaVa0 X-Proofpoint-ORIG-GUID: jM43u270IBOZqBxnyQNqYu_mEDtBaVa0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Update sess pointer while working on next set of packets. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index a3a13c032e..0d5a9ab5ef 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -1460,6 +1460,8 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, vf = cdev->data->dev_private; + const int nb_pkts_per_loop = 2 * CN10K_PKTS_PER_LOOP; + lmt_base = vf->rx_inj_lmtline.lmt_base; io_addr = vf->rx_inj_lmtline.io_addr; fc_addr = vf->rx_inj_lmtline.fc_addr; @@ -1479,7 +1481,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, if (unlikely(fc.s.qsize > fc_thresh)) goto exit; - for (; i < RTE_MIN(2 * CN10K_PKTS_PER_LOOP, nb_pkts); i++) { + for (; i < RTE_MIN(nb_pkts_per_loop, nb_pkts); i++) { m = pkts[i]; sec_sess = (struct cn10k_sec_session *)sess[i]; @@ -1540,10 +1542,11 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); - if (nb_pkts - i > 0 && i == 2 * CN10K_PKTS_PER_LOOP) { - nb_pkts -= i; - pkts += i; - count += i; + if (nb_pkts - i > 0 && i == nb_pkts_per_loop) { + nb_pkts -= nb_pkts_per_loop; + pkts += nb_pkts_per_loop; + count += nb_pkts_per_loop; + sess += nb_pkts_per_loop; goto again; } From patchwork Mon Jun 24 06:23:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141521 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 004E1454A7; Mon, 24 Jun 2024 08:32:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6529B40E22; Mon, 24 Jun 2024 08:31:45 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 786654027B for ; Mon, 24 Jun 2024 08:24:40 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NMJR0W027065 for ; Sun, 23 Jun 2024 23:24:40 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=x V5HlxcGoYTPw7ZFJrLNCaQTeS5tSY6mLGTaBcITRIU=; b=K1C0B/inPWwQAS1ps s5BKula8mEOggZcr/hmue3ov6Tp7XDbLtMrtpUIn06H0z5biw2QR5VCOJueOzY0R IpCfTDXM+SXgTvbKWAO63KNfc0IM9mawoUFJh+woxTmtXaaXYNj+bIYw9vhluHiW JKr5SZLQ9QLEOMlNDXym2HrBfz1FIjJ56ByYzizoCYNn/EGCwv7Z00dSY+emRK1O TIMnE1RFzExZRlR6IU0ijSA8D3SOXqSKluMCbMXfbCZmes3ySrTtIJ1PrTcO3FB5 aIJ3LDI31R0n3yGHCFzycwr9vyNNaFWqH2y6+YPWTBgOgIREwYF1jgK0JZLrzjry Ys/dg== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ywx4gbp7s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:39 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:38 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:38 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id EAA703F7040; Sun, 23 Jun 2024 23:24:35 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Akhil Goyal CC: , , , Subject: [PATCH v2 10/12] crypto/cnxk: fix aes-gcm zero len input cases Date: Mon, 24 Jun 2024 11:53:59 +0530 Message-ID: <20240624062401.4143606-11-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: nd9XIRcJPegBN9cxxrCA0Z0l5PH_D_zU X-Proofpoint-ORIG-GUID: nd9XIRcJPegBN9cxxrCA0Z0l5PH_D_zU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For aes-gcm (AEAD) zero length input, sg code path is taken unlike the digest only cases as AAD is treated as a separate input component. Fix the zero len case in SG path by avoiding the gather component only when it is a non AEAD algorithm. Also add sg version check as the fix only applies to specific model. Fixes: 4d8166d64988 ("crypto/cnxk: enable digest for zero length input") Signed-off-by: Aakash Sasidharan --- drivers/crypto/cnxk/cnxk_se.h | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 6374718a82..63dbef4411 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2468,13 +2468,14 @@ fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) } static __rte_always_inline uint32_t -prepare_iov_from_pkt(struct rte_mbuf *pkt, struct roc_se_iov_ptr *iovec, uint32_t start_offset) +prepare_iov_from_pkt(struct rte_mbuf *pkt, struct roc_se_iov_ptr *iovec, uint32_t start_offset, + const bool is_aead, const bool is_sg_ver2) { uint16_t index = 0; void *seg_data = NULL; int32_t seg_size = 0; - if (!pkt || pkt->data_len == 0) { + if (!pkt || (is_sg_ver2 && (pkt->data_len == 0) && !is_aead)) { iovec->buf_cnt = 0; return 0; } @@ -2619,13 +2620,13 @@ fill_sm_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.dst_iov = (void *)dst; /* Store SG I/O in the api for reuse */ - if (prepare_iov_from_pkt(m_src, fc_params.src_iov, 0)) { + if (prepare_iov_from_pkt(m_src, fc_params.src_iov, 0, false, is_sg_ver2)) { plt_dp_err("Prepare src iov failed"); ret = -EINVAL; goto err_exit; } - if (prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0)) { + if (prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0, false, is_sg_ver2)) { plt_dp_err("Prepare dst iov failed for m_dst %p", m_dst); ret = -EINVAL; goto err_exit; @@ -2816,14 +2817,15 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.dst_iov = (void *)dst; /* Store SG I/O in the api for reuse */ - if (prepare_iov_from_pkt(m_src, fc_params.src_iov, 0)) { + if (prepare_iov_from_pkt(m_src, fc_params.src_iov, 0, is_aead, is_sg_ver2)) { plt_dp_err("Prepare src iov failed"); ret = -EINVAL; goto err_exit; } if (unlikely(m_dst != NULL)) { - if (prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0)) { + if (prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0, is_aead, + is_sg_ver2)) { plt_dp_err("Prepare dst iov failed for " "m_dst %p", m_dst); @@ -2957,13 +2959,15 @@ fill_pdcp_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.dst_iov = (void *)dst; /* Store SG I/O in the api for reuse */ - if (unlikely(prepare_iov_from_pkt(m_src, fc_params.src_iov, 0))) { + if (unlikely( + prepare_iov_from_pkt(m_src, fc_params.src_iov, 0, false, is_sg_ver2))) { plt_dp_err("Prepare src iov failed"); ret = -EINVAL; goto err_exit; } - if (unlikely(prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0))) { + if (unlikely( + prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0, false, is_sg_ver2))) { plt_dp_err("Prepare dst iov failed for m_dst %p", m_dst); ret = -EINVAL; goto err_exit; @@ -3080,14 +3084,16 @@ fill_pdcp_chain_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.dst_iov = (void *)dst; /* Store SG I/O in the api for reuse */ - if (unlikely(prepare_iov_from_pkt(m_src, fc_params.src_iov, 0))) { + if (unlikely( + prepare_iov_from_pkt(m_src, fc_params.src_iov, 0, false, is_sg_ver2))) { plt_dp_err("Could not prepare src iov"); ret = -EINVAL; goto err_exit; } if (unlikely(m_dst != NULL)) { - if (unlikely(prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0))) { + if (unlikely(prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0, false, + is_sg_ver2))) { plt_dp_err("Could not prepare m_dst iov %p", m_dst); ret = -EINVAL; goto err_exit; @@ -3306,7 +3312,7 @@ fill_digest_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, params.src_iov = (void *)src; /*Store SG I/O in the api for reuse */ - if (prepare_iov_from_pkt(m_src, params.src_iov, auth_range_off)) { + if (prepare_iov_from_pkt(m_src, params.src_iov, auth_range_off, false, is_sg_ver2)) { plt_dp_err("Prepare src iov failed"); ret = -EINVAL; goto free_mdata_and_exit; From patchwork Mon Jun 24 06:24:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141522 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 659BE454A7; Mon, 24 Jun 2024 08:32:59 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9C2E440E2B; Mon, 24 Jun 2024 08:31:46 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 076A14027B for ; Mon, 24 Jun 2024 08:24:43 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NNFJXU005643 for ; Sun, 23 Jun 2024 23:24:43 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=j cif+WxeU4jjzNUCkGATxOk+hLTsckyNLKZBmXGs9xw=; b=JMZXlcnCCS/jgrLgj NiMYiky8PD6UGiG2juq1LoWKl0CkutM2Y6lwrcStPEN5JxgsOxxXARvsH7+aQfh9 fCmF0COuxqT/6wuNtBFGXCTVSjmHTHqdQWe0lASqQN7+KDB4oUxMdZ1PWOSH7/p/ s1KIriU3piO9fhC8I7vKF7PVzHm3lkEK1g3HB9nqMPAdqY2pGv9t9ESeJIv4JBko 4BifTzByn0MjIEit/z+oPya26dr5svIx0BWnl6zE4I9yisp+EAAzAXbAlHcZQDnt 2bbYKwTuTyPUCN1QWXt4IRjA89VgZsxbDbmmYCtlxYJNVjyoK/QLh4c/ok9mgiK/ QsPqQ== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3yxcgtjbqu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:42 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:41 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:41 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 09FE13F7040; Sun, 23 Jun 2024 23:24:38 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v2 11/12] crypto/cnxk: make pack IV variable as const Date: Mon, 24 Jun 2024 11:54:00 +0530 Message-ID: <20240624062401.4143606-12-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: M7xJSrEP1SN0AjfWlnVbaESKLTYeMrFD X-Proofpoint-ORIG-GUID: M7xJSrEP1SN0AjfWlnVbaESKLTYeMrFD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Make 'pack_iv' variable as const to avoid multiple checks. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 63dbef4411..dbd36a8a54 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -105,7 +105,7 @@ cpt_pack_iv(uint8_t *iv_src, uint8_t *iv_dst) } static inline void -pdcp_iv_copy(uint8_t *iv_d, const uint8_t *iv_s, const uint8_t pdcp_alg_type, uint8_t pack_iv) +pdcp_iv_copy(uint8_t *iv_d, const uint8_t *iv_s, const uint8_t pdcp_alg_type, const bool pack_iv) { const uint32_t *iv_s_temp; uint32_t iv_temp[4]; @@ -261,7 +261,7 @@ cpt_mac_len_verify(struct rte_crypto_auth_xform *auth) static __rte_always_inline int sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t offset_ctrl, - const uint8_t *iv_s, int iv_len, uint8_t pack_iv, uint8_t pdcp_alg_type, + const uint8_t *iv_s, int iv_len, const bool pack_iv, uint8_t pdcp_alg_type, int32_t inputlen, int32_t outputlen, uint32_t passthrough_len, uint32_t req_flags, int pdcp_flag, int decrypt) { @@ -457,7 +457,7 @@ sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t static __rte_always_inline int sg2_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t offset_ctrl, - const uint8_t *iv_s, int iv_len, uint8_t pack_iv, uint8_t pdcp_alg_type, + const uint8_t *iv_s, int iv_len, const bool pack_iv, uint8_t pdcp_alg_type, int32_t inputlen, int32_t outputlen, uint32_t passthrough_len, uint32_t req_flags, int pdcp_flag, int decrypt) { @@ -882,7 +882,7 @@ static inline int pdcp_chain_sg1_prep(struct roc_se_fc_params *params, struct roc_se_ctx *cpt_ctx, struct cpt_inst_s *inst, union cpt_inst_w4 w4, int32_t inputlen, uint8_t hdr_len, uint64_t offset_ctrl, uint32_t req_flags, - const uint8_t *cipher_iv, const uint8_t *auth_iv, const int pack_iv, + const uint8_t *cipher_iv, const uint8_t *auth_iv, const bool pack_iv, const uint8_t pdcp_ci_alg, const uint8_t pdcp_auth_alg) { struct roc_sglist_comp *scatter_comp, *gather_comp; @@ -991,7 +991,7 @@ static inline int pdcp_chain_sg2_prep(struct roc_se_fc_params *params, struct roc_se_ctx *cpt_ctx, struct cpt_inst_s *inst, union cpt_inst_w4 w4, int32_t inputlen, uint8_t hdr_len, uint64_t offset_ctrl, uint32_t req_flags, - const uint8_t *cipher_iv, const uint8_t *auth_iv, const int pack_iv, + const uint8_t *cipher_iv, const uint8_t *auth_iv, const bool pack_iv, const uint8_t pdcp_ci_alg, const uint8_t pdcp_auth_alg) { struct roc_sg2list_comp *gather_comp, *scatter_comp; @@ -1528,7 +1528,6 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, struct roc_se_ctx *se_ctx; uint64_t *offset_vaddr; uint64_t offset_ctrl; - uint8_t pack_iv = 0; int32_t inputlen; void *dm_vaddr; uint8_t *iv_d; @@ -1606,10 +1605,10 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, cpt_inst_w4.s.dlen = inputlen + ROC_SE_OFF_CTRL_LEN; iv_d = ((uint8_t *)offset_vaddr + ROC_SE_OFF_CTRL_LEN); - pdcp_iv_copy(iv_d, cipher_iv, pdcp_ci_alg, pack_iv); + pdcp_iv_copy(iv_d, cipher_iv, pdcp_ci_alg, false); iv_d = ((uint8_t *)offset_vaddr + ROC_SE_OFF_CTRL_LEN + pdcp_iv_off); - pdcp_iv_copy(iv_d, auth_iv, pdcp_auth_alg, pack_iv); + pdcp_iv_copy(iv_d, auth_iv, pdcp_auth_alg, false); inst->w4.u64 = cpt_inst_w4.u64; return 0; @@ -1618,11 +1617,11 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, if (is_sg_ver2) return pdcp_chain_sg2_prep(params, se_ctx, inst, cpt_inst_w4, inputlen, hdr_len, offset_ctrl, req_flags, cipher_iv, - auth_iv, pack_iv, pdcp_ci_alg, pdcp_auth_alg); + auth_iv, false, pdcp_ci_alg, pdcp_auth_alg); else return pdcp_chain_sg1_prep(params, se_ctx, inst, cpt_inst_w4, inputlen, hdr_len, offset_ctrl, req_flags, cipher_iv, - auth_iv, pack_iv, pdcp_ci_alg, pdcp_auth_alg); + auth_iv, false, pdcp_ci_alg, pdcp_auth_alg); } } @@ -1647,9 +1646,9 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, uint64_t *offset_vaddr; uint8_t pdcp_alg_type; uint32_t mac_len = 0; - const uint8_t *iv_s; - uint8_t pack_iv = 0; uint64_t offset_ctrl; + bool pack_iv = false; + const uint8_t *iv_s; int ret; mac_len = se_ctx->mac_len; @@ -1671,7 +1670,7 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, if (pdcp_alg_type != ROC_SE_PDCP_ALG_TYPE_AES_CMAC) { if (params->auth_iv_len == 25) - pack_iv = 1; + pack_iv = true; auth_offset = auth_offset / 8; auth_data_len = RTE_ALIGN(auth_data_len, 8) / 8; @@ -1694,7 +1693,7 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, pdcp_alg_type = se_ctx->pdcp_ci_alg; if (params->cipher_iv_len == 25) - pack_iv = 1; + pack_iv = true; /* * Microcode expects offsets in bytes From patchwork Mon Jun 24 06:24:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141523 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D5B46454A7; Mon, 24 Jun 2024 08:33:05 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EB5EA40E37; Mon, 24 Jun 2024 08:31:47 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id E5BB24027B for ; Mon, 24 Jun 2024 08:24:48 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45NLvS3X026147 for ; Sun, 23 Jun 2024 23:24:48 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=9 yfN+hLBJYnaaTIgl7UXakJLvU//mtLgEt0nI0xotn8=; b=gVo1ovjWBqEFvUgS2 /w/b8hRu6pqolj+4EiSgB+s757eR0CZDmoaYaavuoAFNHUF5JAm0LC+oBQCs3Hxj Cp2qXzJNNZBhftLDs4MHpxr+6s0lYMizLU8dcxdPczpMOHXoSUIJSKWFKduynmUH zWI1PqVCfSYzeUIcfylg08Akd2q8/7RMQ9vTkBkb/xMucrvvn215keZu0LGP7hs3 s08wUTIM3QloDKEr4uiLfZSXIb3pjrhIpRNwpPJYY1Q0wT0ilFRISfY/ci1ECizv QWTaRVWrKCTAKnMe0HebigF7XF20h9lP+e0BCQXXTiTm7N2EUj8uGBNorfkNwEtf MQwTw== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ywx4gbp7x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 23 Jun 2024 23:24:47 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 23 Jun 2024 23:24:46 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 23 Jun 2024 23:24:46 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 329DA3F7040; Sun, 23 Jun 2024 23:24:41 -0700 (PDT) From: Aakash Sasidharan To: Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Harman Kalra , Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Pavan Nikhilesh , "Shijith Thotton" CC: , , , , Subject: [PATCH v2 12/12] crypto/cnxk: enable dual submission to CPT Date: Mon, 24 Jun 2024 11:54:01 +0530 Message-ID: <20240624062401.4143606-13-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240624062401.4143606-1-asasidharan@marvell.com> References: <20240620145848.3461844-1-asasidharan@marvell.com> <20240624062401.4143606-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: Gd4O3OCFwKr_C8yBMQfxwdFImPi-sgCj X-Proofpoint-ORIG-GUID: Gd4O3OCFwKr_C8yBMQfxwdFImPi-sgCj X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-24_05,2024-06-21_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Submit two instructions in one LMTLINE. Signed-off-by: Anoob Joseph --- drivers/common/cnxk/roc_cpt.c | 17 +- drivers/common/cnxk/roc_cpt.h | 8 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 182 +++++----------------- drivers/crypto/cnxk/cn10k_cryptodev_ops.h | 60 ++++++- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 47 ++---- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 2 + drivers/event/cnxk/cnxk_eventdev_adptr.c | 4 +- 7 files changed, 124 insertions(+), 196 deletions(-) diff --git a/drivers/common/cnxk/roc_cpt.c b/drivers/common/cnxk/roc_cpt.c index 9f283ceb2e..aba2a49d19 100644 --- a/drivers/common/cnxk/roc_cpt.c +++ b/drivers/common/cnxk/roc_cpt.c @@ -1135,8 +1135,8 @@ roc_cpt_iq_enable(struct roc_cpt_lf *lf) } int -roc_cpt_lmtline_init(struct roc_cpt *roc_cpt, struct roc_cpt_lmtline *lmtline, - int lf_id) +roc_cpt_lmtline_init(struct roc_cpt *roc_cpt, struct roc_cpt_lmtline *lmtline, int lf_id, + bool is_dual) { struct roc_cpt_lf *lf; @@ -1145,12 +1145,19 @@ roc_cpt_lmtline_init(struct roc_cpt *roc_cpt, struct roc_cpt_lmtline *lmtline, return -ENOTSUP; lmtline->io_addr = lf->io_addr; - if (roc_model_is_cn10k()) - lmtline->io_addr |= ROC_CN10K_CPT_INST_DW_M1 << 4; + lmtline->fc_thresh = lf->nb_desc - CPT_LF_FC_MIN_THRESHOLD; + + if (roc_model_is_cn10k()) { + if (is_dual) { + lmtline->io_addr |= ROC_CN10K_TWO_CPT_INST_DW_M1 << 4; + lmtline->fc_thresh = lf->nb_desc - 2 * CPT_LF_FC_MIN_THRESHOLD; + } else { + lmtline->io_addr |= ROC_CN10K_CPT_INST_DW_M1 << 4; + } + } lmtline->fc_addr = lf->fc_addr; lmtline->lmt_base = lf->lmt_base; - lmtline->fc_thresh = lf->nb_desc - CPT_LF_FC_MIN_THRESHOLD; return 0; } diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h index 8ef9062ae0..e2e919f80f 100644 --- a/drivers/common/cnxk/roc_cpt.h +++ b/drivers/common/cnxk/roc_cpt.h @@ -200,12 +200,12 @@ int __roc_api roc_cpt_afs_print(struct roc_cpt *roc_cpt); int __roc_api roc_cpt_lfs_print(struct roc_cpt *roc_cpt); void __roc_api roc_cpt_iq_disable(struct roc_cpt_lf *lf); void __roc_api roc_cpt_iq_enable(struct roc_cpt_lf *lf); -int __roc_api roc_cpt_lmtline_init(struct roc_cpt *roc_cpt, - struct roc_cpt_lmtline *lmtline, int lf_id); +int __roc_api roc_cpt_lmtline_init(struct roc_cpt *roc_cpt, struct roc_cpt_lmtline *lmtline, + int lf_id, bool is_dual); void __roc_api roc_cpt_parse_hdr_dump(FILE *file, const struct cpt_parse_hdr_s *cpth); -int __roc_api roc_cpt_ctx_write(struct roc_cpt_lf *lf, void *sa_dptr, - void *sa_cptr, uint16_t sa_len); +int __roc_api roc_cpt_ctx_write(struct roc_cpt_lf *lf, void *sa_dptr, void *sa_cptr, + uint16_t sa_len); void __roc_api roc_cpt_int_misc_cb_register(roc_cpt_int_misc_cb_t cb, void *args); int __roc_api roc_cpt_int_misc_cb_unregister(roc_cpt_int_misc_cb_t cb, void *args); diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 0d5a9ab5ef..9d6ac06bd2 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -12,11 +12,6 @@ #include #include "roc_cpt.h" -#if defined(__aarch64__) -#include "roc_io.h" -#else -#include "roc_io_generic.h" -#endif #include "roc_idev.h" #include "roc_sso.h" #include "roc_sso_dp.h" @@ -40,8 +35,8 @@ /* Holds information required to send crypto operations in one burst */ struct ops_burst { - struct rte_crypto_op *op[CN10K_PKTS_PER_LOOP]; - uint64_t w2[CN10K_PKTS_PER_LOOP]; + struct rte_crypto_op *op[CN10K_CPT_PKTS_PER_LOOP]; + uint64_t w2[CN10K_CPT_PKTS_PER_LOOP]; struct cn10k_sso_hws *ws; struct cnxk_cpt_qp *qp; uint16_t nb_ops; @@ -55,54 +50,6 @@ struct vec_request { uint64_t w2; }; -static __rte_always_inline void __rte_hot -cn10k_cpt_lmtst_dual_submit(uint64_t *io_addr, const uint16_t lmt_id, int *i) -{ - uint64_t lmt_arg; - - /* Check if the total number of instructions is odd or even. */ - const int flag_odd = *i & 0x1; - - /* Reduce i by 1 when odd number of instructions.*/ - *i -= flag_odd; - - if (*i > 2 * CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, *io_addr); - lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | (*i / 2 - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, *io_addr); - if (flag_odd) { - *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | - (ROC_CN10K_CPT_INST_DW_M1 << 4); - lmt_arg = (uint64_t)(lmt_id + *i / 2); - roc_lmt_submit_steorl(lmt_arg, *io_addr); - *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | - (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); - *i += 1; - } - } else { - if (*i != 0) { - lmt_arg = - ROC_CN10K_DUAL_CPT_LMT_ARG | (*i / 2 - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, *io_addr); - } - - if (flag_odd) { - *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | - (ROC_CN10K_CPT_INST_DW_M1 << 4); - lmt_arg = (uint64_t)(lmt_id + *i / 2); - roc_lmt_submit_steorl(lmt_arg, *io_addr); - *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | - (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); - *i += 1; - } - } - - rte_io_wmb(); -} - static inline struct cnxk_se_sess * cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op) { @@ -385,8 +332,8 @@ static uint16_t cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops, const bool is_sg_ver2) { - uint64_t lmt_base, lmt_arg, io_addr; struct cpt_inflight_req *infl_req; + uint64_t head, lmt_base, io_addr; uint16_t nb_allowed, count = 0; struct cnxk_cpt_qp *qp = qptr; struct pending_queue *pend_q; @@ -394,7 +341,6 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops, union cpt_fc_write_s fc; uint64_t *fc_addr; uint16_t lmt_id; - uint64_t head; int ret, i; pend_q = &qp->pend_q; @@ -424,11 +370,11 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops, goto pend_q_commit; } - for (i = 0; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_ops); i++) { + for (i = 0; i < RTE_MIN(CN10K_CPT_PKTS_PER_LOOP, nb_ops); i++) { infl_req = &pend_q->req_queue[head]; infl_req->op_flags = 0; - ret = cn10k_cpt_fill_inst(qp, ops + i, &inst[2 * i], infl_req, is_sg_ver2); + ret = cn10k_cpt_fill_inst(qp, ops + i, &inst[i], infl_req, is_sg_ver2); if (unlikely(ret != 1)) { plt_dp_err("Could not process op: %p", ops + i); if (i == 0) @@ -439,24 +385,12 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops, pending_queue_advance(&head, pq_mask); } - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); - if (nb_ops - i > 0 && i == CN10K_PKTS_PER_LOOP) { - nb_ops -= i; - ops += i; - count += i; + if (nb_ops - i > 0 && i == CN10K_CPT_PKTS_PER_LOOP) { + nb_ops -= CN10K_CPT_PKTS_PER_LOOP; + ops += CN10K_CPT_PKTS_PER_LOOP; + count += CN10K_CPT_PKTS_PER_LOOP; goto again; } @@ -631,7 +565,7 @@ cn10k_cpt_vec_pkt_submission_timeout_handle(void) static inline void cn10k_cpt_vec_submit(struct vec_request vec_tbl[], uint16_t vec_tbl_len, struct cnxk_cpt_qp *qp) { - uint64_t lmt_base, lmt_arg, lmt_id, io_addr; + uint64_t lmt_base, lmt_id, io_addr; union cpt_fc_write_s fc; struct cpt_inst_s *inst; uint16_t burst_size; @@ -659,7 +593,7 @@ cn10k_cpt_vec_submit(struct vec_request vec_tbl[], uint16_t vec_tbl_len, struct again: burst_size = RTE_MIN(CN10K_PKTS_PER_STEORL, vec_tbl_len); for (i = 0; i < burst_size; i++) - cn10k_cpt_vec_inst_fill(&vec_tbl[i], &inst[i * 2], qp, vec_tbl[0].w7); + cn10k_cpt_vec_inst_fill(&vec_tbl[i], &inst[i], qp, vec_tbl[0].w7); do { fc.u64[0] = __atomic_load_n(fc_addr, __ATOMIC_RELAXED); @@ -669,10 +603,7 @@ cn10k_cpt_vec_submit(struct vec_request vec_tbl[], uint16_t vec_tbl_len, struct cn10k_cpt_vec_pkt_submission_timeout_handle(); } while (true); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); vec_tbl_len -= i; @@ -686,12 +617,12 @@ static inline int ca_lmtst_vec_submit(struct ops_burst *burst, struct vec_request vec_tbl[], uint16_t *vec_tbl_len, const bool is_sg_ver2) { - struct cpt_inflight_req *infl_reqs[CN10K_PKTS_PER_LOOP]; - uint64_t lmt_base, lmt_arg, io_addr; + struct cpt_inflight_req *infl_reqs[CN10K_CPT_PKTS_PER_LOOP]; uint16_t lmt_id, len = *vec_tbl_len; struct cpt_inst_s *inst, *inst_base; struct cpt_inflight_req *infl_req; struct rte_event_vector *vec; + uint64_t lmt_base, io_addr; union cpt_fc_write_s fc; struct cnxk_cpt_qp *qp; uint64_t *fc_addr; @@ -728,7 +659,7 @@ ca_lmtst_vec_submit(struct ops_burst *burst, struct vec_request vec_tbl[], uint1 } for (i = 0; i < burst->nb_ops; i++) { - inst = &inst_base[2 * i]; + inst = &inst_base[i]; infl_req = infl_reqs[i]; infl_req->op_flags = 0; @@ -788,24 +719,12 @@ next_op:; if (CNXK_TT_FROM_TAG(burst->ws->gw_rdata) == SSO_TT_ORDERED) roc_sso_hws_head_wait(burst->ws->base); - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); /* Store w7 of last successfully filled instruction */ inst = &inst_base[2 * (i - 1)]; vec_tbl[0].w7 = inst->w7; - rte_io_wmb(); - put: if (i != burst->nb_ops) rte_mempool_put_bulk(qp->ca.req_mp, (void *)&infl_reqs[i], burst->nb_ops - i); @@ -818,10 +737,10 @@ next_op:; static inline uint16_t ca_lmtst_burst_submit(struct ops_burst *burst, const bool is_sg_ver2) { - struct cpt_inflight_req *infl_reqs[CN10K_PKTS_PER_LOOP]; - uint64_t lmt_base, lmt_arg, io_addr; + struct cpt_inflight_req *infl_reqs[CN10K_CPT_PKTS_PER_LOOP]; struct cpt_inst_s *inst, *inst_base; struct cpt_inflight_req *infl_req; + uint64_t lmt_base, io_addr; union cpt_fc_write_s fc; struct cnxk_cpt_qp *qp; uint64_t *fc_addr; @@ -852,7 +771,7 @@ ca_lmtst_burst_submit(struct ops_burst *burst, const bool is_sg_ver2) } for (i = 0; i < burst->nb_ops; i++) { - inst = &inst_base[2 * i]; + inst = &inst_base[i]; infl_req = infl_reqs[i]; infl_req->op_flags = 0; @@ -889,19 +808,7 @@ ca_lmtst_burst_submit(struct ops_burst *burst, const bool is_sg_ver2) if (CNXK_TT_FROM_TAG(burst->ws->gw_rdata) == SSO_TT_ORDERED) roc_sso_hws_head_wait(burst->ws->base); - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); put: if (unlikely(i != burst->nb_ops)) @@ -963,7 +870,7 @@ cn10k_cpt_crypto_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_ev burst.op[burst.nb_ops] = op; /* Max nb_ops per burst check */ - if (++burst.nb_ops == CN10K_PKTS_PER_LOOP) { + if (++burst.nb_ops == CN10K_CPT_PKTS_PER_LOOP) { if (is_vector) submitted = ca_lmtst_vec_submit(&burst, vec_tbl, &vec_tbl_len, is_sg_ver2); @@ -1460,8 +1367,6 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, vf = cdev->data->dev_private; - const int nb_pkts_per_loop = 2 * CN10K_PKTS_PER_LOOP; - lmt_base = vf->rx_inj_lmtline.lmt_base; io_addr = vf->rx_inj_lmtline.io_addr; fc_addr = vf->rx_inj_lmtline.fc_addr; @@ -1481,7 +1386,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, if (unlikely(fc.s.qsize > fc_thresh)) goto exit; - for (; i < RTE_MIN(nb_pkts_per_loop, nb_pkts); i++) { + for (; i < RTE_MIN(CN10K_CPT_PKTS_PER_LOOP, nb_pkts); i++) { m = pkts[i]; sec_sess = (struct cn10k_sec_session *)sess[i]; @@ -1542,11 +1447,11 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); - if (nb_pkts - i > 0 && i == nb_pkts_per_loop) { - nb_pkts -= nb_pkts_per_loop; - pkts += nb_pkts_per_loop; - count += nb_pkts_per_loop; - sess += nb_pkts_per_loop; + if (nb_pkts - i > 0 && i == CN10K_CPT_PKTS_PER_LOOP) { + nb_pkts -= CN10K_CPT_PKTS_PER_LOOP; + pkts += CN10K_CPT_PKTS_PER_LOOP; + count += CN10K_CPT_PKTS_PER_LOOP; + sess += CN10K_CPT_PKTS_PER_LOOP; goto again; } @@ -1645,8 +1550,8 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym const bool is_sgv2) { uint16_t lmt_id, nb_allowed, nb_ops = vec->num; - uint64_t lmt_base, lmt_arg, io_addr, head; struct cpt_inflight_req *infl_req; + uint64_t lmt_base, io_addr, head; struct cnxk_cpt_qp *qp = qpair; struct cnxk_sym_dp_ctx *dp_ctx; struct pending_queue *pend_q; @@ -1683,7 +1588,7 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym goto pend_q_commit; } - for (i = 0; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_ops); i++) { + for (i = 0; i < RTE_MIN(CN10K_CPT_PKTS_PER_LOOP, nb_ops); i++) { struct cnxk_iov iov; index = count + i; @@ -1691,7 +1596,7 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym infl_req->op_flags = 0; cnxk_raw_burst_to_iov(vec, &ofs, index, &iov); - ret = cn10k_cpt_raw_fill_inst(&iov, qp, dp_ctx, &inst[2 * i], infl_req, + ret = cn10k_cpt_raw_fill_inst(&iov, qp, dp_ctx, &inst[i], infl_req, user_data[index], is_sgv2); if (unlikely(ret != 1)) { plt_dp_err("Could not process vec: %d", index); @@ -1705,21 +1610,9 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym pending_queue_advance(&head, pq_mask); } - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); - if (nb_ops - i > 0 && i == CN10K_PKTS_PER_LOOP) { + if (nb_ops - i > 0 && i == CN10K_CPT_PKTS_PER_LOOP) { nb_ops -= i; count += i; goto again; @@ -1760,8 +1653,8 @@ cn10k_cpt_raw_enqueue(void *qpair, uint8_t *drv_ctx, struct rte_crypto_vec *data struct rte_crypto_va_iova_ptr *aad_or_auth_iv, void *user_data, const bool is_sgv2) { - uint64_t lmt_base, lmt_arg, io_addr, head; struct cpt_inflight_req *infl_req; + uint64_t lmt_base, io_addr, head; struct cnxk_cpt_qp *qp = qpair; struct cnxk_sym_dp_ctx *dp_ctx; uint16_t lmt_id, nb_allowed; @@ -1769,7 +1662,7 @@ cn10k_cpt_raw_enqueue(void *qpair, uint8_t *drv_ctx, struct rte_crypto_vec *data union cpt_fc_write_s fc; struct cnxk_iov iov; uint64_t *fc_addr; - int ret; + int ret, i = 1; struct pending_queue *pend_q = &qp->pend_q; const uint64_t pq_mask = pend_q->pq_mask; @@ -1806,10 +1699,7 @@ cn10k_cpt_raw_enqueue(void *qpair, uint8_t *drv_ctx, struct rte_crypto_vec *data pending_queue_advance(&head, pq_mask); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); pend_q->head = head; pend_q->time_out = rte_get_timer_cycles() + DEFAULT_COMMAND_TIMEOUT * rte_get_timer_hz(); diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h index 406c4abc7f..be76c49a65 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h @@ -5,15 +5,21 @@ #ifndef _CN10K_CRYPTODEV_OPS_H_ #define _CN10K_CRYPTODEV_OPS_H_ -#include #include +#include #include #include +#if defined(__aarch64__) +#include "roc_io.h" +#else +#include "roc_io_generic.h" +#endif + #include "cnxk_cryptodev.h" -#define CN10K_PKTS_PER_LOOP 32 -#define CN10K_PKTS_PER_STEORL 16 +#define CN10K_PKTS_PER_STEORL 32 +#define CN10K_LMTLINES_PER_STEORL 16 extern struct rte_cryptodev_ops cn10k_cpt_ops; @@ -34,4 +40,52 @@ __rte_internal uint16_t __rte_hot cn10k_cpt_sg_ver2_crypto_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_events); +static __rte_always_inline void __rte_hot +cn10k_cpt_lmtst_dual_submit(uint64_t *io_addr, const uint16_t lmt_id, int *i) +{ + uint64_t lmt_arg; + + /* Check if the total number of instructions is odd or even. */ + const int flag_odd = *i & 0x1; + + /* Reduce i by 1 when odd number of instructions.*/ + *i -= flag_odd; + + if (*i > CN10K_PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | (CN10K_LMTLINES_PER_STEORL - 1) << 12 | + (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, *io_addr); + lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | + (*i / 2 - CN10K_LMTLINES_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + CN10K_LMTLINES_PER_STEORL); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + if (flag_odd) { + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_CPT_INST_DW_M1 << 4); + lmt_arg = (uint64_t)(lmt_id + *i / 2); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); + *i += 1; + } + } else { + if (*i != 0) { + lmt_arg = + ROC_CN10K_DUAL_CPT_LMT_ARG | (*i / 2 - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, *io_addr); + } + + if (flag_odd) { + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_CPT_INST_DW_M1 << 4); + lmt_arg = (uint64_t)(lmt_id + *i / 2); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); + *i += 1; + } + } + + rte_io_wmb(); +} #endif /* _CN10K_CRYPTODEV_OPS_H_ */ diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index 6acaa4413b..cfcfa79fdf 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -431,7 +431,6 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, struct rte_pci_device *pci_dev; struct cnxk_cpt_qp *qp; uint32_t nb_desc; - uint64_t io_addr; int ret; if (dev->data->queue_pairs[qp_id] != NULL) @@ -467,7 +466,7 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, roc_cpt->lf[qp_id] = &qp->lf; - ret = roc_cpt_lmtline_init(roc_cpt, &qp->lmtline, qp_id); + ret = roc_cpt_lmtline_init(roc_cpt, &qp->lmtline, qp_id, true); if (ret < 0) { roc_cpt->lf[qp_id] = NULL; plt_err("Could not init lmtline for queue pair %d", qp_id); @@ -478,7 +477,7 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, dev->data->queue_pairs[qp_id] = qp; if (qp_id == vf->rx_inject_qp) { - ret = roc_cpt_lmtline_init(roc_cpt, &vf->rx_inj_lmtline, vf->rx_inject_qp); + ret = roc_cpt_lmtline_init(roc_cpt, &vf->rx_inj_lmtline, vf->rx_inject_qp, true); if (ret) { plt_err("Could not init lmtline Rx inject"); goto exit; @@ -486,14 +485,6 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, vf->rx_inj_sso_pf_func = roc_idev_nix_inl_dev_pffunc_get(); - /* Update IO addr to enable dual submission */ - io_addr = vf->rx_inj_lmtline.io_addr; - io_addr = (io_addr & ~(uint64_t)(0x7 << 4)) | ROC_CN10K_TWO_CPT_INST_DW_M1 << 4; - vf->rx_inj_lmtline.io_addr = io_addr; - - /* Update FC threshold to reflect dual submission */ - vf->rx_inj_lmtline.fc_thresh -= 32; - /* Block the queue for other submissions */ qp->pend_q.pq_mask = 0; } @@ -969,44 +960,28 @@ rte_pmd_cnxk_crypto_qptr_get(uint8_t dev_id, uint16_t qp_id) static inline void cnxk_crypto_cn10k_submit(void *qptr, void *inst, uint16_t nb_inst) { - uint64_t lmt_base, lmt_arg, io_addr; struct cnxk_cpt_qp *qp = qptr; - uint16_t i, j, lmt_id; + uint64_t lmt_base, io_addr; + uint16_t lmt_id; void *lmt_dst; + int i; lmt_base = qp->lmtline.lmt_base; io_addr = qp->lmtline.io_addr; ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); -again: - i = RTE_MIN(nb_inst, CN10K_PKTS_PER_LOOP); lmt_dst = PLT_PTR_CAST(lmt_base); +again: + i = RTE_MIN(nb_inst, CN10K_CPT_PKTS_PER_LOOP); - for (j = 0; j < i; j++) { - rte_memcpy(lmt_dst, inst, sizeof(struct cpt_inst_s)); - inst = RTE_PTR_ADD(inst, sizeof(struct cpt_inst_s)); - lmt_dst = RTE_PTR_ADD(lmt_dst, 2 * sizeof(struct cpt_inst_s)); - } - - rte_io_wmb(); - - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } + memcpy(lmt_dst, inst, i * sizeof(struct cpt_inst_s)); - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); if (nb_inst - i > 0) { - nb_inst -= i; + nb_inst -= CN10K_CPT_PKTS_PER_LOOP; + inst = RTE_PTR_ADD(inst, CN10K_CPT_PKTS_PER_LOOP * sizeof(struct cpt_inst_s)); goto again; } } diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h index 9de7e432e4..caf6ac35e5 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h @@ -25,6 +25,8 @@ #define MOD_INC(i, l) ((i) == (l - 1) ? (i) = 0 : (i)++) +#define CN10K_CPT_PKTS_PER_LOOP 64 + /* Macros to form words in CPT instruction */ #define CNXK_CPT_INST_W2(tag, tt, grp, rvu_pf_func) \ ((tag) | ((uint64_t)(tt) << 32) | ((uint64_t)(grp) << 34) | \ diff --git a/drivers/event/cnxk/cnxk_eventdev_adptr.c b/drivers/event/cnxk/cnxk_eventdev_adptr.c index 98db11ad61..2c049e7041 100644 --- a/drivers/event/cnxk/cnxk_eventdev_adptr.c +++ b/drivers/event/cnxk/cnxk_eventdev_adptr.c @@ -632,7 +632,7 @@ crypto_adapter_qp_setup(const struct rte_cryptodev *cdev, struct cnxk_cpt_qp *qp * simultaneous enqueue from all available cores. */ if (roc_model_is_cn10k()) - nb_desc_min = rte_lcore_count() * 32; + nb_desc_min = rte_lcore_count() * CN10K_CPT_PKTS_PER_LOOP; else nb_desc_min = rte_lcore_count() * 2; @@ -707,7 +707,7 @@ crypto_adapter_qp_free(struct cnxk_cpt_qp *qp) rte_mempool_free(qp->ca.req_mp); qp->ca.enabled = false; - ret = roc_cpt_lmtline_init(qp->lf.roc_cpt, &qp->lmtline, qp->lf.lf_id); + ret = roc_cpt_lmtline_init(qp->lf.roc_cpt, &qp->lmtline, qp->lf.lf_id, true); if (ret < 0) { plt_err("Could not reset lmtline for queue pair %d", qp->lf.lf_id); return ret;