From patchwork Tue Sep 8 10:09:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Dwivedi X-Patchwork-Id: 76910 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5AC93A04AA; Tue, 8 Sep 2020 12:10:37 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3001E1C023; Tue, 8 Sep 2020 12:10:37 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 228E81C023 for ; Tue, 8 Sep 2020 12:10:34 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 088A56u8020360; Tue, 8 Sep 2020 03:10:34 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=FSk+2ztx9EzZ1NrproOLbsYRm3EKlhfEoIRzWb3SW6E=; b=gfBja3L2L5UN50wBz0Q7gHcgtptM5IfAlwyqKuAggbIvW9A7fXsHq2fJAtMVyKo9Gnmm 2oXSrOVCArKattzaUkN+h7V3zmWbgeg5Gnk6f2XVknanI7oyGC612W4AoG1fEAJz8aeu WXGYYSeDi1zfE9z6Kxc8d4K2p3e2+awFPvvkK/Gm3cQyvC5hTyEK2reBFaYd/jq+TKzw 6HBGkc//DJknIO3VCaqvOZTH+kAckjig/u0IZ4QVN+Wc32HUTRMgaNqDw98YOE9nIhKh lbaKPTrKhBWGWLNY0EdSq6ejw70Ox0Pgqb/KUFdl0BcKRX4pBmAKwJr4f84g04ZVzR9b 4w== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 33c81pu21v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 08 Sep 2020 03:10:34 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 8 Sep 2020 03:10:32 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 8 Sep 2020 03:10:31 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 8 Sep 2020 03:10:31 -0700 Received: from hyd1349.t110.caveonetworks.com (unknown [10.29.45.13]) by maili.marvell.com (Postfix) with ESMTP id D23713F703F; Tue, 8 Sep 2020 03:10:29 -0700 (PDT) From: Ankur Dwivedi To: CC: , , , , Ankur Dwivedi Date: Tue, 8 Sep 2020 15:39:54 +0530 Message-ID: <20200908100956.25868-2-adwivedi@marvell.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200908100956.25868-1-adwivedi@marvell.com> References: <20200908100956.25868-1-adwivedi@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-09-08_05:2020-09-08, 2020-09-08 signatures=0 Subject: [dpdk-dev] [PATCH 1/3] crypto/octeontx2: move functions to helper file X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Some functions are common across cryptodev pmd and the event crypto adapter. This patch moves them into a helper file. Signed-off-by: Ankur Dwivedi --- drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 65 +--------------- .../octeontx2/otx2_cryptodev_ops_helper.h | 74 +++++++++++++++++++ 2 files changed, 75 insertions(+), 64 deletions(-) create mode 100644 drivers/crypto/octeontx2/otx2_cryptodev_ops_helper.h diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c index ccf566d5f..77842b4ad 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c @@ -13,6 +13,7 @@ #include "otx2_cryptodev_hw_access.h" #include "otx2_cryptodev_mbox.h" #include "otx2_cryptodev_ops.h" +#include "otx2_cryptodev_ops_helper.h" #include "otx2_ipsec_po_ops.h" #include "otx2_mbox.h" #include "otx2_sec_idev.h" @@ -416,24 +417,6 @@ sym_session_configure(int driver_id, struct rte_crypto_sym_xform *xform, return -ENOTSUP; } -static void -sym_session_clear(int driver_id, struct rte_cryptodev_sym_session *sess) -{ - void *priv = get_sym_session_private_data(sess, driver_id); - struct rte_mempool *pool; - - if (priv == NULL) - return; - - memset(priv, 0, cpt_get_session_size()); - - pool = rte_mempool_from_obj(priv); - - set_sym_session_private_data(sess, driver_id, NULL); - - rte_mempool_put(pool, priv); -} - static __rte_always_inline int32_t __rte_hot otx2_cpt_enqueue_req(const struct otx2_cpt_qp *qp, struct pending_queue *pend_q, @@ -917,52 +900,6 @@ otx2_cpt_dequeue_post_process(struct otx2_cpt_qp *qp, struct rte_crypto_op *cop, } } -static __rte_always_inline uint8_t -otx2_cpt_compcode_get(struct cpt_request_info *req) -{ - volatile struct cpt_res_s_9s *res; - uint8_t ret; - - res = (volatile struct cpt_res_s_9s *)req->completion_addr; - - if (unlikely(res->compcode == CPT_9X_COMP_E_NOTDONE)) { - if (rte_get_timer_cycles() < req->time_out) - return ERR_REQ_PENDING; - - CPT_LOG_DP_ERR("Request timed out"); - return ERR_REQ_TIMEOUT; - } - - if (likely(res->compcode == CPT_9X_COMP_E_GOOD)) { - ret = NO_ERR; - if (unlikely(res->uc_compcode)) { - ret = res->uc_compcode; - CPT_LOG_DP_DEBUG("Request failed with microcode error"); - CPT_LOG_DP_DEBUG("MC completion code 0x%x", - res->uc_compcode); - } - } else { - CPT_LOG_DP_DEBUG("HW completion code 0x%x", res->compcode); - - ret = res->compcode; - switch (res->compcode) { - case CPT_9X_COMP_E_INSTERR: - CPT_LOG_DP_ERR("Request failed with instruction error"); - break; - case CPT_9X_COMP_E_FAULT: - CPT_LOG_DP_ERR("Request failed with DMA fault"); - break; - case CPT_9X_COMP_E_HWERR: - CPT_LOG_DP_ERR("Request failed with hardware error"); - break; - default: - CPT_LOG_DP_ERR("Request failed with unknown completion code"); - } - } - - return ret; -} - static uint16_t otx2_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) { diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_ops_helper.h b/drivers/crypto/octeontx2/otx2_cryptodev_ops_helper.h new file mode 100644 index 000000000..764daadea --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_cryptodev_ops_helper.h @@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#ifndef _OTX2_CRYPTODEV_OPS_HELPER_H_ +#define _OTX2_CRYPTODEV_OPS_HELPER_H_ + +#include "cpt_pmd_logs.h" + +static void +sym_session_clear(int driver_id, struct rte_cryptodev_sym_session *sess) +{ + void *priv = get_sym_session_private_data(sess, driver_id); + struct rte_mempool *pool; + + if (priv == NULL) + return; + + memset(priv, 0, cpt_get_session_size()); + + pool = rte_mempool_from_obj(priv); + + set_sym_session_private_data(sess, driver_id, NULL); + + rte_mempool_put(pool, priv); +} + +static __rte_always_inline uint8_t +otx2_cpt_compcode_get(struct cpt_request_info *req) +{ + volatile struct cpt_res_s_9s *res; + uint8_t ret; + + res = (volatile struct cpt_res_s_9s *)req->completion_addr; + + if (unlikely(res->compcode == CPT_9X_COMP_E_NOTDONE)) { + if (rte_get_timer_cycles() < req->time_out) + return ERR_REQ_PENDING; + + CPT_LOG_DP_ERR("Request timed out"); + return ERR_REQ_TIMEOUT; + } + + if (likely(res->compcode == CPT_9X_COMP_E_GOOD)) { + ret = NO_ERR; + if (unlikely(res->uc_compcode)) { + ret = res->uc_compcode; + CPT_LOG_DP_DEBUG("Request failed with microcode error"); + CPT_LOG_DP_DEBUG("MC completion code 0x%x", + res->uc_compcode); + } + } else { + CPT_LOG_DP_DEBUG("HW completion code 0x%x", res->compcode); + + ret = res->compcode; + switch (res->compcode) { + case CPT_9X_COMP_E_INSTERR: + CPT_LOG_DP_ERR("Request failed with instruction error"); + break; + case CPT_9X_COMP_E_FAULT: + CPT_LOG_DP_ERR("Request failed with DMA fault"); + break; + case CPT_9X_COMP_E_HWERR: + CPT_LOG_DP_ERR("Request failed with hardware error"); + break; + default: + CPT_LOG_DP_ERR("Request failed with unknown completion code"); + } + } + + return ret; +} + +#endif /* _OTX2_CRYPTODEV_OPS_HELPER_H_ */ From patchwork Tue Sep 8 10:09:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Dwivedi X-Patchwork-Id: 76911 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B2674A04AA; Tue, 8 Sep 2020 12:10:44 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 982AF1C0CE; Tue, 8 Sep 2020 12:10:44 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 463791C0CD for ; Tue, 8 Sep 2020 12:10:43 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 088A51Qu024501; Tue, 8 Sep 2020 03:10:42 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=IL9QhcwUTP2jiAq/x3C+yB4jbDToGunynREZzVTVHyk=; b=hMdIy8DjnyyM6akHoZtABF2SRra4svUkNq17YIZWNbt/l8GcXDjhQdGAdWzldyv7tqS/ 8l2I2ePN5z72Lg5YOfopW2MWD9KlIHURRyuj2B28Kd8Y2hiQYd4uq42zvTEDfHwWTfyq TXxcPkG7IH9HAKbazeOEuQaVxn+gHOUiZ7QRw0SLK1S+S24UBOkrDpRGg7z5Ekp0lWKa VuGdG8+2nF7kwp1DLFgFmNlf+Ou7RFndX5Y1rb2bmAvxMVuwPRPVgBMSS+Qmp6f/ySqS YOP3Au/442R4U6raTXjdqqGz8ScW6yg8Dz6U8idiMSDPIzkBa0Lp006DjNE4AWHy7sgs 3w== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 33ccvr1v13-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 08 Sep 2020 03:10:42 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 8 Sep 2020 03:10:41 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 8 Sep 2020 03:10:40 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 8 Sep 2020 03:10:40 -0700 Received: from hyd1349.t110.caveonetworks.com (unknown [10.29.45.13]) by maili.marvell.com (Postfix) with ESMTP id 214733F703F; Tue, 8 Sep 2020 03:10:37 -0700 (PDT) From: Ankur Dwivedi To: CC: , , , , Ankur Dwivedi Date: Tue, 8 Sep 2020 15:39:55 +0530 Message-ID: <20200908100956.25868-3-adwivedi@marvell.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200908100956.25868-1-adwivedi@marvell.com> References: <20200908100956.25868-1-adwivedi@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-09-08_05:2020-09-08, 2020-09-08 signatures=0 Subject: [dpdk-dev] [PATCH 2/3] event/octeontx2: add crypto adapter framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The crypto adapter callback functions and associated data structures are added. Signed-off-by: Ankur Dwivedi --- drivers/crypto/octeontx2/meson.build | 4 +- .../crypto/octeontx2/otx2_crypto_adapter.c | 77 +++++++++++++++++++ .../crypto/octeontx2/otx2_crypto_adapter.h | 21 +++++ .../octeontx2/otx2_cryptodev_hw_access.h | 12 +++ drivers/crypto/octeontx2/otx2_cryptodev_qp.h | 7 ++ .../rte_pmd_octeontx2_crypto_version.map | 10 +++ drivers/event/octeontx2/meson.build | 2 +- drivers/event/octeontx2/otx2_evdev.c | 5 ++ 8 files changed, 136 insertions(+), 2 deletions(-) create mode 100644 drivers/crypto/octeontx2/otx2_crypto_adapter.c create mode 100644 drivers/crypto/octeontx2/otx2_crypto_adapter.h diff --git a/drivers/crypto/octeontx2/meson.build b/drivers/crypto/octeontx2/meson.build index 148ec184a..063a553f2 100644 --- a/drivers/crypto/octeontx2/meson.build +++ b/drivers/crypto/octeontx2/meson.build @@ -10,6 +10,7 @@ deps += ['bus_pci'] deps += ['common_cpt'] deps += ['common_octeontx2'] deps += ['ethdev'] +deps += ['eventdev'] deps += ['security'] name = 'octeontx2_crypto' @@ -18,7 +19,8 @@ sources = files('otx2_cryptodev.c', 'otx2_cryptodev_hw_access.c', 'otx2_cryptodev_mbox.c', 'otx2_cryptodev_ops.c', - 'otx2_cryptodev_sec.c') + 'otx2_cryptodev_sec.c', + 'otx2_crypto_adapter.c') extra_flags = [] # This integrated controller runs only on a arm64 machine, remove 32bit warnings diff --git a/drivers/crypto/octeontx2/otx2_crypto_adapter.c b/drivers/crypto/octeontx2/otx2_crypto_adapter.c new file mode 100644 index 000000000..cfde7ec4f --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_crypto_adapter.c @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#include +#include + +#include "otx2_cryptodev_hw_access.h" +#include "otx2_cryptodev_qp.h" +#include "otx2_cryptodev_mbox.h" +#include "otx2_crypto_adapter.h" + +int +otx2_ca_caps_get(const struct rte_eventdev *dev, + const struct rte_cryptodev *cdev, uint32_t *caps) +{ + RTE_SET_USED(dev); + RTE_SET_USED(cdev); + + *caps = RTE_EVENT_CRYPTO_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND | + RTE_EVENT_CRYPTO_ADAPTER_CAP_INTERNAL_PORT_OP_NEW; + + return 0; +} + +int +otx2_ca_qp_add(const struct rte_eventdev *dev, const struct rte_cryptodev *cdev, + int32_t queue_pair_id, const struct rte_event *event) +{ + union otx2_cpt_af_lf_ctl2 af_lf_ctl2; + struct otx2_cpt_qp *qp; + int ret; + + RTE_SET_USED(dev); + + qp = cdev->data->queue_pairs[queue_pair_id]; + + qp->ca_enable = 1; + rte_memcpy(&qp->ev, event, sizeof(struct rte_event)); + + ret = otx2_cpt_af_reg_read(cdev, OTX2_CPT_AF_LF_CTL2(qp->id), + &af_lf_ctl2.u); + if (ret) + return ret; + + af_lf_ctl2.s.sso_pf_func = otx2_sso_pf_func_get(); + ret = otx2_cpt_af_reg_write(cdev, OTX2_CPT_AF_LF_CTL2(qp->id), + af_lf_ctl2.u); + + return ret; +} + +int +otx2_ca_qp_del(const struct rte_eventdev *dev, const struct rte_cryptodev *cdev, + int32_t queue_pair_id) +{ + union otx2_cpt_af_lf_ctl2 af_lf_ctl2; + struct otx2_cpt_qp *qp; + int ret; + + RTE_SET_USED(dev); + + qp = cdev->data->queue_pairs[queue_pair_id]; + qp->ca_enable = 0; + memset(&qp->ev, 0, sizeof(struct rte_event)); + + ret = otx2_cpt_af_reg_read(cdev, OTX2_CPT_AF_LF_CTL2(qp->id), + &af_lf_ctl2.u); + if (ret) + return ret; + + af_lf_ctl2.s.sso_pf_func = 0; + ret = otx2_cpt_af_reg_write(cdev, OTX2_CPT_AF_LF_CTL2(qp->id), + af_lf_ctl2.u); + + return ret; +} diff --git a/drivers/crypto/octeontx2/otx2_crypto_adapter.h b/drivers/crypto/octeontx2/otx2_crypto_adapter.h new file mode 100644 index 000000000..c8f02c0af --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_crypto_adapter.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#ifndef _OTX2_CRYPTO_ADAPTER_H_ +#define _OTX2_CRYPTO_ADAPTER_H_ + +__rte_internal +int otx2_ca_caps_get(const struct rte_eventdev *dev, + const struct rte_cryptodev *cdev, uint32_t *caps); + +__rte_internal +int otx2_ca_qp_add(const struct rte_eventdev *dev, + const struct rte_cryptodev *cdev, int32_t queue_pair_id, + const struct rte_event *event); + +__rte_internal +int otx2_ca_qp_del(const struct rte_eventdev *dev, + const struct rte_cryptodev *cdev, int32_t queue_pair_id); + +#endif /* _OTX2_CRYPTO_ADAPTER_H_ */ diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h b/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h index 43db6a642..a435818e0 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h +++ b/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h @@ -42,6 +42,7 @@ #define OTX2_CPT_LF_NQ(a) (0x400ull | (uint64_t)(a) << 3) #define OTX2_CPT_AF_LF_CTL(a) (0x27000ull | (uint64_t)(a) << 3) +#define OTX2_CPT_AF_LF_CTL2(a) (0x29000ull | (uint64_t)(a) << 3) #define OTX2_CPT_LF_BAR2(vf, q_id) \ ((vf)->otx2_dev.bar2 + \ @@ -110,6 +111,17 @@ union otx2_cpt_af_lf_ctl { } s; }; +union otx2_cpt_af_lf_ctl2 { + uint64_t u; + struct { + uint64_t exe_no_swap : 1; + uint64_t exe_ldwb : 1; + uint64_t reserved_2_31 : 30; + uint64_t sso_pf_func : 16; + uint64_t nix_pf_func : 16; + } s; +}; + union otx2_cpt_lf_q_grp_ptr { uint64_t u; struct { diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_qp.h b/drivers/crypto/octeontx2/otx2_cryptodev_qp.h index 9d48da45f..96ff4eb41 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_qp.h +++ b/drivers/crypto/octeontx2/otx2_cryptodev_qp.h @@ -6,6 +6,7 @@ #define _OTX2_CRYPTODEV_QP_H_ #include +#include #include #include @@ -30,6 +31,12 @@ struct otx2_cpt_qp { /**< Metabuf info required to support operations on the queue pair */ rte_iova_t iq_dma_addr; /**< Instruction queue address */ + struct rte_event ev; + /**< Event information required for binding cryptodev queue to + * eventdev queue. Used by crypto adapter. + */ + uint8_t ca_enable; + /**< Set when queue pair is added to crypto adapter */ }; #endif /* _OTX2_CRYPTODEV_QP_H_ */ diff --git a/drivers/crypto/octeontx2/rte_pmd_octeontx2_crypto_version.map b/drivers/crypto/octeontx2/rte_pmd_octeontx2_crypto_version.map index 4a76d1d52..b47a7ad3e 100644 --- a/drivers/crypto/octeontx2/rte_pmd_octeontx2_crypto_version.map +++ b/drivers/crypto/octeontx2/rte_pmd_octeontx2_crypto_version.map @@ -1,3 +1,13 @@ DPDK_21 { local: *; }; + +INTERNAL { + global: + + otx2_ca_caps_get; + otx2_ca_qp_add; + otx2_ca_qp_del; + + local: *; +}; diff --git a/drivers/event/octeontx2/meson.build b/drivers/event/octeontx2/meson.build index 0ade51cec..8585c54e4 100644 --- a/drivers/event/octeontx2/meson.build +++ b/drivers/event/octeontx2/meson.build @@ -24,6 +24,6 @@ foreach flag: extra_flags endif endforeach -deps += ['bus_pci', 'common_octeontx2', 'mempool_octeontx2', 'pmd_octeontx2'] +deps += ['bus_pci', 'common_octeontx2', 'mempool_octeontx2', 'pmd_octeontx2', 'pmd_octeontx2_crypto'] includes += include_directories('../../crypto/octeontx2') diff --git a/drivers/event/octeontx2/otx2_evdev.c b/drivers/event/octeontx2/otx2_evdev.c index b8b57c388..fe21787b6 100644 --- a/drivers/event/octeontx2/otx2_evdev.c +++ b/drivers/event/octeontx2/otx2_evdev.c @@ -12,6 +12,7 @@ #include #include +#include "otx2_crypto_adapter.h" #include "otx2_evdev_stats.h" #include "otx2_evdev.h" #include "otx2_irq.h" @@ -1587,6 +1588,10 @@ static struct rte_eventdev_ops otx2_sso_ops = { .timer_adapter_caps_get = otx2_tim_caps_get, + .crypto_adapter_caps_get = otx2_ca_caps_get, + .crypto_adapter_queue_pair_add = otx2_ca_qp_add, + .crypto_adapter_queue_pair_del = otx2_ca_qp_del, + .xstats_get = otx2_sso_xstats_get, .xstats_reset = otx2_sso_xstats_reset, .xstats_get_names = otx2_sso_xstats_get_names, From patchwork Tue Sep 8 10:09:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Dwivedi X-Patchwork-Id: 76912 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F1288A04AA; Tue, 8 Sep 2020 12:10:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CE05B2BAB; Tue, 8 Sep 2020 12:10:53 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id A03021C0D2 for ; Tue, 8 Sep 2020 12:10:52 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 088A57pb020476; Tue, 8 Sep 2020 03:10:51 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=R43x4jYdVSOhpDAbdt57Lc7kb1meNx5oJiuOKW3h9U4=; b=eiuo4Ysf73VBUz/DPsM6JNqJK1AiEScqbfH0t+lQDPtJbqYAFlbfD0k2hiLD0jAdX59j tnKLf7yRFME+DZ9TndDlp1CCaELA893xU8hCop4MqsVZac/JtWZTZEtEEtHMHUVTy8pi VSyLO83IXEgcNQnbVUJ7qQM+Kg6OFuiPr55fhJPv/rMoCIHG8WkN6I03KALVNyU6H+dC tmg0GpD05eS2ZAwtjwNw19LPghqVnyWEquuTEvG1uVV3GM80imY2O716G5CE5f3XNaLU /4mqbCAIkYQFLGXkMsVurR1bIkhdd07KgFCH/b37RKHWmtLOJ4Bb0EaZPjcqE/+X1Q9o 2A== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 33c81pu22q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 08 Sep 2020 03:10:51 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 8 Sep 2020 03:10:50 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 8 Sep 2020 03:10:49 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 8 Sep 2020 03:10:49 -0700 Received: from hyd1349.t110.caveonetworks.com (unknown [10.29.45.13]) by maili.marvell.com (Postfix) with ESMTP id AEDA63F703F; Tue, 8 Sep 2020 03:10:47 -0700 (PDT) From: Ankur Dwivedi To: CC: , , , , Ankur Dwivedi Date: Tue, 8 Sep 2020 15:39:56 +0530 Message-ID: <20200908100956.25868-4-adwivedi@marvell.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200908100956.25868-1-adwivedi@marvell.com> References: <20200908100956.25868-1-adwivedi@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-09-08_05:2020-09-08, 2020-09-08 signatures=0 Subject: [dpdk-dev] [PATCH 3/3] event/octeontx2: add crypto adapter datapath X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In the op new mode of crypto adapter, the completed crypto operation is submitted to the event device by the OCTEON TX2 crypto PMD. During event device dequeue the result of crypto operation is checked. Signed-off-by: Ankur Dwivedi --- drivers/common/cpt/cpt_common.h | 1 + drivers/crypto/octeontx2/otx2_ca_helper.h | 75 +++++++++++++++++++ drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 47 ++++++++++++ .../rte_pmd_octeontx2_crypto_version.map | 1 + drivers/event/octeontx2/meson.build | 1 + drivers/event/octeontx2/otx2_worker.h | 10 ++- drivers/event/octeontx2/otx2_worker_dual.h | 11 ++- 7 files changed, 142 insertions(+), 4 deletions(-) create mode 100644 drivers/crypto/octeontx2/otx2_ca_helper.h diff --git a/drivers/common/cpt/cpt_common.h b/drivers/common/cpt/cpt_common.h index 0141b2aed..1ce28e90b 100644 --- a/drivers/common/cpt/cpt_common.h +++ b/drivers/common/cpt/cpt_common.h @@ -72,6 +72,7 @@ struct cpt_request_info { uint64_t ei3; } ist; uint8_t *rptr; + const struct otx2_cpt_qp *qp; /** Control path fields */ uint64_t time_out; diff --git a/drivers/crypto/octeontx2/otx2_ca_helper.h b/drivers/crypto/octeontx2/otx2_ca_helper.h new file mode 100644 index 000000000..a6c758cf9 --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_ca_helper.h @@ -0,0 +1,75 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#ifndef _OTX2_CA_HELPER_H_ +#define _OTX2_CA_HELPER_H_ + +#include +#include +#include + +#include "cpt_pmd_logs.h" +#include "cpt_ucode.h" + +#include "otx2_cryptodev.h" +#include "otx2_cryptodev_hw_access.h" +#include "otx2_cryptodev_ops_helper.h" +#include "otx2_cryptodev_qp.h" + +static inline void +otx2_ca_deq_post_process(const struct otx2_cpt_qp *qp, + struct rte_crypto_op *cop, uintptr_t *rsp, + uint8_t cc) +{ + if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + if (likely(cc == NO_ERR)) { + /* Verify authentication data if required */ + if (unlikely(rsp[2])) + compl_auth_verify(cop, (uint8_t *)rsp[2], + rsp[3]); + else + cop->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + } else { + if (cc == ERR_GC_ICV_MISCOMPARE) + cop->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; + else + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + } + + if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) { + sym_session_clear(otx2_cryptodev_driver_id, + cop->sym->session); + memset(cop->sym->session, 0, + rte_cryptodev_sym_get_existing_header_session_size( + cop->sym->session)); + rte_mempool_put(qp->sess_mp, cop->sym->session); + cop->sym->session = NULL; + } + } + +} + +static __rte_always_inline uint64_t +otx2_handle_crypto_event(uint64_t get_work1) +{ + struct cpt_request_info *req; + struct rte_crypto_op *cop; + uintptr_t *rsp; + void *metabuf; + uint8_t cc; + + req = (struct cpt_request_info *)(get_work1 >> 3); + cc = otx2_cpt_compcode_get(req); + + rsp = req->op; + metabuf = (void *)rsp[0]; + cop = (void *)rsp[1]; + + otx2_ca_deq_post_process(req->qp, cop, rsp, cc); + + rte_mempool_put(req->qp->meta_info.pool, metabuf); + + return (uint64_t)(cop); +} +#endif /* _OTX2_CA_HELPER_H_ */ diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c index 77842b4ad..53c118287 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c @@ -417,6 +417,48 @@ sym_session_configure(int driver_id, struct rte_crypto_sym_xform *xform, return -ENOTSUP; } +static __rte_always_inline void __rte_hot +otx2_ca_enqueue_req(const struct otx2_cpt_qp *qp, + struct cpt_request_info *req, + void *lmtline) +{ + union cpt_inst_s inst; + uint64_t lmt_status; + + inst.u[0] = 0; + inst.s9x.res_addr = req->comp_baddr; + inst.u[2] = 0; + inst.u[3] = 0; + + inst.s9x.ei0 = req->ist.ei0; + inst.s9x.ei1 = req->ist.ei1; + inst.s9x.ei2 = req->ist.ei2; + inst.s9x.ei3 = req->ist.ei3; + + inst.s9x.qord = 1; + inst.s9x.grp = qp->ev.queue_id; + inst.s9x.tt = qp->ev.sched_type; + inst.s9x.tag = (RTE_EVENT_TYPE_CRYPTODEV << 28) | + qp->ev.flow_id; + inst.s9x.wq_ptr = (uint64_t)req; + req->qp = qp; + + do { + /* Copy CPT command to LMTLINE */ + memcpy(lmtline, &inst, sizeof(inst)); + + /* + * Make sure compiler does not reorder memcpy and ldeor. + * LMTST transactions are always flushed from the write + * buffer immediately, a DMB is not required to push out + * LMTSTs. + */ + rte_cio_wmb(); + lmt_status = otx2_lmt_submit(qp->lf_nq_reg); + } while (lmt_status == 0); + +} + static __rte_always_inline int32_t __rte_hot otx2_cpt_enqueue_req(const struct otx2_cpt_qp *qp, struct pending_queue *pend_q, @@ -426,6 +468,11 @@ otx2_cpt_enqueue_req(const struct otx2_cpt_qp *qp, union cpt_inst_s inst; uint64_t lmt_status; + if (qp->ca_enable) { + otx2_ca_enqueue_req(qp, req, lmtline); + return 0; + } + if (unlikely(pend_q->pending_count >= OTX2_CPT_DEFAULT_CMD_QLEN)) return -EAGAIN; diff --git a/drivers/crypto/octeontx2/rte_pmd_octeontx2_crypto_version.map b/drivers/crypto/octeontx2/rte_pmd_octeontx2_crypto_version.map index b47a7ad3e..d23764d2c 100644 --- a/drivers/crypto/octeontx2/rte_pmd_octeontx2_crypto_version.map +++ b/drivers/crypto/octeontx2/rte_pmd_octeontx2_crypto_version.map @@ -5,6 +5,7 @@ DPDK_21 { INTERNAL { global: + otx2_cryptodev_driver_id; otx2_ca_caps_get; otx2_ca_qp_add; otx2_ca_qp_del; diff --git a/drivers/event/octeontx2/meson.build b/drivers/event/octeontx2/meson.build index 8585c54e4..08863a3e3 100644 --- a/drivers/event/octeontx2/meson.build +++ b/drivers/event/octeontx2/meson.build @@ -27,3 +27,4 @@ endforeach deps += ['bus_pci', 'common_octeontx2', 'mempool_octeontx2', 'pmd_octeontx2', 'pmd_octeontx2_crypto'] includes += include_directories('../../crypto/octeontx2') +includes += include_directories('../../common/cpt') diff --git a/drivers/event/octeontx2/otx2_worker.h b/drivers/event/octeontx2/otx2_worker.h index 924ff7ff4..5ea2c2de2 100644 --- a/drivers/event/octeontx2/otx2_worker.h +++ b/drivers/event/octeontx2/otx2_worker.h @@ -9,6 +9,7 @@ #include #include +#include "otx2_ca_helper.h" #include "otx2_evdev.h" #include "otx2_ethdev_sec_tx.h" @@ -66,8 +67,10 @@ otx2_ssogws_get_work(struct otx2_ssogws *ws, struct rte_event *ev, ws->cur_tt = event.sched_type; ws->cur_grp = event.queue_id; - if (event.sched_type != SSO_TT_EMPTY && - event.event_type == RTE_EVENT_TYPE_ETHDEV) { + if (event.sched_type == SSO_TT_EMPTY) + goto setev_and_return; + + if (event.event_type == RTE_EVENT_TYPE_ETHDEV) { otx2_wqe_to_mbuf(get_work1, mbuf, event.sub_event_type, (uint32_t) event.get_work0, flags, lookup_mem); /* Extracting tstamp, if PTP enabled*/ @@ -76,8 +79,11 @@ otx2_ssogws_get_work(struct otx2_ssogws *ws, struct rte_event *ev, otx2_nix_mbuf_to_tstamp((struct rte_mbuf *)mbuf, ws->tstamp, flags, (uint64_t *)tstamp_ptr); get_work1 = mbuf; + } else if (event.event_type == RTE_EVENT_TYPE_CRYPTODEV) { + get_work1 = otx2_handle_crypto_event(get_work1); } +setev_and_return: ev->event = event.get_work0; ev->u64 = get_work1; diff --git a/drivers/event/octeontx2/otx2_worker_dual.h b/drivers/event/octeontx2/otx2_worker_dual.h index 60aa14cca..4bf4e0170 100644 --- a/drivers/event/octeontx2/otx2_worker_dual.h +++ b/drivers/event/octeontx2/otx2_worker_dual.h @@ -9,6 +9,8 @@ #include #include + +#include "otx2_ca_helper.h" #include "otx2_evdev.h" /* SSO Operations */ @@ -63,8 +65,10 @@ otx2_ssogws_dual_get_work(struct otx2_ssogws_state *ws, ws->cur_tt = event.sched_type; ws->cur_grp = event.queue_id; - if (event.sched_type != SSO_TT_EMPTY && - event.event_type == RTE_EVENT_TYPE_ETHDEV) { + if (event.sched_type == SSO_TT_EMPTY) + goto setev_and_return; + + if (event.event_type == RTE_EVENT_TYPE_ETHDEV) { uint8_t port = event.sub_event_type; event.sub_event_type = 0; @@ -82,8 +86,11 @@ otx2_ssogws_dual_get_work(struct otx2_ssogws_state *ws, otx2_nix_mbuf_to_tstamp((struct rte_mbuf *)mbuf, tstamp, flags, (uint64_t *)tstamp_ptr); get_work1 = mbuf; + } else if (event.event_type == RTE_EVENT_TYPE_CRYPTODEV) { + get_work1 = otx2_handle_crypto_event(get_work1); } +setev_and_return: ev->event = event.get_work0; ev->u64 = get_work1;