From patchwork Fri Jun 25 05:56:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 94831 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 196E7A0C40; Fri, 25 Jun 2021 07:58:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AE12B41102; Fri, 25 Jun 2021 07:57:40 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 2D42340E25 for ; Fri, 25 Jun 2021 07:57:39 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 15P5nxkH007987; Thu, 24 Jun 2021 22:57:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=ckwI5lqdupf0gsEHTKL9GeWdZVdox2rsTC5luoVcDrs=; b=Tf+IchrEZ2nvUqJoFuicwXwiwh35WnCjaIOScbTrjpLwDMUTbrgH7BttDQc0hEDI3hs2 tkhJgHJ7gzKs8ocMTpkQKqzkyF1jyW/hhTjfIO2EjCkGPUBX0wHx24lEl0aqPp3PUObD ZFgo31jPFPmJ+6zVIWwxd1Cm1upqZ4mJismBLef/ZiCdF6NHk+1fPuOdrHt4An9fNNdp /Ny0PIMI7BR9Dp27ImEQIz5DAsfltRMCo30J+lC3TVlN8H+rvLgj4jgS6Bk7nN2rlyHb UNuEkRllWkZKgA1N5As1BtAs7RGY+TgMhya4Dz+MkjJD2+iC6cM4usxgQBrkiAFSjoSb hw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 39d24dhk1r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 24 Jun 2021 22:57:38 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 24 Jun 2021 22:57:36 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Thu, 24 Jun 2021 22:57:36 -0700 Received: from HY-LT1002.marvell.com (HY-LT1002.marvell.com [10.28.176.218]) by maili.marvell.com (Postfix) with ESMTP id 917DE3F7041; Thu, 24 Jun 2021 22:57:33 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , "Ankur Dwivedi" , Tejasree Kondoj , , Archana Muniganti Date: Fri, 25 Jun 2021 11:26:18 +0530 Message-ID: <1624600591-29841-8-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1624600591-29841-1-git-send-email-anoobj@marvell.com> References: <1624600591-29841-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: eay93H_mC08NKPauM9vgrCt9qKFqX7Gn X-Proofpoint-ORIG-GUID: eay93H_mC08NKPauM9vgrCt9qKFqX7Gn X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-06-25_02:2021-06-24, 2021-06-25 signatures=0 Subject: [dpdk-dev] [PATCH v2 07/20] crypto/cnxk: add dequeue burst op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add dequeue_burst op in cn9k & cn10k. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj --- doc/guides/cryptodevs/features/cn10k.ini | 3 + doc/guides/cryptodevs/features/cn9k.ini | 3 + drivers/crypto/cnxk/cn10k_cryptodev.c | 4 ++ drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 105 ++++++++++++++++++++++++++++++ drivers/crypto/cnxk/cn9k_cryptodev.c | 4 ++ drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 103 +++++++++++++++++++++++++++++ 6 files changed, 222 insertions(+) diff --git a/doc/guides/cryptodevs/features/cn10k.ini b/doc/guides/cryptodevs/features/cn10k.ini index 0aa097d..7f433fa 100644 --- a/doc/guides/cryptodevs/features/cn10k.ini +++ b/doc/guides/cryptodevs/features/cn10k.ini @@ -4,6 +4,9 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Symmetric crypto = Y +HW Accelerated = Y +Symmetric sessionless = Y ; ; Supported crypto algorithms of 'cn10k' crypto driver. diff --git a/doc/guides/cryptodevs/features/cn9k.ini b/doc/guides/cryptodevs/features/cn9k.ini index 64ee929..9c9d54d 100644 --- a/doc/guides/cryptodevs/features/cn9k.ini +++ b/doc/guides/cryptodevs/features/cn9k.ini @@ -4,6 +4,9 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Symmetric crypto = Y +HW Accelerated = Y +Symmetric sessionless = Y ; ; Supported crypto algorithms of 'cn9k' crypto driver. diff --git a/drivers/crypto/cnxk/cn10k_cryptodev.c b/drivers/crypto/cnxk/cn10k_cryptodev.c index 53f7a94..31addc0 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev.c @@ -80,6 +80,10 @@ cn10k_cpt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, dev->dev_ops = &cn10k_cpt_ops; dev->driver_id = cn10k_cryptodev_driver_id; + dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | + RTE_CRYPTODEV_FF_HW_ACCELERATED | + RTE_CRYPTODEV_FF_SYM_SESSIONLESS; + cn10k_cpt_set_enqdeq_fns(dev); return 0; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 5dd2cd2..ec301f4 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -190,10 +190,115 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return count + i; } +static inline void +cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, + struct rte_crypto_op *cop, + struct cpt_inflight_req *infl_req) +{ + struct cpt_cn10k_res_s *res = (struct cpt_cn10k_res_s *)&infl_req->res; + unsigned int sz; + + if (likely(res->compcode == CPT_COMP_GOOD || + res->compcode == CPT_COMP_WARN)) { + if (unlikely(res->uc_compcode)) { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + + plt_dp_info("Request failed with microcode error"); + plt_dp_info("MC completion code 0x%x", + res->uc_compcode); + goto temp_sess_free; + } + + cop->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + } else { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + plt_dp_info("HW completion code 0x%x", res->compcode); + + switch (res->compcode) { + case CPT_COMP_INSTERR: + plt_dp_err("Request failed with instruction error"); + break; + case CPT_COMP_FAULT: + plt_dp_err("Request failed with DMA fault"); + break; + case CPT_COMP_HWERR: + plt_dp_err("Request failed with hardware error"); + break; + default: + plt_dp_err( + "Request failed with unknown completion code"); + } + } + +temp_sess_free: + if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) { + if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + sym_session_clear(cn10k_cryptodev_driver_id, + cop->sym->session); + sz = rte_cryptodev_sym_get_existing_header_session_size( + cop->sym->session); + memset(cop->sym->session, 0, sz); + rte_mempool_put(qp->sess_mp, cop->sym->session); + cop->sym->session = NULL; + } + } +} + +static uint16_t +cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) +{ + struct cpt_inflight_req *infl_req; + struct cnxk_cpt_qp *qp = qptr; + struct pending_queue *pend_q; + struct cpt_cn10k_res_s *res; + struct rte_crypto_op *cop; + int i, nb_pending; + + pend_q = &qp->pend_q; + + nb_pending = pend_q->pending_count; + + if (nb_ops > nb_pending) + nb_ops = nb_pending; + + for (i = 0; i < nb_ops; i++) { + infl_req = &pend_q->req_queue[pend_q->deq_head]; + + res = (struct cpt_cn10k_res_s *)&infl_req->res; + + if (unlikely(res->compcode == CPT_COMP_NOT_DONE)) { + if (unlikely(rte_get_timer_cycles() > + pend_q->time_out)) { + plt_err("Request timed out"); + pend_q->time_out = rte_get_timer_cycles() + + DEFAULT_COMMAND_TIMEOUT * + rte_get_timer_hz(); + } + break; + } + + MOD_INC(pend_q->deq_head, qp->lf.nb_desc); + + cop = infl_req->cop; + + ops[i] = cop; + + cn10k_cpt_dequeue_post_process(qp, cop, infl_req); + + if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF)) + rte_mempool_put(qp->meta_info.pool, infl_req->mdata); + } + + pend_q->pending_count -= i; + + return i; +} + void cn10k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev) { dev->enqueue_burst = cn10k_cpt_enqueue_burst; + dev->dequeue_burst = cn10k_cpt_dequeue_burst; rte_mb(); } diff --git a/drivers/crypto/cnxk/cn9k_cryptodev.c b/drivers/crypto/cnxk/cn9k_cryptodev.c index 4dbb40d..7908896 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev.c @@ -78,6 +78,10 @@ cn9k_cpt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, dev->dev_ops = &cn9k_cpt_ops; dev->driver_id = cn9k_cryptodev_driver_id; + dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | + RTE_CRYPTODEV_FF_HW_ACCELERATED | + RTE_CRYPTODEV_FF_SYM_SESSIONLESS; + cn9k_cpt_set_enqdeq_fns(dev); return 0; diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index f09f9ee..27076a8 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -155,10 +155,113 @@ cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return count; } +static inline void +cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, + struct cpt_inflight_req *infl_req) +{ + struct cpt_cn9k_res_s *res = (struct cpt_cn9k_res_s *)&infl_req->res; + unsigned int sz; + + if (likely(res->compcode == CPT_COMP_GOOD)) { + if (unlikely(res->uc_compcode)) { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + + plt_dp_info("Request failed with microcode error"); + plt_dp_info("MC completion code 0x%x", + res->uc_compcode); + goto temp_sess_free; + } + + cop->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + } else { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + plt_dp_info("HW completion code 0x%x", res->compcode); + + switch (res->compcode) { + case CPT_COMP_INSTERR: + plt_dp_err("Request failed with instruction error"); + break; + case CPT_COMP_FAULT: + plt_dp_err("Request failed with DMA fault"); + break; + case CPT_COMP_HWERR: + plt_dp_err("Request failed with hardware error"); + break; + default: + plt_dp_err( + "Request failed with unknown completion code"); + } + } + +temp_sess_free: + if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) { + if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + sym_session_clear(cn9k_cryptodev_driver_id, + cop->sym->session); + sz = rte_cryptodev_sym_get_existing_header_session_size( + cop->sym->session); + memset(cop->sym->session, 0, sz); + rte_mempool_put(qp->sess_mp, cop->sym->session); + cop->sym->session = NULL; + } + } +} + +static uint16_t +cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) +{ + struct cnxk_cpt_qp *qp = qptr; + struct pending_queue *pend_q; + struct cpt_inflight_req *infl_req; + struct cpt_cn9k_res_s *res; + struct rte_crypto_op *cop; + uint32_t pq_deq_head; + int i; + + pend_q = &qp->pend_q; + + nb_ops = RTE_MIN(nb_ops, pend_q->pending_count); + + pq_deq_head = pend_q->deq_head; + + for (i = 0; i < nb_ops; i++) { + infl_req = &pend_q->req_queue[pq_deq_head]; + + res = (struct cpt_cn9k_res_s *)&infl_req->res; + + if (unlikely(res->compcode == CPT_COMP_NOT_DONE)) { + if (unlikely(rte_get_timer_cycles() > + pend_q->time_out)) { + plt_err("Request timed out"); + pend_q->time_out = rte_get_timer_cycles() + + DEFAULT_COMMAND_TIMEOUT * + rte_get_timer_hz(); + } + break; + } + + MOD_INC(pq_deq_head, qp->lf.nb_desc); + + cop = infl_req->cop; + + ops[i] = cop; + + cn9k_cpt_dequeue_post_process(qp, cop, infl_req); + + if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF)) + rte_mempool_put(qp->meta_info.pool, infl_req->mdata); + } + + pend_q->pending_count -= i; + pend_q->deq_head = pq_deq_head; + + return i; +} void cn9k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev) { dev->enqueue_burst = cn9k_cpt_enqueue_burst; + dev->dequeue_burst = cn9k_cpt_dequeue_burst; rte_mb(); }