From patchwork Wed Jun 2 16:43:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 93816 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA44FA0524; Wed, 2 Jun 2021 18:45:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EBD1F410DE; Wed, 2 Jun 2021 18:44:52 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 14DD3410DE for ; Wed, 2 Jun 2021 18:44:51 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 152Ga7SM032332; Wed, 2 Jun 2021 09:44:51 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=TOXPQ6kJYRLdmsytlMQ2ET5O7lC/5daEuJbJOobu+m0=; b=PGsLzH/235GvUk89I8dVQPIPG0WslYMi16M9xi7bdJ2ToGTOE40queGkE+imJZhULY3Q UIYx4mWm4oa+u89KFuw1O7Q7nC0IQ4//W6EY1jwft2XQTZqf42qkLs3/VBJHIQEHTf/y PNrFvw9Z+sdhSOKR1lLQOVKmtzmGB+Jjig9oWRvQ/mq4tLg5bGzs1AyhT+4QANAtaU1r ATI5bOoqTXF+gZi3pfa743DyiM9tQya682RDe9DIOimR39FLx8QL+Wf/uRfoaPYhXixp LoUusZRvUQKeITldGrbm3lEUr2o37dkrRBcFKy6hUHuc0Od2p9/SbXJ2tgB5zEOi0do9 1g== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 38wufgur01-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 02 Jun 2021 09:44:51 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 2 Jun 2021 09:44:49 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 2 Jun 2021 09:44:49 -0700 Received: from HY-LT1002.marvell.com (unknown [10.193.70.1]) by maili.marvell.com (Postfix) with ESMTP id B901C3F703F; Wed, 2 Jun 2021 09:44:44 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , "Ankur Dwivedi" , Tejasree Kondoj , , Archana Muniganti Date: Wed, 2 Jun 2021 22:13:29 +0530 Message-ID: <1622652221-22732-9-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1622652221-22732-1-git-send-email-anoobj@marvell.com> References: <1622652221-22732-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: hViZWwhg2a9v04KXCsP5wpG4Z5ocAYmS X-Proofpoint-GUID: hViZWwhg2a9v04KXCsP5wpG4Z5ocAYmS X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-06-02_09:2021-06-02, 2021-06-02 signatures=0 Subject: [dpdk-dev] [PATCH 08/20] crypto/cnxk: add dequeue burst op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add dequeue_burst op in cn9k & cn10k. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj --- doc/guides/cryptodevs/features/cn10k.ini | 3 + doc/guides/cryptodevs/features/cn9k.ini | 3 + drivers/crypto/cnxk/cn10k_cryptodev.c | 4 ++ drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 105 ++++++++++++++++++++++++++++++ drivers/crypto/cnxk/cn9k_cryptodev.c | 4 ++ drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 103 +++++++++++++++++++++++++++++ 6 files changed, 222 insertions(+) diff --git a/doc/guides/cryptodevs/features/cn10k.ini b/doc/guides/cryptodevs/features/cn10k.ini index 0aa097d..7f433fa 100644 --- a/doc/guides/cryptodevs/features/cn10k.ini +++ b/doc/guides/cryptodevs/features/cn10k.ini @@ -4,6 +4,9 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Symmetric crypto = Y +HW Accelerated = Y +Symmetric sessionless = Y ; ; Supported crypto algorithms of 'cn10k' crypto driver. diff --git a/doc/guides/cryptodevs/features/cn9k.ini b/doc/guides/cryptodevs/features/cn9k.ini index 64ee929..9c9d54d 100644 --- a/doc/guides/cryptodevs/features/cn9k.ini +++ b/doc/guides/cryptodevs/features/cn9k.ini @@ -4,6 +4,9 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Symmetric crypto = Y +HW Accelerated = Y +Symmetric sessionless = Y ; ; Supported crypto algorithms of 'cn9k' crypto driver. diff --git a/drivers/crypto/cnxk/cn10k_cryptodev.c b/drivers/crypto/cnxk/cn10k_cryptodev.c index a34dbbf..2abd396 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev.c @@ -79,6 +79,10 @@ cn10k_cpt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, cnxk_cpt_caps_populate(vf); + dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | + RTE_CRYPTODEV_FF_HW_ACCELERATED | + RTE_CRYPTODEV_FF_SYM_SESSIONLESS; + cn10k_cpt_set_enqdeq_fns(dev); return 0; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index afdd43c..83b24c9 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -190,10 +190,115 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return count + i; } +static inline void +cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, + struct rte_crypto_op *cop, + struct cpt_inflight_req *infl_req) +{ + struct cpt_cn10k_res_s *res = (struct cpt_cn10k_res_s *)&infl_req->res; + unsigned int sz; + + if (likely(res->compcode == CPT_COMP_GOOD || + res->compcode == CPT_COMP_WARN)) { + if (unlikely(res->uc_compcode)) { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + + CPT_LOG_DP_DEBUG("Request failed with microcode error"); + CPT_LOG_DP_DEBUG("MC completion code 0x%x", + res->uc_compcode); + goto temp_sess_free; + } + + cop->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + } else { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + CPT_LOG_DP_DEBUG("HW completion code 0x%x", res->compcode); + + switch (res->compcode) { + case CPT_COMP_INSTERR: + CPT_LOG_DP_ERR("Request failed with instruction error"); + break; + case CPT_COMP_FAULT: + CPT_LOG_DP_ERR("Request failed with DMA fault"); + break; + case CPT_COMP_HWERR: + CPT_LOG_DP_ERR("Request failed with hardware error"); + break; + default: + CPT_LOG_DP_ERR( + "Request failed with unknown completion code"); + } + } + +temp_sess_free: + if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) { + if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + sym_session_clear(cn10k_cryptodev_driver_id, + cop->sym->session); + sz = rte_cryptodev_sym_get_existing_header_session_size( + cop->sym->session); + memset(cop->sym->session, 0, sz); + rte_mempool_put(qp->sess_mp, cop->sym->session); + cop->sym->session = NULL; + } + } +} + +static uint16_t +cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) +{ + struct cpt_inflight_req *infl_req; + struct cnxk_cpt_qp *qp = qptr; + struct pending_queue *pend_q; + struct cpt_cn10k_res_s *res; + struct rte_crypto_op *cop; + int i, nb_pending; + + pend_q = &qp->pend_q; + + nb_pending = pend_q->pending_count; + + if (nb_ops > nb_pending) + nb_ops = nb_pending; + + for (i = 0; i < nb_ops; i++) { + infl_req = &pend_q->req_queue[pend_q->deq_head]; + + res = (struct cpt_cn10k_res_s *)&infl_req->res; + + if (unlikely(res->compcode == CPT_COMP_NOT_DONE)) { + if (unlikely(rte_get_timer_cycles() > + pend_q->time_out)) { + plt_err("Request timed out"); + pend_q->time_out = rte_get_timer_cycles() + + DEFAULT_COMMAND_TIMEOUT * + rte_get_timer_hz(); + } + break; + } + + MOD_INC(pend_q->deq_head, qp->lf.nb_desc); + + cop = infl_req->cop; + + ops[i] = cop; + + cn10k_cpt_dequeue_post_process(qp, cop, infl_req); + + if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF)) + rte_mempool_put(qp->meta_info.pool, infl_req->mdata); + } + + pend_q->pending_count -= i; + + return i; +} + void cn10k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev) { dev->enqueue_burst = cn10k_cpt_enqueue_burst; + dev->dequeue_burst = cn10k_cpt_dequeue_burst; rte_mb(); } diff --git a/drivers/crypto/cnxk/cn9k_cryptodev.c b/drivers/crypto/cnxk/cn9k_cryptodev.c index 7470397..db61175 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev.c @@ -77,6 +77,10 @@ cn9k_cpt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, cnxk_cpt_caps_populate(vf); + dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | + RTE_CRYPTODEV_FF_HW_ACCELERATED | + RTE_CRYPTODEV_FF_SYM_SESSIONLESS; + cn9k_cpt_set_enqdeq_fns(dev); return 0; diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index 59e3cb0..41c411b 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -155,10 +155,113 @@ cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return count; } +static inline void +cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, + struct cpt_inflight_req *infl_req) +{ + struct cpt_cn9k_res_s *res = (struct cpt_cn9k_res_s *)&infl_req->res; + unsigned int sz; + + if (likely(res->compcode == CPT_COMP_GOOD)) { + if (unlikely(res->uc_compcode)) { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + + CPT_LOG_DP_DEBUG("Request failed with microcode error"); + CPT_LOG_DP_DEBUG("MC completion code 0x%x", + res->uc_compcode); + goto temp_sess_free; + } + + cop->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + } else { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + CPT_LOG_DP_DEBUG("HW completion code 0x%x", res->compcode); + + switch (res->compcode) { + case CPT_COMP_INSTERR: + CPT_LOG_DP_ERR("Request failed with instruction error"); + break; + case CPT_COMP_FAULT: + CPT_LOG_DP_ERR("Request failed with DMA fault"); + break; + case CPT_COMP_HWERR: + CPT_LOG_DP_ERR("Request failed with hardware error"); + break; + default: + CPT_LOG_DP_ERR( + "Request failed with unknown completion code"); + } + } + +temp_sess_free: + if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) { + if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + sym_session_clear(cn9k_cryptodev_driver_id, + cop->sym->session); + sz = rte_cryptodev_sym_get_existing_header_session_size( + cop->sym->session); + memset(cop->sym->session, 0, sz); + rte_mempool_put(qp->sess_mp, cop->sym->session); + cop->sym->session = NULL; + } + } +} + +static uint16_t +cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) +{ + struct cnxk_cpt_qp *qp = qptr; + struct pending_queue *pend_q; + struct cpt_inflight_req *infl_req; + struct cpt_cn9k_res_s *res; + struct rte_crypto_op *cop; + uint32_t pq_deq_head; + int i; + + pend_q = &qp->pend_q; + + nb_ops = RTE_MIN(nb_ops, pend_q->pending_count); + + pq_deq_head = pend_q->deq_head; + + for (i = 0; i < nb_ops; i++) { + infl_req = &pend_q->req_queue[pq_deq_head]; + + res = (struct cpt_cn9k_res_s *)&infl_req->res; + + if (unlikely(res->compcode == CPT_COMP_NOT_DONE)) { + if (unlikely(rte_get_timer_cycles() > + pend_q->time_out)) { + plt_err("Request timed out"); + pend_q->time_out = rte_get_timer_cycles() + + DEFAULT_COMMAND_TIMEOUT * + rte_get_timer_hz(); + } + break; + } + + MOD_INC(pq_deq_head, qp->lf.nb_desc); + + cop = infl_req->cop; + + ops[i] = cop; + + cn9k_cpt_dequeue_post_process(qp, cop, infl_req); + + if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF)) + rte_mempool_put(qp->meta_info.pool, infl_req->mdata); + } + + pend_q->pending_count -= i; + pend_q->deq_head = pq_deq_head; + + return i; +} void cn9k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev) { dev->enqueue_burst = cn9k_cpt_enqueue_burst; + dev->dequeue_burst = cn9k_cpt_dequeue_burst; rte_mb(); }