From patchwork Fri Oct 7 13:46:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 117560 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9A3B8A00C3; Fri, 7 Oct 2022 15:47:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AFED842B82; Fri, 7 Oct 2022 15:47:07 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id AE48D40042 for ; Fri, 7 Oct 2022 15:47:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665150425; x=1696686425; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Kqub+7FKR/7XKv8tN0wbaFn5+qRhVItfGHksWl6lWjc=; b=FQsvX8+1rSKugi1gpHtFa78AdHkJK1RYA6oRcn/Fwz0Iohh51zql3EuB Rk1O4cGJ0ES73X4X+HVm2Og9GGk8jxw5SPqY25iIUMCGHRCbr7X04iAp+ hHmgvQ94zjW8YO1F54hxKgIieqTVptGY2dwYBRe23HyDZBqu5S/x8xPcl AyEe2iMvXyTJ/1l2EMAq6VZr0Mc1eUhRB4BtMoxCpNe+JD38OUmQ2+K/y ZA9rP2OJhRmSTLLtS90M0JoMeiMYjFU9fOHLpS6kA1gGMnlD1e3u6MUMr cTyVzzdguBGpv01zhvKo0gwU9mAceIcwm43euB2but8f2pHKZHfM43bW0 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10493"; a="390032310" X-IronPort-AV: E=Sophos;i="5.95,166,1661842800"; d="scan'208";a="390032310" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Oct 2022 06:47:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10493"; a="625150221" X-IronPort-AV: E=Sophos;i="5.95,166,1661842800"; d="scan'208";a="625150221" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.222.163]) by orsmga002.jf.intel.com with ESMTP; 07 Oct 2022 06:47:03 -0700 From: Ciara Power To: Akhil Goyal , Fan Zhang Cc: dev@dpdk.org, kai.ji@intel.com, Ciara Power , Fan Zhang , Pablo de Lara Subject: [PATCH v5 1/4] test/crypto: fix wireless auth digest segment Date: Fri, 7 Oct 2022 13:46:50 +0000 Message-Id: <20221007134653.929034-2-ciara.power@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221007134653.929034-1-ciara.power@intel.com> References: <20220812132334.75707-1-ciara.power@intel.com> <20221007134653.929034-1-ciara.power@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The segment size for some tests was too small to hold the auth digest. This caused issues when using op->sym->auth.digest.data for comparisons in AESNI_MB PMD after a subsequent patch enables SGL. For example, if segment size is 2, and digest size is 4, then 4 bytes are read from op->sym->auth.digest.data, which overflows into the memory after the segment, rather than using the second segment that contains the remaining half of the digest. Fixes: 11c5485bb276 ("test/crypto: add scatter-gather tests for IP and OOP") Signed-off-by: Ciara Power Acked-by: Fan Zhang Acked-by: Pablo de Lara --- v4: Added failure check when appending digest size to buffer. --- app/test/test_cryptodev.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c index c6d47a035e..203b8b61fa 100644 --- a/app/test/test_cryptodev.c +++ b/app/test/test_cryptodev.c @@ -2990,6 +2990,16 @@ create_wireless_algo_auth_cipher_operation( remaining_off -= rte_pktmbuf_data_len(sgl_buf); sgl_buf = sgl_buf->next; } + + /* The last segment should be large enough to hold full digest */ + if (sgl_buf->data_len < auth_tag_len) { + rte_pktmbuf_free(sgl_buf->next); + sgl_buf->next = NULL; + TEST_ASSERT_NOT_NULL(rte_pktmbuf_append(sgl_buf, + auth_tag_len - sgl_buf->data_len), + "No room to append auth tag"); + } + sym_op->auth.digest.data = rte_pktmbuf_mtod_offset(sgl_buf, uint8_t *, remaining_off); sym_op->auth.digest.phys_addr = rte_pktmbuf_iova_offset(sgl_buf, From patchwork Fri Oct 7 13:46:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 117561 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 81C04A00C3; Fri, 7 Oct 2022 15:47:16 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 981AB42B89; Fri, 7 Oct 2022 15:47:10 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 2122942B87 for ; Fri, 7 Oct 2022 15:47:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665150429; x=1696686429; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=y1hEUA9wh9PDJBNY+nkreJkmRQAOWUr1SJrWjHeVS24=; b=UxahD88WCxs5eCDb4U7Fx2JMBWH03kFYO9yysFMEDsfos+OYDjD6RwBG Y7b5IRcjkif4H6M6akBpGSiCol/WF2y5ntJUw1OWoBdv6gu81i12LDLCP JE37n2or5kMeYKRcHG8LCgM819mIu/fS8KRd6PKtPcNYCL9zlu7wDj7iN YkMcgSuV7oSqicLUXHtp+SpNi3YuHo/O0qvYvGLM5IbTi8IfLf+pCbdhP 9nNhVAV+2BIpT/VydeOCuekVcYt9PUBy33SH+TpgavqmERSIjY5FNgo4F fRTTDW2MsvSblyTGYuIzVMX6xXPzri/cFda/9PU51ExvC8/MbLvlaAXqt w==; X-IronPort-AV: E=McAfee;i="6500,9779,10493"; a="390032319" X-IronPort-AV: E=Sophos;i="5.95,166,1661842800"; d="scan'208";a="390032319" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Oct 2022 06:47:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10493"; a="625150227" X-IronPort-AV: E=Sophos;i="5.95,166,1661842800"; d="scan'208";a="625150227" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.222.163]) by orsmga002.jf.intel.com with ESMTP; 07 Oct 2022 06:47:05 -0700 From: Ciara Power To: Kai Ji , Pablo de Lara Cc: dev@dpdk.org, gakhil@marvell.com, Ciara Power , Fan Zhang Subject: [PATCH v5 2/4] crypto/ipsec_mb: add remaining SGL support Date: Fri, 7 Oct 2022 13:46:51 +0000 Message-Id: <20221007134653.929034-3-ciara.power@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221007134653.929034-1-ciara.power@intel.com> References: <20220812132334.75707-1-ciara.power@intel.com> <20221007134653.929034-1-ciara.power@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The intel-ipsec-mb library supports SGL for GCM and ChaChaPoly algorithms using the JOB API. This support was added to AESNI_MB PMD previously, but the SGL feature flags could not be added due to no SGL support for other algorithms. This patch adds a workaround SGL approach for other algorithms using the JOB API. The segmented input buffers are copied into a linear buffer, which is passed as a single job to intel-ipsec-mb. The job is processed, and on return, the linear buffer is split into the original destination segments. Existing AESNI_MB testcases are passing with these feature flags added. Signed-off-by: Ciara Power Acked-by: Fan Zhang Acked-by: Pablo de Lara --- v3: - Reduced code duplication by adding a reusable function. - Changed int to uint64_t for total_len. v2: - Small improvements when copying segments to linear buffer. - Added documentation changes. --- doc/guides/cryptodevs/aesni_mb.rst | 1 - doc/guides/cryptodevs/features/aesni_mb.ini | 4 + doc/guides/rel_notes/release_22_11.rst | 5 + drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 180 ++++++++++++++++---- 4 files changed, 156 insertions(+), 34 deletions(-) diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst index 07222ee117..59c134556f 100644 --- a/doc/guides/cryptodevs/aesni_mb.rst +++ b/doc/guides/cryptodevs/aesni_mb.rst @@ -72,7 +72,6 @@ Protocol offloads: Limitations ----------- -* Chained mbufs are not supported. * Out-of-place is not supported for combined Crypto-CRC DOCSIS security protocol. * RTE_CRYPTO_CIPHER_DES_DOCSISBPI is not supported for combined Crypto-CRC diff --git a/doc/guides/cryptodevs/features/aesni_mb.ini b/doc/guides/cryptodevs/features/aesni_mb.ini index 3c648a391e..e4e965c35a 100644 --- a/doc/guides/cryptodevs/features/aesni_mb.ini +++ b/doc/guides/cryptodevs/features/aesni_mb.ini @@ -12,6 +12,10 @@ CPU AVX = Y CPU AVX2 = Y CPU AVX512 = Y CPU AESNI = Y +In Place SGL = Y +OOP SGL In SGL Out = Y +OOP SGL In LB Out = Y +OOP LB In SGL Out = Y OOP LB In LB Out = Y CPU crypto = Y Symmetric sessionless = Y diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index fc1649f5a2..3afc083563 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -122,6 +122,11 @@ New Features * Added AES-CCM support in lookaside protocol (IPsec) for CN9K & CN10K. * Added AES & DES DOCSIS algorithm support in lookaside crypto for CN9K. +* **Added SGL support to AESNI_MB PMD.** + + Added support for SGL to AESNI_MB PMD. Support for inplace, + OOP SGL in SGL out, OOP LB in SGL out, and OOP SGL in LB out added. + * **Added new operation for FFT processing in bbdev.** Added a new operation type in bbdev for FFT processing with new functions diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c index 8ec2364aa7..8477a09061 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c +++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c @@ -937,7 +937,7 @@ static inline uint64_t auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session, uint32_t oop, const uint32_t auth_offset, const uint32_t cipher_offset, const uint32_t auth_length, - const uint32_t cipher_length) + const uint32_t cipher_length, uint8_t lb_sgl) { struct rte_mbuf *m_src, *m_dst; uint8_t *p_src, *p_dst; @@ -945,7 +945,7 @@ auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session, uint32_t cipher_end, auth_end; /* Only cipher then hash needs special calculation. */ - if (!oop || session->chain_order != IMB_ORDER_CIPHER_HASH) + if (!oop || session->chain_order != IMB_ORDER_CIPHER_HASH || lb_sgl) return auth_offset; m_src = op->sym->m_src; @@ -1159,6 +1159,81 @@ handle_aead_sgl_job(IMB_JOB *job, IMB_MGR *mb_mgr, return 0; } +static uint64_t +sgl_linear_cipher_auth_len(IMB_JOB *job, uint64_t *auth_len) +{ + uint64_t cipher_len; + + if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN || + job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN) + cipher_len = (job->msg_len_to_cipher_in_bits >> 3) + + (job->cipher_start_src_offset_in_bits >> 3); + else + cipher_len = job->msg_len_to_cipher_in_bytes + + job->cipher_start_src_offset_in_bytes; + + if (job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN || + job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN) + *auth_len = (job->msg_len_to_hash_in_bits >> 3) + + job->hash_start_src_offset_in_bytes; + else if (job->hash_alg == IMB_AUTH_AES_GMAC) + *auth_len = job->u.GCM.aad_len_in_bytes; + else + *auth_len = job->msg_len_to_hash_in_bytes + + job->hash_start_src_offset_in_bytes; + + return RTE_MAX(*auth_len, cipher_len); +} + +static int +handle_sgl_linear(IMB_JOB *job, struct rte_crypto_op *op, uint32_t dst_offset, + struct aesni_mb_session *session) +{ + uint64_t auth_len, total_len; + uint8_t *src, *linear_buf = NULL; + int lb_offset = 0; + struct rte_mbuf *src_seg; + uint16_t src_len; + + total_len = sgl_linear_cipher_auth_len(job, &auth_len); + linear_buf = rte_zmalloc(NULL, total_len + job->auth_tag_output_len_in_bytes, 0); + if (linear_buf == NULL) { + IPSEC_MB_LOG(ERR, "Error allocating memory for SGL Linear Buffer\n"); + return -1; + } + + for (src_seg = op->sym->m_src; (src_seg != NULL) && + (total_len - lb_offset > 0); + src_seg = src_seg->next) { + src = rte_pktmbuf_mtod(src_seg, uint8_t *); + src_len = RTE_MIN(src_seg->data_len, total_len - lb_offset); + rte_memcpy(linear_buf + lb_offset, src, src_len); + lb_offset += src_len; + } + + job->src = linear_buf; + job->dst = linear_buf + dst_offset; + job->user_data2 = linear_buf; + + if (job->hash_alg == IMB_AUTH_AES_GMAC) + job->u.GCM.aad = linear_buf; + + if (session->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY) + job->auth_tag_output = linear_buf + lb_offset; + else + job->auth_tag_output = linear_buf + auth_len; + + return 0; +} + +static inline int +imb_lib_support_sgl_algo(IMB_CIPHER_MODE alg) +{ + if (alg == IMB_CIPHER_CHACHA20_POLY1305 + || alg == IMB_CIPHER_GCM) + return 1; + return 0; +} /** * Process a crypto operation and complete a IMB_JOB job structure for @@ -1171,7 +1246,8 @@ handle_aead_sgl_job(IMB_JOB *job, IMB_MGR *mb_mgr, * * @return * - 0 on success, the IMB_JOB will be filled - * - -1 if invalid session, IMB_JOB will not be filled + * - -1 if invalid session or errors allocationg SGL linear buffer, + * IMB_JOB will not be filled */ static inline int set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, @@ -1191,6 +1267,7 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, uint32_t total_len; IMB_JOB base_job; uint8_t sgl = 0; + uint8_t lb_sgl = 0; int ret; session = ipsec_mb_get_session_private(qp, op); @@ -1199,18 +1276,6 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, return -1; } - if (op->sym->m_src->nb_segs > 1) { - if (session->cipher.mode != IMB_CIPHER_GCM - && session->cipher.mode != - IMB_CIPHER_CHACHA20_POLY1305) { - op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; - IPSEC_MB_LOG(ERR, "Device only supports SGL for AES-GCM" - " or CHACHA20_POLY1305 algorithms."); - return -1; - } - sgl = 1; - } - /* Set crypto operation */ job->chain_order = session->chain_order; @@ -1233,6 +1298,26 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, job->dec_keys = session->cipher.expanded_aes_keys.decode; } + if (!op->sym->m_dst) { + /* in-place operation */ + m_dst = m_src; + oop = 0; + } else if (op->sym->m_dst == op->sym->m_src) { + /* in-place operation */ + m_dst = m_src; + oop = 0; + } else { + /* out-of-place operation */ + m_dst = op->sym->m_dst; + oop = 1; + } + + if (m_src->nb_segs > 1 || m_dst->nb_segs > 1) { + sgl = 1; + if (!imb_lib_support_sgl_algo(session->cipher.mode)) + lb_sgl = 1; + } + switch (job->hash_alg) { case IMB_AUTH_AES_XCBC: job->u.XCBC._k1_expanded = session->auth.xcbc.k1_expanded; @@ -1331,20 +1416,6 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, m_offset = 0; } - if (!op->sym->m_dst) { - /* in-place operation */ - m_dst = m_src; - oop = 0; - } else if (op->sym->m_dst == op->sym->m_src) { - /* in-place operation */ - m_dst = m_src; - oop = 0; - } else { - /* out-of-place operation */ - m_dst = op->sym->m_dst; - oop = 1; - } - /* Set digest output location */ if (job->hash_alg != IMB_AUTH_NULL && session->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY) { @@ -1435,7 +1506,7 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, job->hash_start_src_offset_in_bytes = auth_start_offset(op, session, oop, auth_off_in_bytes, ciph_off_in_bytes, auth_len_in_bytes, - ciph_len_in_bytes); + ciph_len_in_bytes, lb_sgl); job->msg_len_to_hash_in_bits = op->sym->auth.data.length; job->iv = rte_crypto_op_ctod_offset(op, uint8_t *, @@ -1452,7 +1523,7 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, job->hash_start_src_offset_in_bytes = auth_start_offset(op, session, oop, auth_off_in_bytes, ciph_off_in_bytes, auth_len_in_bytes, - ciph_len_in_bytes); + ciph_len_in_bytes, lb_sgl); job->msg_len_to_hash_in_bytes = auth_len_in_bytes; job->iv = rte_crypto_op_ctod_offset(op, uint8_t *, @@ -1464,7 +1535,7 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, session, oop, op->sym->auth.data.offset, op->sym->cipher.data.offset, op->sym->auth.data.length, - op->sym->cipher.data.length); + op->sym->cipher.data.length, lb_sgl); job->msg_len_to_hash_in_bytes = op->sym->auth.data.length; job->iv = rte_crypto_op_ctod_offset(op, uint8_t *, @@ -1525,6 +1596,10 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, job->user_data = op; if (sgl) { + + if (lb_sgl) + return handle_sgl_linear(job, op, m_offset, session); + base_job = *job; job->sgl_state = IMB_SGL_INIT; job = IMB_SUBMIT_JOB(mb_mgr); @@ -1694,6 +1769,31 @@ generate_digest(IMB_JOB *job, struct rte_crypto_op *op, sess->auth.req_digest_len); } +static void +post_process_sgl_linear(struct rte_crypto_op *op, IMB_JOB *job, + struct aesni_mb_session *sess, uint8_t *linear_buf) +{ + + int lb_offset = 0; + struct rte_mbuf *m_dst = op->sym->m_dst == NULL ? + op->sym->m_src : op->sym->m_dst; + uint16_t total_len, dst_len; + uint64_t auth_len; + uint8_t *dst; + + total_len = sgl_linear_cipher_auth_len(job, &auth_len); + + if (sess->auth.operation != RTE_CRYPTO_AUTH_OP_VERIFY) + total_len += job->auth_tag_output_len_in_bytes; + + for (; (m_dst != NULL) && (total_len - lb_offset > 0); m_dst = m_dst->next) { + dst = rte_pktmbuf_mtod(m_dst, uint8_t *); + dst_len = RTE_MIN(m_dst->data_len, total_len - lb_offset); + rte_memcpy(dst, linear_buf + lb_offset, dst_len); + lb_offset += dst_len; + } +} + /** * Process a completed job and return rte_mbuf which job processed * @@ -1709,6 +1809,7 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) { struct rte_crypto_op *op = (struct rte_crypto_op *)job->user_data; struct aesni_mb_session *sess = NULL; + uint8_t *linear_buf = NULL; #ifdef AESNI_MB_DOCSIS_SEC_ENABLED uint8_t is_docsis_sec = 0; @@ -1729,6 +1830,14 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) case IMB_STATUS_COMPLETED: op->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + if ((op->sym->m_src->nb_segs > 1 || + (op->sym->m_dst != NULL && + op->sym->m_dst->nb_segs > 1)) && + !imb_lib_support_sgl_algo(sess->cipher.mode)) { + linear_buf = (uint8_t *) job->user_data2; + post_process_sgl_linear(op, job, sess, linear_buf); + } + if (job->hash_alg == IMB_AUTH_NULL) break; @@ -1755,6 +1864,7 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) default: op->status = RTE_CRYPTO_OP_STATUS_ERROR; } + rte_free(linear_buf); } /* Free session if a session-less crypto op */ @@ -2213,7 +2323,11 @@ RTE_INIT(ipsec_mb_register_aesni_mb) RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT | RTE_CRYPTODEV_FF_SYM_CPU_CRYPTO | RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA | - RTE_CRYPTODEV_FF_SYM_SESSIONLESS; + RTE_CRYPTODEV_FF_SYM_SESSIONLESS | + RTE_CRYPTODEV_FF_IN_PLACE_SGL | + RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT | + RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT | + RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT; aesni_mb_data->internals_priv_size = 0; aesni_mb_data->ops = &aesni_mb_pmd_ops; From patchwork Fri Oct 7 13:46:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 117562 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 98C96A00C3; Fri, 7 Oct 2022 15:47:24 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CFB2E42B94; Fri, 7 Oct 2022 15:47:11 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 3747742B87 for ; Fri, 7 Oct 2022 15:47:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665150430; x=1696686430; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1WbWQuWaOoytZmRacVfIyjqesTcPWB0XM127f5tG4SM=; b=EjXuJaCOjdZkGUgToo89xqES+4xylafNVDngrGPPFFwjIE2Fe/c+UYWj D+T/xWZFNfKLAhWzhaMETA3IPN7502lQUWiT9DsB1JlihTRgyF24p8tVi 0cF0gZGpWVO7qCzqio6MrFIQInEJYqEJ+u7IGzUGXePraaVKlNrdoERwL +Ccs/KO2Fnr9yukcjK9AdAQExL9837v0IrJs8ITl2ycC/QwtN7aTgHkvh so/knxKF7B7mFsJC9jV3mPEgwQFjhvq91ViTcj3b1P6PKDQx2f0uH7RkF ypSPlAP25FuaxcGpkVaL4V/v7P9QYKrLTtzxiQUl2mGgd5An4krsFUGUB g==; X-IronPort-AV: E=McAfee;i="6500,9779,10493"; a="390032324" X-IronPort-AV: E=Sophos;i="5.95,166,1661842800"; d="scan'208";a="390032324" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Oct 2022 06:47:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10493"; a="625150233" X-IronPort-AV: E=Sophos;i="5.95,166,1661842800"; d="scan'208";a="625150233" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.222.163]) by orsmga002.jf.intel.com with ESMTP; 07 Oct 2022 06:47:07 -0700 From: Ciara Power To: Akhil Goyal , Fan Zhang Cc: dev@dpdk.org, kai.ji@intel.com, Ciara Power , Fan Zhang , Pablo de Lara Subject: [PATCH v5 3/4] test/crypto: add OOP snow3g SGL tests Date: Fri, 7 Oct 2022 13:46:52 +0000 Message-Id: <20221007134653.929034-4-ciara.power@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221007134653.929034-1-ciara.power@intel.com> References: <20220812132334.75707-1-ciara.power@intel.com> <20221007134653.929034-1-ciara.power@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org More tests are added to test variations of OOP SGL for snow3g. This includes LB_IN_SGL_OUT and SGL_IN_LB_OUT. Signed-off-by: Ciara Power Acked-by: Fan Zhang Acked-by: Pablo de Lara --- app/test/test_cryptodev.c | 48 +++++++++++++++++++++++++++++++-------- 1 file changed, 39 insertions(+), 9 deletions(-) diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c index 203b8b61fa..c2b33686ed 100644 --- a/app/test/test_cryptodev.c +++ b/app/test/test_cryptodev.c @@ -4299,7 +4299,8 @@ test_snow3g_encryption_oop(const struct snow3g_test_data *tdata) } static int -test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata) +test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata, + uint8_t sgl_in, uint8_t sgl_out) { struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; @@ -4330,9 +4331,12 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata) uint64_t feat_flags = dev_info.feature_flags; - if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) { - printf("Device doesn't support out-of-place scatter-gather " - "in both input and output mbufs. " + if (((sgl_in && sgl_out) && !(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) + || ((!sgl_in && sgl_out) && + !(feat_flags & RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT)) + || ((sgl_in && !sgl_out) && + !(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))) { + printf("Device doesn't support out-of-place scatter gather type. " "Test Skipped.\n"); return TEST_SKIPPED; } @@ -4357,10 +4361,21 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata) /* the algorithms block size */ plaintext_pad_len = RTE_ALIGN_CEIL(plaintext_len, 16); - ut_params->ibuf = create_segmented_mbuf(ts_params->mbuf_pool, - plaintext_pad_len, 10, 0); - ut_params->obuf = create_segmented_mbuf(ts_params->mbuf_pool, - plaintext_pad_len, 3, 0); + if (sgl_in) + ut_params->ibuf = create_segmented_mbuf(ts_params->mbuf_pool, + plaintext_pad_len, 10, 0); + else { + ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + rte_pktmbuf_append(ut_params->ibuf, plaintext_pad_len); + } + + if (sgl_out) + ut_params->obuf = create_segmented_mbuf(ts_params->mbuf_pool, + plaintext_pad_len, 3, 0); + else { + ut_params->obuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + rte_pktmbuf_append(ut_params->obuf, plaintext_pad_len); + } TEST_ASSERT_NOT_NULL(ut_params->ibuf, "Failed to allocate input buffer in mempool"); @@ -6714,9 +6729,20 @@ test_snow3g_encryption_test_case_1_oop(void) static int test_snow3g_encryption_test_case_1_oop_sgl(void) { - return test_snow3g_encryption_oop_sgl(&snow3g_test_case_1); + return test_snow3g_encryption_oop_sgl(&snow3g_test_case_1, 1, 1); +} + +static int +test_snow3g_encryption_test_case_1_oop_lb_in_sgl_out(void) +{ + return test_snow3g_encryption_oop_sgl(&snow3g_test_case_1, 0, 1); } +static int +test_snow3g_encryption_test_case_1_oop_sgl_in_lb_out(void) +{ + return test_snow3g_encryption_oop_sgl(&snow3g_test_case_1, 1, 0); +} static int test_snow3g_encryption_test_case_1_offset_oop(void) @@ -15842,6 +15868,10 @@ static struct unit_test_suite cryptodev_snow3g_testsuite = { test_snow3g_encryption_test_case_1_oop), TEST_CASE_ST(ut_setup, ut_teardown, test_snow3g_encryption_test_case_1_oop_sgl), + TEST_CASE_ST(ut_setup, ut_teardown, + test_snow3g_encryption_test_case_1_oop_lb_in_sgl_out), + TEST_CASE_ST(ut_setup, ut_teardown, + test_snow3g_encryption_test_case_1_oop_sgl_in_lb_out), TEST_CASE_ST(ut_setup, ut_teardown, test_snow3g_encryption_test_case_1_offset_oop), TEST_CASE_ST(ut_setup, ut_teardown, From patchwork Fri Oct 7 13:46:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 117563 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2ED4CA00C3; Fri, 7 Oct 2022 15:47:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C4B4B42B8D; Fri, 7 Oct 2022 15:47:15 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 9CB1042B88 for ; Fri, 7 Oct 2022 15:47:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665150433; x=1696686433; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BCDbJYBMAY/y3cYGhycviRyvENe2lQ2WQFoenqn8BkQ=; b=QHdA3qEOjj/dPg8UGbWvq+ZXJh80/Q1BnCFX4S8d0U2yKP+jCiD0uvXK VjiOOsNlILSdbokgbKyBIcV3dGHSB7Jkq3MCvCWXbtZUXan8/zdMkcGRa Uv6+tRQJ3BaTSU8T60mV4Wc/otZfAzGG2q5rweBHJarPopqJQ/UfO2TAA 43Evy+YTYfWa3r8ITN95JOB1kjYGgrNaovbs+6FHXw/Nj8YMWvpB1hU1n QLFzT5VEbC/y39c/BrBT9AXxSYOunhQ41iBYfy+xXWotje0OQWKuR1hgM djq8iP7wG7YyYOYAXfs0gD7RkJflAaybvd2qJ5zpndlqk0/PN0e45+WQZ A==; X-IronPort-AV: E=McAfee;i="6500,9779,10493"; a="390032332" X-IronPort-AV: E=Sophos;i="5.95,166,1661842800"; d="scan'208";a="390032332" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Oct 2022 06:47:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10493"; a="625150240" X-IronPort-AV: E=Sophos;i="5.95,166,1661842800"; d="scan'208";a="625150240" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.222.163]) by orsmga002.jf.intel.com with ESMTP; 07 Oct 2022 06:47:09 -0700 From: Ciara Power To: Akhil Goyal , Fan Zhang , Yipeng Wang , Sameh Gobriel , Bruce Richardson , Vladimir Medvedkin Cc: dev@dpdk.org, kai.ji@intel.com, Ciara Power , Fan Zhang , Pablo de Lara Subject: [PATCH v5 4/4] test/crypto: add remaining blockcipher SGL tests Date: Fri, 7 Oct 2022 13:46:53 +0000 Message-Id: <20221007134653.929034-5-ciara.power@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221007134653.929034-1-ciara.power@intel.com> References: <20220812132334.75707-1-ciara.power@intel.com> <20221007134653.929034-1-ciara.power@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The current blockcipher test function only has support for two types of SGL test, INPLACE or OOP_SGL_IN_LB_OUT. These types are hardcoded into the function, with the number of segments always set to 3. To ensure all SGL types are tested, blockcipher test vectors now have fields to specify SGL type, and the number of segments. If these fields are missing, the previous defaults are used, either INPLACE or OOP_SGL_IN_LB_OUT, with 3 segments. Some AES and Hash vectors are modified to use these new fields, and new AES tests are added to test the SGL types that were not previously being tested. Signed-off-by: Ciara Power Acked-by: Fan Zhang Acked-by: Pablo de Lara --- v5: - Fixed AES-XTS tests to add in some that were accidentally removed in previous versions. - Removed some AES-XTS tests added in previous version that are skipped for AESNI_MB and QAT, as they haven't been verified locally. --- app/test/test_cryptodev_aes_test_vectors.h | 310 +++++++++++++++++++- app/test/test_cryptodev_blockcipher.c | 50 ++-- app/test/test_cryptodev_blockcipher.h | 2 + app/test/test_cryptodev_hash_test_vectors.h | 8 +- 4 files changed, 338 insertions(+), 32 deletions(-) diff --git a/app/test/test_cryptodev_aes_test_vectors.h b/app/test/test_cryptodev_aes_test_vectors.h index a797af1b00..ea7b21ce53 100644 --- a/app/test/test_cryptodev_aes_test_vectors.h +++ b/app/test/test_cryptodev_aes_test_vectors.h @@ -4163,12 +4163,44 @@ static const struct blockcipher_test_case aes_chain_test_cases[] = { }, { .test_descr = "AES-192-CTR XCBC Decryption Digest Verify " - "Scatter Gather", + "Scatter Gather (Inplace)", + .test_data = &aes_test_data_2, + .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 + }, + { + .test_descr = "AES-192-CTR XCBC Decryption Digest Verify " + "Scatter Gather OOP (SGL in SGL out)", + .test_data = &aes_test_data_2, + .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-192-CTR XCBC Decryption Digest Verify " + "Scatter Gather OOP (LB in SGL out)", .test_data = &aes_test_data_2, .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT, + .sgl_segs = 3 }, + { + .test_descr = "AES-192-CTR XCBC Decryption Digest Verify " + "Scatter Gather OOP (SGL in LB out)", + .test_data = &aes_test_data_2, + .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 + }, + { .test_descr = "AES-256-CTR HMAC-SHA1 Encryption Digest", .test_data = &aes_test_data_3, @@ -4193,11 +4225,52 @@ static const struct blockcipher_test_case aes_chain_test_cases[] = { }, { .test_descr = "AES-128-CBC HMAC-SHA1 Encryption Digest " - "Scatter Gather", + "Scatter Gather (Inplace)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA1 Encryption Digest " + "Scatter Gather OOP (SGL in SGL out)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA1 Encryption Digest " + "Scatter Gather OOP 16 segs (SGL in SGL out)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT, + .sgl_segs = 16 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA1 Encryption Digest " + "Scatter Gather OOP (LB in SGL out)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA1 Encryption Digest " + "Scatter Gather OOP (SGL in LB out)", .test_data = &aes_test_data_4, .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest " @@ -4207,10 +4280,52 @@ static const struct blockcipher_test_case aes_chain_test_cases[] = { }, { .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest " - "Verify Scatter Gather", + "Verify Scatter Gather (Inplace)", .test_data = &aes_test_data_4, .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest " + "Verify Scatter Gather OOP (SGL in SGL out)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest " + "Verify Scatter Gather OOP 16 segs (SGL in SGL out)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT, + .sgl_segs = 16 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest " + "Verify Scatter Gather OOP (LB in SGL out)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest " + "Verify Scatter Gather OOP (SGL in LB out)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest " @@ -4255,12 +4370,46 @@ static const struct blockcipher_test_case aes_chain_test_cases[] = { }, { .test_descr = "AES-128-CBC HMAC-SHA512 Encryption Digest " - "Scatter Gather Sessionless", + "Scatter Gather Sessionless (Inplace)", + .test_data = &aes_test_data_6, + .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SESSIONLESS | + BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA512 Encryption Digest " + "Scatter Gather Sessionless OOP (SGL in SGL out)", + .test_data = &aes_test_data_6, + .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SESSIONLESS | + BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA512 Encryption Digest " + "Scatter Gather Sessionless OOP (LB in SGL out)", .test_data = &aes_test_data_6, .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SESSIONLESS | BLOCKCIPHER_TEST_FEATURE_SG | BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA512 Encryption Digest " + "Scatter Gather Sessionless OOP (SGL in LB out)", + .test_data = &aes_test_data_6, + .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SESSIONLESS | + BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-128-CBC HMAC-SHA512 Decryption Digest " @@ -4270,11 +4419,42 @@ static const struct blockcipher_test_case aes_chain_test_cases[] = { }, { .test_descr = "AES-128-CBC HMAC-SHA512 Decryption Digest " - "Verify Scatter Gather", + "Verify Scatter Gather (Inplace)", + .test_data = &aes_test_data_6, + .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 2 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA512 Decryption Digest " + "Verify Scatter Gather OOP (SGL in SGL out)", + .test_data = &aes_test_data_6, + .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA512 Decryption Digest " + "Verify Scatter Gather OOP (LB in SGL out)", + .test_data = &aes_test_data_6, + .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC HMAC-SHA512 Decryption Digest " + "Verify Scatter Gather OOP (SGL in LB out)", .test_data = &aes_test_data_6, .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-128-CBC XCBC Encryption Digest", @@ -4358,6 +4538,8 @@ static const struct blockcipher_test_case aes_chain_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_AUTH_GEN_ENC, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | BLOCKCIPHER_TEST_FEATURE_DIGEST_ENCRYPTED, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 }, { .test_descr = "AES-128-CBC HMAC-SHA1 Encryption Digest " @@ -4382,6 +4564,8 @@ static const struct blockcipher_test_case aes_chain_test_cases[] = { .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | BLOCKCIPHER_TEST_FEATURE_SESSIONLESS | BLOCKCIPHER_TEST_FEATURE_DIGEST_ENCRYPTED, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 }, { .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest " @@ -4397,6 +4581,8 @@ static const struct blockcipher_test_case aes_chain_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_DEC_AUTH_VERIFY, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | BLOCKCIPHER_TEST_FEATURE_DIGEST_ENCRYPTED, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 }, { .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest " @@ -4421,6 +4607,8 @@ static const struct blockcipher_test_case aes_chain_test_cases[] = { .feature_mask = BLOCKCIPHER_TEST_FEATURE_SESSIONLESS | BLOCKCIPHER_TEST_FEATURE_SG | BLOCKCIPHER_TEST_FEATURE_DIGEST_ENCRYPTED, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 }, { .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest " @@ -4504,6 +4692,41 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .test_data = &aes_test_data_4, .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, }, + { + .test_descr = "AES-128-CBC Encryption Scatter gather (Inplace)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC Encryption Scatter gather OOP (SGL in SGL out)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC Encryption Scatter gather OOP (LB in SGL out)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-128-CBC Encryption Scatter gather OOP (SGL in LB out)", + .test_data = &aes_test_data_4, + .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 + }, { .test_descr = "AES-128-CBC Decryption", .test_data = &aes_test_data_4, @@ -4515,11 +4738,39 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, }, { - .test_descr = "AES-192-CBC Encryption Scatter gather", + .test_descr = "AES-192-CBC Encryption Scatter gather (Inplace)", + .test_data = &aes_test_data_10, + .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 + }, + { + .test_descr = "AES-192-CBC Encryption Scatter gather OOP (SGL in SGL out)", + .test_data = &aes_test_data_10, + .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-192-CBC Encryption Scatter gather OOP (LB in SGL out)", + .test_data = &aes_test_data_10, + .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-192-CBC Encryption Scatter gather OOP (SGL in LB out)", .test_data = &aes_test_data_10, .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-192-CBC Decryption", @@ -4527,10 +4778,39 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_DECRYPT, }, { - .test_descr = "AES-192-CBC Decryption Scatter Gather", + .test_descr = "AES-192-CBC Decryption Scatter Gather (Inplace)", .test_data = &aes_test_data_10, .op_mask = BLOCKCIPHER_TEST_OP_DECRYPT, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 + }, + { + .test_descr = "AES-192-CBC Decryption Scatter Gather OOP (SGL in SGL out)", + .test_data = &aes_test_data_10, + .op_mask = BLOCKCIPHER_TEST_OP_DECRYPT, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-192-CBC Decryption Scatter Gather OOP (LB in SGL out)", + .test_data = &aes_test_data_10, + .op_mask = BLOCKCIPHER_TEST_OP_DECRYPT, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT, + .sgl_segs = 3 + }, + { + .test_descr = "AES-192-CBC Decryption Scatter Gather OOP (SGL in LB out)", + .test_data = &aes_test_data_10, + .op_mask = BLOCKCIPHER_TEST_OP_DECRYPT, + .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG | + BLOCKCIPHER_TEST_FEATURE_OOP, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-256-CBC Encryption", @@ -4694,6 +4974,8 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, .feature_mask = BLOCKCIPHER_TEST_FEATURE_OOP | BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-256-XTS Decryption (512-byte plaintext" @@ -4702,6 +4984,8 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_DECRYPT, .feature_mask = BLOCKCIPHER_TEST_FEATURE_OOP | BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-256-XTS Encryption (512-byte plaintext" @@ -4710,6 +4994,8 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, .feature_mask = BLOCKCIPHER_TEST_FEATURE_OOP | BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-256-XTS Decryption (512-byte plaintext" @@ -4718,6 +5004,8 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_DECRYPT, .feature_mask = BLOCKCIPHER_TEST_FEATURE_OOP | BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-256-XTS Encryption (4096-byte plaintext" @@ -4726,6 +5014,8 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, .feature_mask = BLOCKCIPHER_TEST_FEATURE_OOP | BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-256-XTS Decryption (4096-byte plaintext" @@ -4734,6 +5024,8 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_DECRYPT, .feature_mask = BLOCKCIPHER_TEST_FEATURE_OOP | BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-256-XTS Encryption (4096-byte plaintext" @@ -4742,6 +5034,8 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT, .feature_mask = BLOCKCIPHER_TEST_FEATURE_OOP | BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "AES-256-XTS Decryption (4096-byte plaintext" @@ -4750,6 +5044,8 @@ static const struct blockcipher_test_case aes_cipheronly_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_DECRYPT, .feature_mask = BLOCKCIPHER_TEST_FEATURE_OOP | BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT, + .sgl_segs = 3 }, { .test_descr = "cipher-only - NULL algo - x8 - encryption", diff --git a/app/test/test_cryptodev_blockcipher.c b/app/test/test_cryptodev_blockcipher.c index 28479812ee..6c9a5964ea 100644 --- a/app/test/test_cryptodev_blockcipher.c +++ b/app/test/test_cryptodev_blockcipher.c @@ -95,7 +95,9 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, uint8_t tmp_dst_buf[MBUF_SIZE]; uint32_t pad_len; - int nb_segs = 1; + int nb_segs_in = 1; + int nb_segs_out = 1; + uint64_t sgl_type = t->sgl_flag; uint32_t nb_iterates = 0; rte_cryptodev_info_get(dev_id, &dev_info); @@ -120,30 +122,31 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, } } if (t->feature_mask & BLOCKCIPHER_TEST_FEATURE_SG) { - uint64_t oop_flag = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT; + if (sgl_type == 0) { + if (t->feature_mask & BLOCKCIPHER_TEST_FEATURE_OOP) + sgl_type = RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT; + else + sgl_type = RTE_CRYPTODEV_FF_IN_PLACE_SGL; + } - if (t->feature_mask & BLOCKCIPHER_TEST_FEATURE_OOP) { - if (!(feat_flags & oop_flag)) { - printf("Device doesn't support out-of-place " - "scatter-gather in input mbuf. " - "Test Skipped.\n"); - snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, - "SKIPPED"); - return TEST_SKIPPED; - } - } else { - if (!(feat_flags & RTE_CRYPTODEV_FF_IN_PLACE_SGL)) { - printf("Device doesn't support in-place " - "scatter-gather mbufs. " - "Test Skipped.\n"); - snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, - "SKIPPED"); - return TEST_SKIPPED; - } + if (!(feat_flags & sgl_type)) { + printf("Device doesn't support scatter-gather type." + " Test Skipped.\n"); + snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, + "SKIPPED"); + return TEST_SKIPPED; } - nb_segs = 3; + if (sgl_type == RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT || + sgl_type == RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT || + sgl_type == RTE_CRYPTODEV_FF_IN_PLACE_SGL) + nb_segs_in = t->sgl_segs == 0 ? 3 : t->sgl_segs; + + if (sgl_type == RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT || + sgl_type == RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT) + nb_segs_out = t->sgl_segs == 0 ? 3 : t->sgl_segs; } + if (!!(feat_flags & RTE_CRYPTODEV_FF_CIPHER_WRAPPED_KEY) ^ tdata->wrapped_key) { snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, @@ -206,7 +209,7 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, /* for contiguous mbuf, nb_segs is 1 */ ibuf = create_segmented_mbuf(mbuf_pool, - tdata->ciphertext.len, nb_segs, src_pattern); + tdata->ciphertext.len, nb_segs_in, src_pattern); if (ibuf == NULL) { snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, "line %u FAILED: %s", @@ -255,7 +258,8 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, } if (t->feature_mask & BLOCKCIPHER_TEST_FEATURE_OOP) { - obuf = rte_pktmbuf_alloc(mbuf_pool); + obuf = create_segmented_mbuf(mbuf_pool, + tdata->ciphertext.len, nb_segs_out, dst_pattern); if (!obuf) { snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, "line %u " "FAILED: %s", __LINE__, diff --git a/app/test/test_cryptodev_blockcipher.h b/app/test/test_cryptodev_blockcipher.h index 84f5d57787..bad93a5ec1 100644 --- a/app/test/test_cryptodev_blockcipher.h +++ b/app/test/test_cryptodev_blockcipher.h @@ -57,6 +57,8 @@ struct blockcipher_test_case { const struct blockcipher_test_data *test_data; uint8_t op_mask; /* operation mask */ uint8_t feature_mask; + uint64_t sgl_flag; + uint8_t sgl_segs; }; struct blockcipher_test_data { diff --git a/app/test/test_cryptodev_hash_test_vectors.h b/app/test/test_cryptodev_hash_test_vectors.h index 5bd7858de4..62602310b2 100644 --- a/app/test/test_cryptodev_hash_test_vectors.h +++ b/app/test/test_cryptodev_hash_test_vectors.h @@ -467,10 +467,12 @@ static const struct blockcipher_test_case hash_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_AUTH_GEN, }, { - .test_descr = "HMAC-SHA1 Digest Scatter Gather", + .test_descr = "HMAC-SHA1 Digest Scatter Gather (Inplace)", .test_data = &hmac_sha1_test_vector, .op_mask = BLOCKCIPHER_TEST_OP_AUTH_GEN, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 }, { .test_descr = "HMAC-SHA1 Digest Verify", @@ -478,10 +480,12 @@ static const struct blockcipher_test_case hash_test_cases[] = { .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY, }, { - .test_descr = "HMAC-SHA1 Digest Verify Scatter Gather", + .test_descr = "HMAC-SHA1 Digest Verify Scatter Gather (Inplace)", .test_data = &hmac_sha1_test_vector, .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY, .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG, + .sgl_flag = RTE_CRYPTODEV_FF_IN_PLACE_SGL, + .sgl_segs = 3 }, { .test_descr = "SHA224 Digest",