From patchwork Thu Sep 13 06:08:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gagandeep Singh X-Patchwork-Id: 44644 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9E2B35F1A; Thu, 13 Sep 2018 08:09:47 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20076.outbound.protection.outlook.com [40.107.2.76]) by dpdk.org (Postfix) with ESMTP id 2474B58CB for ; Thu, 13 Sep 2018 08:09:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2dUBZ8AVMbIbhd+ZqNVstCyjeutCh0vkpYY/XA1e7OA=; b=QPI0M924FPdaH7VAIZ1EoR0bzopihukD1a9rsdMad77Etdca0xNFukRqiVe2CW/kml9/VtIBIENYdZU/1L0fgRL+8r55HzdAoMhlBoHJ7mR/gCnWN3LhvdNQc5XS/IQfuYvffG8QmZqsX/Iz0gaGutJFzN4Cbh+iXBAyPufbJeE= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=G.Singh@nxp.com; Received: from Tophie.ap.freescale.net (14.142.187.166) by HE1PR04MB1529.eurprd04.prod.outlook.com (2a01:111:e400:59a8::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1143.15; Thu, 13 Sep 2018 06:09:34 +0000 From: Gagandeep Singh To: dev@dpdk.org, akhil.goyal@nxp.com Cc: Hemant Agrawal , Gagandeep Singh Date: Thu, 13 Sep 2018 11:38:44 +0530 Message-Id: <20180913060846.29930-9-g.singh@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180913060846.29930-1-g.singh@nxp.com> References: <20180913060846.29930-1-g.singh@nxp.com> MIME-Version: 1.0 X-Originating-IP: [14.142.187.166] X-ClientProxiedBy: BM1PR01CA0095.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00::11) To HE1PR04MB1529.eurprd04.prod.outlook.com (2a01:111:e400:59a8::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3888705f-f368-43be-20ec-08d6193f7b1b X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:HE1PR04MB1529; X-Microsoft-Exchange-Diagnostics: 1; HE1PR04MB1529; 3:puaDeqrzj/Au1S/o2li1oNSg4dl4Sf2QflkVHN5m4OqF4Vt+nZlE792fi7ZV7sjTY6Z9iRAYLv94gh9o72oSwhVaU0K2ePGDDYJAQl4V3TIpnhVd07kv6kcwHoGt2xc4+LV2XvnFdQM/2nnrFCaDg6t/ajWEk0NAjakKF0DZz7zx96DGY9zKQ8vjgQtyLqyJCKqNRoyALM+2MnqjuBZdnsiaiwwJuNbB6Rq5CRqBlZL3GJMWXYdbm/u0E82z+bUc; 25:UpPI2qU5bibnduekbY/tIuXATczDSyqBfTDuBEZQYmeC6lYS7FlHYN9ip9ougf9fe6gtTaG9dygclyDI35FgOuF+kUuR4y/qe2Ay2pnw9oeobXvg6IxxipvmHbYcWGisyCk/vEJudeK/9OwzccqztvyVjTo+XDdCIBWoIbkVwiJRlrEvsP0uGf4A4yf1q0e7hvQyp0v/vSM2fatG792iRyiR0np/I/mWcXEX5hYE8Ah2CmHsNpl6+gkXoHDJ+7am5tLOre49hUJlx1o5oVWSkJEiO2x0gydBlWrQIlQo7yetiox3mwoIG49BYI9Ua36ziOcdS/MLslexnCvf8hCYig==; 31:I2orOn0f+AzQiK19udMJ3h00caG1ZFgYU9iNFV5LfNUUHWmjT33ZzwHFwRasHDPzAt+NSmkyS24KpRiNG9vkx3SFO2+Ao/TPrRVswnVWhXiMZUx2OhNl9UboSLHhBIeDTar/hmK5Z/b2vN0WXFfdNiI0aLXVO7w0tJQWw3qm8rOKS5KdpBOytHPLe9xy01pGXvsgGsnA0Md+EW4fIyBypeK92sEX0FdtxguKSFoIDPw= X-MS-TrafficTypeDiagnostic: HE1PR04MB1529: X-Microsoft-Exchange-Diagnostics: 1; HE1PR04MB1529; 20:oU79fACEjjyEyLKwG7dnl0hQlNp2Yq81SzTXBpgmdAYsxYX9gWJdY5qcUKnxulprpQASq03rNG8SzeJKMeLBsiTgKabVFE2v7VzL4Qp7TVkP5T83QE0wqwyCWbBeGQ68qprpqtZ5eij1x0nGOVwG36sdhv7JJo9xf/ojgUoeIlI9ZMEQYFeXKCMnLtUABxZORAtZl7Iy7w+pgYJaoXzUiQR+cFY9WdMzwZaU4sOiCoGvKse2cQkeoNsTIAv8hnEVW0mMq+DXf/wnY/TLuFaG8kGBQ9U4i4hTIazTvxLUzOFz63q4vsSrtzGm7p/NmBMEo+9GGyVqXf2NizHcCtz4r97c/tj6zqK66QC8KJl7oqAyz7kfONHQjVWo85Zv3WeNh0hJ4nR2T530GqFWdSZCMTwMySwaQJ/wBzKo7V1Fr19FZ3sbRQr1c7chRGMYAavK/APop+AW6Pzt5BoxxuN/ZPSqSb11opJWuDK5KT5HxUE6+anzfwY5S163hgJUPVA8; 4:/rfAlu9+5JyG2zBzLCna+GdgKAHSDrA5zykN01mZdXsbx8bEAuylZSg9kK0flrHzfeEXVhTQluoUXkq+Y80rVYQ2rW+uj6tDMASl9FHaYSdLjvbyjnNKJi/LdEWSXjXE8CwMLpEF9/W8i/xOfp1bXuI19LecVV71FPBrlaXqGbjwVYRY+FQr8QNhR1hgDF4GHrF7fUaUI0Q48soHIdZmr5V4ZNAzAuMwpXpC5BpxrW+YV9UodyAAIUiyjaN8qzoLQjJ+ecadY27605yjmTi88hs4RLCXOOyQBN1Wucd3bJPNBy8WNWIXd84NooecGll0 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(823301075)(3231311)(944501410)(52105095)(3002001)(93006095)(93001095)(10201501046)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123562045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(201708071742011)(7699050); SRVR:HE1PR04MB1529; BCL:0; PCL:0; RULEID:; SRVR:HE1PR04MB1529; X-Forefront-PRVS: 07943272E1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(366004)(376002)(39860400002)(136003)(346002)(396003)(189003)(199004)(2906002)(6506007)(386003)(5009440100003)(6486002)(51416003)(6512007)(52116002)(486006)(2616005)(68736007)(956004)(53936002)(476003)(446003)(11346002)(305945005)(26005)(7736002)(55236004)(76176011)(186003)(16526019)(50226002)(478600001)(47776003)(1076002)(3846002)(6116002)(48376002)(105586002)(106356001)(25786009)(14444005)(50466002)(5660300001)(36756003)(72206003)(66066001)(316002)(97736004)(8936002)(81166006)(4326008)(8676002)(86362001)(6636002)(6666003)(54906003)(81156014)(16586007)(110426005); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR04MB1529; H:Tophie.ap.freescale.net; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR04MB1529; 23:fVhscXPDbv5O+Mv3yUl0OHWJs4LaxWNUHlrNVYB2Y?= TgPXo8Cw1kWqxzpGf76lnKWgkpMEGcFdMhtJSOx82PemahktFEG9A34JprDnhQByip9cUMIfshev2rMOkn5qNWopHAxjT6q/bjAAgSZAjPTVRYPaB12kMWjwa8Qj1jwOXt3M46nADRdeZBV1D64iogTf5bl/jWYWIV+ZDZqr5UcVh1y7G43JB0Hk5+RmlJLAhtkbB3lAJlPxfwQx7yAcrbt1EW066DP6imZr6e9p+DY1Vn7UY0RPZtc+S0/o5uXqrDNWh2MaqaZjbEvJqe342JTIbvh3QCThI2UcnCGVOLaxb0C/JRCCHW9qGYMW1XWTCR+7sjvbgyRbaokv19hFDxpO5mHsOHSoJsqlz3fNOl2EuOcIpzQfPd/czpNiBXPPzjbgEgcPh3n8jgabr745eCerlbzAaXUufpTp1b3iK4YNk8Z3SZCcd/NS2XPFT0DzYFuoBnwoTyqocdXBKDmrgc0Jsjunl5oK54MM6dKQeTnJfdBcsbEp5+to/oUpGOwo08Gwi0dnmYNu+s5mYP/PsCSl94TLz1GEF6dE4tArqIWQtuUoJRpCLFWqaJZrUwy+biFADz0lW/Qypa9qCx6n6HkQp1yEqBeUz88FIH6DO+6NyRG+Cw66UG957/k8CxSgu0Zoz8rhCkZaXScch0kFYDJzzZ0LG5xw5GaMN+nv+BsfY852+sDplxHhTq/ZrOMSUzYHlPVxDsHTCYrV22EbWlJt9DlVxjcLSmziyibm6YCyS9d8kvO3K4YFY4yL7ShOKemuda+Cdbfe0doF2CsJ5FUgO8b1b285GBDvHcRZkFMTqpOvKm1f2dG0CDGNDXgwj30YlBsTDGYCAbQMXDU3YKgROHlnkr17MfbGnrRpvcJ2LseLsq3CfAGO1UjxJfLWXpxyUgEhfkt39MiKwTsxbID/bLVY7dAczkCCZMVYrmeuIwI8vd6Pj7gsMKrjWD267DJgOX4aly61sAF4lAte4m1p1CXfqNRFQqKC8Cqr1fVGovTYHXlaZp+j7f7J6v5iG9XOr96k7Nyf49B+jfBOGxbd6AfjLmFaMkMIR3Dmt9Ai0Sjr6fUEulgV4h7kCwblf7srT9mt3Efvc8jJTmcmUW4lu6piW3lodVrGF6Kgfr+DyNURhawDVrra1a6BFBUkKdfUPByIKA/gx1/puKUPDumtuXNMxVWsUY+bP6oBe1QQhUXtip2s1AQqYduw/fagmDf9/eB3tYmIDB9DXPsY76FPDLX7NMDJYd+8slOLnIP6Poiu+0s2GKYeUYusEAQ6iNDtHWe9m8thM0rChaOYcMq X-Microsoft-Antispam-Message-Info: XSdI/dFkXVfUqgtopnEzriVzuJE2Dpq8KB7aZKeNytIKG1qE37yeyyzEjdlUD+/IH1/JVO8l0vcckP+DgPJzNpI9T4fFJ9l7b+3jdrjKcteeBCIUTpVSoqX1+/CAip7h19IBMJcEx+CH84nDba016UhwB5inVyxKmK6zCR7u3AYOvTM96R7rm9r/2VoFa0JKZrmxusxBFD6j9OXHKarn2PIdH3nR4+MWytZYJldu4shTmMzgdq4fKW9N6lBVq5ujWfs3UCJRaue4F3Ujoy9RuY7HgXFjXlaRzn3XcvSyc79SviEd7trFfzU7cPWqaod5nCIBAIrvBRjhnIaoj2LHpE1STfI3UclqYRdWmjJH/5o= X-Microsoft-Exchange-Diagnostics: 1; HE1PR04MB1529; 6:DSlpbr7t2DGCZJxq6nvvunBNa9BxnpZFctiu2IMu3oFSCniNNcr01m2CqFuGw3Vl30iIZW0vASwPKlsWxv1ha2I06xQgDYWgPxKcljfF0p5p/e9TKffSpGVd4HnxiHYvyFyKgj52QGS2741G0aIrc9sp+Ft/U3+W5vJ74Syp5bNvaH2+4lJ4UzrxqabSt1sFy+ZHYsA1KjcvQzbcTyKC2kjQIhY56Wpgc0N/46tfKZYsgU17brI+wzl5jixGnoEAihnZZvMYrAciWbV3OVncGI0opXyTuzbfRmK/Y+Ge3ds4LnTm2fgBY9TfEIeQTuI9kOhlwCmdP8nuggOnHHTyXMj3VfBWnjj7n4H+24B90KAlT/wYt5P7wWY/k4VtHsx7Q/qpPz9mXpk+GVZga6wSiZ4cT1u6tkmt2/FodT88U6714hlwfCTCkfnHuurTeJBNMI3B/g+g5fWM/Wllqf69fg==; 5:CETsjxPH0ddxpR8mN5SpKPxLn6QU6iuiQq5S6fDsfCUw733ppqFXpRHqtsxm04rAG8mMDCgsz6W+GxdL/BgIZtay8Ft7Im5sJscEGKAhb8qNJhZBX0GY+8e/XN/UfTwIspbg+cPSDreLUoszH4NX5bEWlq1eVoIINr0Ll3wpOrU=; 7:fujeWX6xOvw8cnaUJ3+unoXgwBqC0naKS0Tv8YFhwme9kJhms2hqX+sTa1fqXegh27L3Y2TA0PKYaOTl8pgjsdNvRzngJpZLCGs2CPFSf89ZH6CpvDFcXMBTS0bFkUulifoaUvPeHFuc1oQFf4y88lDCWVxns+JEBtSMRU3WvXfkml79moRGXOnStERT/ntxQh6CxbwM8/TAPqVH0OLJAuiFf4SGRIcyRqE9w4I5ca7twTLVbe1BQgATiW881r6L SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2018 06:09:34.6950 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3888705f-f368-43be-20ec-08d6193f7b1b X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR04MB1529 Subject: [dpdk-dev] [PATCH 08/10] crypto/caam_jr: add auth cipher and aead session support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Hemant Agrawal Signed-off-by: Gagandeep Singh Signed-off-by: Hemant Agrawal --- drivers/crypto/caam_jr/caam_jr.c | 710 ++++++++++++++++++++++++++++++- 1 file changed, 707 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/caam_jr/caam_jr.c b/drivers/crypto/caam_jr/caam_jr.c index 6d30c4f4d..d582b2fcb 100644 --- a/drivers/crypto/caam_jr/caam_jr.c +++ b/drivers/crypto/caam_jr/caam_jr.c @@ -21,7 +21,9 @@ /* RTA header files */ #include #include +#include #include + #include #include #include @@ -103,6 +105,71 @@ static inline int is_cipher_only(struct caam_jr_session *ses) (ses->auth_alg == RTE_CRYPTO_AUTH_NULL)); } +static inline int is_auth_only(struct caam_jr_session *ses) +{ + return ((ses->cipher_alg == RTE_CRYPTO_CIPHER_NULL) && + (ses->auth_alg != RTE_CRYPTO_AUTH_NULL)); +} + +static inline int is_aead(struct caam_jr_session *ses) +{ + return ((ses->cipher_alg == 0) && + (ses->auth_alg == 0) && + (ses->aead_alg != 0)); +} + +static inline int is_auth_cipher(struct caam_jr_session *ses) +{ + return ((ses->cipher_alg != RTE_CRYPTO_CIPHER_NULL) && + (ses->auth_alg != RTE_CRYPTO_AUTH_NULL)); +} + +static inline int is_encode(struct caam_jr_session *ses) +{ + return ses->dir == DIR_ENC; +} + +static inline int is_decode(struct caam_jr_session *ses) +{ + return ses->dir == DIR_DEC; +} + +static inline void +caam_auth_alg(struct caam_jr_session *ses, struct alginfo *alginfo_a) +{ + switch (ses->auth_alg) { + case RTE_CRYPTO_AUTH_NULL: + ses->digest_length = 0; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + alginfo_a->algtype = OP_ALG_ALGSEL_MD5; + alginfo_a->algmode = OP_ALG_AAI_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA1_HMAC: + alginfo_a->algtype = OP_ALG_ALGSEL_SHA1; + alginfo_a->algmode = OP_ALG_AAI_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + alginfo_a->algtype = OP_ALG_ALGSEL_SHA224; + alginfo_a->algmode = OP_ALG_AAI_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + alginfo_a->algtype = OP_ALG_ALGSEL_SHA256; + alginfo_a->algmode = OP_ALG_AAI_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + alginfo_a->algtype = OP_ALG_ALGSEL_SHA384; + alginfo_a->algmode = OP_ALG_AAI_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + alginfo_a->algtype = OP_ALG_ALGSEL_SHA512; + alginfo_a->algmode = OP_ALG_AAI_HMAC; + break; + default: + CAAM_JR_DEBUG("unsupported auth alg %u", ses->auth_alg); + } +} + static inline void caam_cipher_alg(struct caam_jr_session *ses, struct alginfo *alginfo_c) { @@ -126,13 +193,27 @@ caam_cipher_alg(struct caam_jr_session *ses, struct alginfo *alginfo_c) } } +static inline void +caam_aead_alg(struct caam_jr_session *ses, struct alginfo *alginfo) +{ + switch (ses->aead_alg) { + case RTE_CRYPTO_AEAD_AES_GCM: + alginfo->algtype = OP_ALG_ALGSEL_AES; + alginfo->algmode = OP_ALG_AAI_GCM; + break; + default: + CAAM_JR_DEBUG("unsupported AEAD alg %d", ses->aead_alg); + } +} + /* prepare command block of the session */ static int caam_jr_prep_cdb(struct caam_jr_session *ses) { - struct alginfo alginfo_c = {0}; + struct alginfo alginfo_c = {0}, alginfo_a = {0}, alginfo = {0}; int32_t shared_desc_len = 0; struct sec_cdb *cdb; + int err; #if RTE_BYTE_ORDER == RTE_BIG_ENDIAN int swap = false; #else @@ -171,6 +252,108 @@ caam_jr_prep_cdb(struct caam_jr_session *ses) NULL, ses->iv.length, ses->dir); + } else if (is_auth_only(ses)) { + caam_auth_alg(ses, &alginfo_a); + if (alginfo_a.algtype == (unsigned int)CAAM_JR_ALG_UNSUPPORT) { + CAAM_JR_ERR("not supported auth alg"); + rte_free(cdb); + return -ENOTSUP; + } + + alginfo_a.key = (size_t)ses->auth_key.data; + alginfo_a.keylen = ses->auth_key.length; + alginfo_a.key_enc_flags = 0; + alginfo_a.key_type = RTA_DATA_IMM; + + shared_desc_len = cnstr_shdsc_hmac(cdb->sh_desc, true, + swap, &alginfo_a, + !ses->dir, + ses->digest_length); + } else if (is_aead(ses)) { + caam_aead_alg(ses, &alginfo); + if (alginfo.algtype == (unsigned int)CAAM_JR_ALG_UNSUPPORT) { + CAAM_JR_ERR("not supported aead alg"); + rte_free(cdb); + return -ENOTSUP; + } + alginfo.key = (size_t)ses->aead_key.data; + alginfo.keylen = ses->aead_key.length; + alginfo.key_enc_flags = 0; + alginfo.key_type = RTA_DATA_IMM; + + if (ses->dir == DIR_ENC) + shared_desc_len = cnstr_shdsc_gcm_encap( + cdb->sh_desc, true, swap, + &alginfo, + ses->iv.length, + ses->digest_length); + else + shared_desc_len = cnstr_shdsc_gcm_decap( + cdb->sh_desc, true, swap, + &alginfo, + ses->iv.length, + ses->digest_length); + } else { + caam_cipher_alg(ses, &alginfo_c); + if (alginfo_c.algtype == (unsigned int)CAAM_JR_ALG_UNSUPPORT) { + CAAM_JR_ERR("not supported cipher alg"); + rte_free(cdb); + return -ENOTSUP; + } + + alginfo_c.key = (size_t)ses->cipher_key.data; + alginfo_c.keylen = ses->cipher_key.length; + alginfo_c.key_enc_flags = 0; + alginfo_c.key_type = RTA_DATA_IMM; + + caam_auth_alg(ses, &alginfo_a); + if (alginfo_a.algtype == (unsigned int)CAAM_JR_ALG_UNSUPPORT) { + CAAM_JR_ERR("not supported auth alg"); + rte_free(cdb); + return -ENOTSUP; + } + + alginfo_a.key = (size_t)ses->auth_key.data; + alginfo_a.keylen = ses->auth_key.length; + alginfo_a.key_enc_flags = 0; + alginfo_a.key_type = RTA_DATA_IMM; + + cdb->sh_desc[0] = alginfo_c.keylen; + cdb->sh_desc[1] = alginfo_a.keylen; + err = rta_inline_query(IPSEC_AUTH_VAR_AES_DEC_BASE_DESC_LEN, + MIN_JOB_DESC_SIZE, + (unsigned int *)cdb->sh_desc, + &cdb->sh_desc[2], 2); + + if (err < 0) { + CAAM_JR_ERR("Crypto: Incorrect key lengths"); + rte_free(cdb); + return err; + } + if (cdb->sh_desc[2] & 1) + alginfo_c.key_type = RTA_DATA_IMM; + else { + alginfo_c.key = (size_t)caam_jr_mem_vtop( + (void *)(size_t)alginfo_c.key); + alginfo_c.key_type = RTA_DATA_PTR; + } + if (cdb->sh_desc[2] & (1<<1)) + alginfo_a.key_type = RTA_DATA_IMM; + else { + alginfo_a.key = (size_t)caam_jr_mem_vtop( + (void *)(size_t)alginfo_a.key); + alginfo_a.key_type = RTA_DATA_PTR; + } + cdb->sh_desc[0] = 0; + cdb->sh_desc[1] = 0; + cdb->sh_desc[2] = 0; + /* Auth_only_len is set as 0 here and it will be + * overwritten in fd for each packet. + */ + shared_desc_len = cnstr_shdsc_authenc(cdb->sh_desc, + true, swap, &alginfo_c, &alginfo_a, + ses->iv.length, 0, + ses->digest_length, ses->dir); } if (shared_desc_len < 0) { @@ -422,6 +605,163 @@ caam_jr_dequeue_burst(void *qp, struct rte_crypto_op **ops, return num_rx; } +/** + * packet looks like: + * |<----data_len------->| + * |ip_header|ah_header|icv|payload| + * ^ + * | + * mbuf->pkt.data + */ +static inline struct caam_jr_op_ctx * +build_auth_only_sg(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct rte_mbuf *mbuf = sym->m_src; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg; + int length; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + struct sec_job_descriptor_t *jobdescr; + uint8_t extra_segs; + + if (is_decode(ses)) + extra_segs = 2; + else + extra_segs = 1; + + if ((mbuf->nb_segs + extra_segs) > MAX_SG_ENTRIES) { + CAAM_JR_DP_ERR("Auth: Max sec segs supported is %d", + MAX_SG_ENTRIES); + return NULL; + } + + ctx = caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op = op; + + cdb = ses->cdb; + sdesc_offset = (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + jobdescr = (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + + /* output */ + SEC_JD_SET_OUT_PTR(jobdescr, (uint64_t)sym->auth.digest.phys_addr, + 0, ses->digest_length); + + /*input */ + sg = &ctx->sg[0]; + length = sym->auth.data.length; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf) + sym->auth.data.offset); + sg->len = cpu_to_caam32(mbuf->data_len - sym->auth.data.offset); + + /* Successive segs */ + mbuf = mbuf->next; + while (mbuf) { + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf)); + sg->len = cpu_to_caam32(mbuf->data_len); + mbuf = mbuf->next; + } + + if (is_decode(ses)) { + /* digest verification case */ + sg++; + /* hash result or digest, save digest first */ + rte_memcpy(ctx->digest, sym->auth.digest.data, + ses->digest_length); +#ifdef RTE_LIBRTE_PMD_CAAM_JR_DEBUG + rte_hexdump(stdout, "ICV", ctx->digest, ses->digest_length); +#endif + sg->ptr = cpu_to_caam64(caam_jr_vtop_ctx(ctx, ctx->digest)); + sg->len = cpu_to_caam32(ses->digest_length); + length += ses->digest_length; + } else { + length -= ses->digest_length; + } + + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + SEC_JD_SET_IN_PTR(jobdescr, + (uint64_t)caam_jr_vtop_ctx(ctx, &ctx->sg[0]), 0, length); + /* enabling sg list */ + (jobdescr)->seq_in.command.word |= 0x01000000; + + return ctx; +} + +static inline struct caam_jr_op_ctx * +build_auth_only(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg; + rte_iova_t start_addr; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + struct sec_job_descriptor_t *jobdescr; + + ctx = caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op = op; + + cdb = ses->cdb; + sdesc_offset = (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + start_addr = rte_pktmbuf_iova(sym->m_src); + + jobdescr = (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + + /* output */ + SEC_JD_SET_OUT_PTR(jobdescr, (uint64_t)sym->auth.digest.phys_addr, + 0, ses->digest_length); + + /*input */ + if (is_decode(ses)) { + sg = &ctx->sg[0]; + SEC_JD_SET_IN_PTR(jobdescr, + (uint64_t)caam_jr_vtop_ctx(ctx, sg), 0, + (sym->auth.data.length + ses->digest_length)); + /* enabling sg list */ + (jobdescr)->seq_in.command.word |= 0x01000000; + + /* hash result or digest, save digest first */ + rte_memcpy(ctx->digest, sym->auth.digest.data, + ses->digest_length); + sg->ptr = cpu_to_caam64(start_addr + sym->auth.data.offset); + sg->len = cpu_to_caam32(sym->auth.data.length); + +#ifdef RTE_LIBRTE_PMD_CAAM_JR_DEBUG + rte_hexdump(stdout, "ICV", ctx->digest, ses->digest_length); +#endif + /* let's check digest by hw */ + sg++; + sg->ptr = cpu_to_caam64(caam_jr_vtop_ctx(ctx, ctx->digest)); + sg->len = cpu_to_caam32(ses->digest_length); + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + } else { + SEC_JD_SET_IN_PTR(jobdescr, (uint64_t)start_addr, + sym->auth.data.offset, sym->auth.data.length); + } + return ctx; +} static inline struct caam_jr_op_ctx * build_cipher_only_sg(struct rte_crypto_op *op, struct caam_jr_session *ses) @@ -602,6 +942,269 @@ build_cipher_only(struct rte_crypto_op *op, struct caam_jr_session *ses) return ctx; } +/* For decapsulation: + * Input: + * +----+----------------+--------------------------------+-----+ + * | IV | Auth-only data | Authenticated & Encrypted data | ICV | + * +----+----------------+--------------------------------+-----+ + * Output: + * +----+--------------------------+ + * | Decrypted & authenticated data | + * +----+--------------------------+ + */ + +static inline struct caam_jr_op_ctx * +build_cipher_auth_sg(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg, *out_sg, *in_sg; + struct rte_mbuf *mbuf; + uint32_t length = 0; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + uint8_t req_segs; + uint8_t *IV_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, + ses->iv.offset); + struct sec_job_descriptor_t *jobdescr; + uint32_t auth_only_len; + + auth_only_len = op->sym->auth.data.length - + op->sym->cipher.data.length; + + if (sym->m_dst) { + mbuf = sym->m_dst; + req_segs = mbuf->nb_segs + sym->m_src->nb_segs + 3; + } else { + mbuf = sym->m_src; + req_segs = mbuf->nb_segs * 2 + 3; + } + + if (req_segs > MAX_SG_ENTRIES) { + CAAM_JR_DP_ERR("Cipher-Auth: Max sec segs supported is %d", + MAX_SG_ENTRIES); + return NULL; + } + + ctx = caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op = op; + cdb = ses->cdb; + sdesc_offset = (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + jobdescr = (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + + /* output */ + if (sym->m_dst) + mbuf = sym->m_dst; + else + mbuf = sym->m_src; + + out_sg = &ctx->sg[0]; + if (is_encode(ses)) + length = sym->auth.data.length + ses->digest_length; + else + length = sym->auth.data.length; + + sg = &ctx->sg[0]; + + /* 1st seg */ + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf) + + sym->auth.data.offset); + sg->len = cpu_to_caam32(mbuf->data_len - sym->auth.data.offset); + + /* Successive segs */ + mbuf = mbuf->next; + while (mbuf) { + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf)); + sg->len = cpu_to_caam32(mbuf->data_len); + mbuf = mbuf->next; + } + + if (is_encode(ses)) { + /* set auth output */ + sg++; + sg->ptr = cpu_to_caam64(sym->auth.digest.phys_addr); + sg->len = cpu_to_caam32(ses->digest_length); + } + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + SEC_JD_SET_OUT_PTR(jobdescr, + (uint64_t)caam_jr_dma_vtop(out_sg), 0, length); + /* set sg bit */ + (jobdescr)->seq_out.command.word |= 0x01000000; + + /* input */ + sg++; + mbuf = sym->m_src; + in_sg = sg; + if (is_encode(ses)) + length = ses->iv.length + sym->auth.data.length; + else + length = ses->iv.length + sym->auth.data.length + + ses->digest_length; + + sg->ptr = cpu_to_caam64(caam_jr_dma_vtop(IV_ptr)); + sg->len = cpu_to_caam32(ses->iv.length); + + sg++; + /* 1st seg */ + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf) + + sym->auth.data.offset); + sg->len = cpu_to_caam32(mbuf->data_len - sym->auth.data.offset); + + /* Successive segs */ + mbuf = mbuf->next; + while (mbuf) { + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf)); + sg->len = cpu_to_caam32(mbuf->data_len); + mbuf = mbuf->next; + } + + if (is_decode(ses)) { + sg++; + rte_memcpy(ctx->digest, sym->auth.digest.data, + ses->digest_length); + sg->ptr = cpu_to_caam64(caam_jr_dma_vtop(ctx->digest)); + sg->len = cpu_to_caam32(ses->digest_length); + } + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + SEC_JD_SET_IN_PTR(jobdescr, (uint64_t)caam_jr_dma_vtop(in_sg), 0, + length); + /* set sg bit */ + (jobdescr)->seq_in.command.word |= 0x01000000; + /* Auth_only_len is set as 0 in descriptor and it is + * overwritten here in the jd which will update + * the DPOVRD reg. + */ + if (auth_only_len) + /* set sg bit */ + (jobdescr)->dpovrd = 0x80000000 | auth_only_len; + + return ctx; +} + +static inline struct caam_jr_op_ctx * +build_cipher_auth(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg; + rte_iova_t src_start_addr, dst_start_addr; + uint32_t length = 0; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + uint8_t *IV_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, + ses->iv.offset); + struct sec_job_descriptor_t *jobdescr; + uint32_t auth_only_len; + + auth_only_len = op->sym->auth.data.length - + op->sym->cipher.data.length; + + src_start_addr = rte_pktmbuf_iova(sym->m_src); + if (sym->m_dst) + dst_start_addr = rte_pktmbuf_iova(sym->m_dst); + else + dst_start_addr = src_start_addr; + + ctx = caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op = op; + cdb = ses->cdb; + sdesc_offset = (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + jobdescr = (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + + /* input */ + sg = &ctx->sg[0]; + if (is_encode(ses)) { + sg->ptr = cpu_to_caam64(caam_jr_dma_vtop(IV_ptr)); + sg->len = cpu_to_caam32(ses->iv.length); + length += ses->iv.length; + + sg++; + sg->ptr = cpu_to_caam64(src_start_addr + sym->auth.data.offset); + sg->len = cpu_to_caam32(sym->auth.data.length); + length += sym->auth.data.length; + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + } else { + sg->ptr = cpu_to_caam64(caam_jr_dma_vtop(IV_ptr)); + sg->len = cpu_to_caam32(ses->iv.length); + length += ses->iv.length; + + sg++; + sg->ptr = cpu_to_caam64(src_start_addr + sym->auth.data.offset); + sg->len = cpu_to_caam32(sym->auth.data.length); + length += sym->auth.data.length; + + rte_memcpy(ctx->digest, sym->auth.digest.data, + ses->digest_length); + sg++; + sg->ptr = cpu_to_caam64(caam_jr_dma_vtop(ctx->digest)); + sg->len = cpu_to_caam32(ses->digest_length); + length += ses->digest_length; + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + } + + SEC_JD_SET_IN_PTR(jobdescr, (uint64_t)caam_jr_dma_vtop(&ctx->sg[0]), 0, + length); + /* set sg bit */ + (jobdescr)->seq_in.command.word |= 0x01000000; + + /* output */ + sg = &ctx->sg[6]; + + sg->ptr = cpu_to_caam64(dst_start_addr + sym->cipher.data.offset); + sg->len = cpu_to_caam32(sym->cipher.data.length); + length = sym->cipher.data.length; + + if (is_encode(ses)) { + /* set auth output */ + sg++; + sg->ptr = cpu_to_caam64(sym->auth.digest.phys_addr); + sg->len = cpu_to_caam32(ses->digest_length); + length += ses->digest_length; + } + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + SEC_JD_SET_OUT_PTR(jobdescr, + (uint64_t)caam_jr_dma_vtop(&ctx->sg[6]), 0, length); + /* set sg bit */ + (jobdescr)->seq_out.command.word |= 0x01000000; + + /* Auth_only_len is set as 0 in descriptor and it is + * overwritten here in the jd which will update + * the DPOVRD reg. + */ + if (auth_only_len) + /* set sg bit */ + (jobdescr)->dpovrd = 0x80000000 | auth_only_len; + + return ctx; +} static int caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp) { @@ -629,12 +1232,25 @@ caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp) } if (rte_pktmbuf_is_contiguous(op->sym->m_src)) { - if (is_cipher_only(ses)) + if (is_auth_cipher(ses)) + ctx = build_cipher_auth(op, ses); + else if (is_aead(ses)) + goto err1; + else if (is_auth_only(ses)) + ctx = build_auth_only(op, ses); + else if (is_cipher_only(ses)) ctx = build_cipher_only(op, ses); } else { - if (is_cipher_only(ses)) + if (is_auth_cipher(ses)) + ctx = build_cipher_auth_sg(op, ses); + else if (is_aead(ses)) + goto err1; + else if (is_auth_only(ses)) + ctx = build_auth_only_sg(op, ses); + else if (is_cipher_only(ses)) ctx = build_cipher_only_sg(op, ses); } +err1: if (unlikely(!ctx)) { qp->tx_errs++; CAAM_JR_ERR("not supported sec op"); @@ -817,6 +1433,54 @@ caam_jr_cipher_init(struct rte_cryptodev *dev __rte_unused, return 0; } +static int +caam_jr_auth_init(struct rte_cryptodev *dev __rte_unused, + struct rte_crypto_sym_xform *xform, + struct caam_jr_session *session) +{ + session->auth_alg = xform->auth.algo; + session->auth_key.data = rte_zmalloc(NULL, xform->auth.key.length, + RTE_CACHE_LINE_SIZE); + if (session->auth_key.data == NULL && xform->auth.key.length > 0) { + CAAM_JR_ERR("No Memory for auth key\n"); + return -ENOMEM; + } + session->auth_key.length = xform->auth.key.length; + session->digest_length = xform->auth.digest_length; + + memcpy(session->auth_key.data, xform->auth.key.data, + xform->auth.key.length); + session->dir = (xform->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE) ? + DIR_ENC : DIR_DEC; + + return 0; +} + +static int +caam_jr_aead_init(struct rte_cryptodev *dev __rte_unused, + struct rte_crypto_sym_xform *xform, + struct caam_jr_session *session) +{ + session->aead_alg = xform->aead.algo; + session->iv.length = xform->aead.iv.length; + session->iv.offset = xform->aead.iv.offset; + session->auth_only_len = xform->aead.aad_length; + session->aead_key.data = rte_zmalloc(NULL, xform->aead.key.length, + RTE_CACHE_LINE_SIZE); + if (session->aead_key.data == NULL && xform->aead.key.length > 0) { + CAAM_JR_ERR("No Memory for aead key\n"); + return -ENOMEM; + } + session->aead_key.length = xform->aead.key.length; + session->digest_length = xform->aead.digest_length; + + memcpy(session->aead_key.data, xform->aead.key.data, + xform->aead.key.length); + session->dir = (xform->aead.op == RTE_CRYPTO_AEAD_OP_ENCRYPT) ? + DIR_ENC : DIR_DEC; + + return 0; +} static int caam_jr_set_session_parameters(struct rte_cryptodev *dev, @@ -840,6 +1504,39 @@ caam_jr_set_session_parameters(struct rte_cryptodev *dev, session->auth_alg = RTE_CRYPTO_AUTH_NULL; caam_jr_cipher_init(dev, xform, session); + /* Authentication Only */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && + xform->next == NULL) { + session->cipher_alg = RTE_CRYPTO_CIPHER_NULL; + caam_jr_auth_init(dev, xform, session); + + /* Cipher then Authenticate */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && + xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + if (xform->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) { + caam_jr_cipher_init(dev, xform, session); + caam_jr_auth_init(dev, xform->next, session); + } else { + CAAM_JR_ERR("Not supported: Auth then Cipher"); + goto err1; + } + + /* Authenticate then Cipher */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && + xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + if (xform->next->cipher.op == RTE_CRYPTO_CIPHER_OP_DECRYPT) { + caam_jr_auth_init(dev, xform, session); + caam_jr_cipher_init(dev, xform->next, session); + } else { + CAAM_JR_ERR("Not supported: Auth then Cipher"); + goto err1; + } + + /* AEAD operation for AES-GCM kind of Algorithms */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD && + xform->next == NULL) { + caam_jr_aead_init(dev, xform, session); + } else { CAAM_JR_ERR("Invalid crypto type"); return -EINVAL; @@ -847,6 +1544,13 @@ caam_jr_set_session_parameters(struct rte_cryptodev *dev, session->ctx_pool = internals->ctx_pool; return 0; + +err1: + rte_free(session->cipher_key.data); + rte_free(session->auth_key.data); + memset(session, 0, sizeof(struct caam_jr_session)); + + return -EINVAL; } static int