From patchwork Wed Dec 13 13:56:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 32225 X-Patchwork-Delegate: pablo.de.lara.guarch@intel.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6EF7A1B022; Wed, 13 Dec 2017 15:00:48 +0100 (CET) Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0077.outbound.protection.outlook.com [104.47.41.77]) by dpdk.org (Postfix) with ESMTP id C69E81B00F for ; Wed, 13 Dec 2017 15:00:45 +0100 (CET) Received: from CY4PR03CA0095.namprd03.prod.outlook.com (10.171.242.164) by CY1PR03MB2361.namprd03.prod.outlook.com (10.166.207.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.302.9; Wed, 13 Dec 2017 14:00:44 +0000 Received: from BY2FFO11FD009.protection.gbl (2a01:111:f400:7c0c::113) by CY4PR03CA0095.outlook.office365.com (2603:10b6:910:4d::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.302.9 via Frontend Transport; Wed, 13 Dec 2017 14:00:44 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; NXP1.onmicrosoft.com; dkim=none (message not signed) header.d=none;NXP1.onmicrosoft.com; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BY2FFO11FD009.mail.protection.outlook.com (10.1.14.73) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.20.282.5 via Frontend Transport; Wed, 13 Dec 2017 14:00:36 +0000 Received: from netperf2.ap.freescale.net ([10.232.133.164]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id vBDE0WHk023690; Wed, 13 Dec 2017 07:00:41 -0700 From: Akhil Goyal To: CC: , , , Akhil Goyal Date: Wed, 13 Dec 2017 19:26:57 +0530 Message-ID: <20171213135659.32648-4-akhil.goyal@nxp.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20171213135659.32648-1-akhil.goyal@nxp.com> References: <20171213135659.32648-1-akhil.goyal@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131576472368697235; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(336005)(7966004)(346002)(376002)(39380400002)(39860400002)(2980300002)(1110001)(1109001)(339900001)(189003)(199004)(59450400001)(305945005)(296002)(316002)(85426001)(16586007)(54906003)(2950100002)(5660300001)(6916009)(356003)(8656006)(6666003)(51416003)(1076002)(47776003)(77096006)(76176011)(86362001)(104016004)(106466001)(105606002)(2351001)(68736007)(81166006)(4326008)(8676002)(50226002)(81156014)(8936002)(2906002)(50466002)(97736004)(36756003)(498600001)(5890100001)(48376002)(53936002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR03MB2361; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD009; 1:kIquO8EwcDFgJzQDBC6iMOTA/dQ0/noLeV/fs/HLyS41Lsmq7y/kuNkhaETG+PHmnyRkQBocV8uCRPS0K6BmaTicKR2wGnUjbFIsDbTkWYPpS0FWDGjsa/uSGAH6eFaR MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 34858734-b976-49da-cf93-08d54231e2cc X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(4534020)(4628075)(201703131517081)(5600026)(4604075)(2017052603307); SRVR:CY1PR03MB2361; X-Microsoft-Exchange-Diagnostics: 1; CY1PR03MB2361; 3:QpkEaZWCTjxM3p9mOxUoHR3En08P78kl9itFWjgnaNrzRKeXHbTZJ3+8v8SEnrSPSL+5smBKmcTReBC/Hx/GJ1w8dHrm1tG4yo/M5a2I01NFMZhEYBdYX3vy7GdMIwcsVAQAU4Kx8cMebJUgf46bWktELFju8ORpejdWjVzRUrXhCbXNAONRWqDpnQC31cyl8jRL0pf+oHBULBBQWSgdeUF92Gk90eVC38GTw6hQwRuAYlqf/+0nE7kfRCad335ILyuzo1mTw84Lv3qMXsXa4Kk+3XBxfzsuTQpyBZ3plZTMgnm4YZjc4eeiMnxoxn861G4ErHqWMLr38wDHUv4ZAV6JqpWfnKCj4N8RHaqaneI=; 25:BLbMciB88Mp+kJBkocDQM+OHSznEftQnjvjbR4LbNlRZov/97mPCxIcStKE5dw3GNeoTd2n0TLFHOxhXg+C12PgIZRdPpcFKSfcEni3npUbGzK4mWXc1LdzyGeUL97gVku72paa233QUKI72968Oc2HXMXABn5kMR3pwnKpuHqJQXbC4+fT3YYrlGg5kL4u765nxrWG1Z0EktQbSD4gJddOp5T0iwmk+XtYkpO4PI3VwzF3nf1ErYdfYUKM77geHjZpa0D3QJb4vEv7KW9eaW58IgPAuON/VRHkwZg7WOLwtvCylmAyB9qvwBlmOoHpKze3xnFSHpCc/B9ay5z45wQ== X-MS-TrafficTypeDiagnostic: CY1PR03MB2361: X-Microsoft-Exchange-Diagnostics: 1; CY1PR03MB2361; 31:uaGEUJcWjmq3uuE2LTNjiUoQ1afR0xvOIhvXnQhOUsuzL8SOd0Zv6dbYe4T2JNGc2fgfX8/lfi96Jb27Z9GjdAMtIlr8p9KC+x7HV/9mv23F/cJHSemtlv0oQNSSlhaNN/n5BPBH09SNPf7gHJ9eqeg98/LSo1o1vl1eeLC0jslRKQ5P9nWFjkT9xubUnNPTU0DRAHZqXrOVAQhkz1AaHV8vIx8dbjxn4aHzXmoIG5o=; 4:MNB+zx/bSeSg/mmImQmn0V+kOXWx79SWGaee03zbjlA9uctAHC4rDWVNoYx7CUqCxi/ql577QdS5Kst4U1Lt9u5NtnizT+k5QSQEXdHwj0IU2OrQ+d6DzUzNI5TcbuZ0+60l71zwBXEZLr/61//FqgfTxfNyEw7BnC0j0jx658lOLJ2ALogrVaHlvzCkK86yHeoJGfTDS1hPk95pK8MZ34eXoz682yy4e2U3ZWjNcw8cgnuOva8ezL01T7+1WMIvvwifc/ij213c66CJrPcIHmDSWoz5gt/2xzKpW+e52bTEcMSqMWi7jMAIYMKpO04JqPb8y3AMCbPC7VfL3NSZQWGFb94HlNgUiNyswKyOqOk= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(192374486261705)(185117386973197); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095135)(2401047)(5005006)(8121501046)(3002001)(3231023)(93006095)(93001095)(10201501046)(6055026)(6096035)(20161123556025)(20161123559100)(20161123565025)(20161123563025)(201703131430075)(201703131520075)(201703131441075)(201703131448075)(201703131433075)(201703161259150)(20161123561025)(201708071742011); SRVR:CY1PR03MB2361; BCL:0; PCL:0; RULEID:(100000803101)(100110400095)(400006); SRVR:CY1PR03MB2361; X-Forefront-PRVS: 052017CAF1 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR03MB2361; 23:eAHtOu7D2uDjKfXReWr+tW6cTLYoP/ni1FyXia1Yk?= zAdd/EXIP2l9ilmLtSdY94olWyPy/TyNgN9L69bKyUhU10DVHlSBjvfDoD630mBqe8xWdFv72Z7F7deIX4BkqpNil58Vxv4ZefSlrCEmVus8B3Q6i6rDY95dHXhR4tvOHqhR0Ix5arYcwOxXpOqbq5SAsJKs079fkY6GzasV6cN/TwD4Q3RzbBdDkKSHmu2zV6x7wZ0mEXF0bo7A6Ma3+EWcY9718FrJF+AkOxtNWUAzbyiftySqndZjTvRA7BUfC1Y0ZM0SKmFbuScbDlrNOVjftyvC8zTI7ZkPaKAqrmG+RdaK7bA4t9R8KCytQAUN/o4f1o5Qf7wlu8MfOF9TuzdnUKsHgL13h2FDYOtOdnP3GLRpU60XJZfQ0g+1Mof3ylCbYLccLF6x6G9L3nySBEyvq711tZzjqk/L1GeQrYiBIaPKMYo9c4thCkvAxTVECobRYoU6a1yXMJOmbGt/3iKEHXqhBtqYRjdNi3W1Q9OKsGydP0Y2Ovvdd7fuZpjP8ycZORduOlloHUckJr3TsHw814eDbS6fJ6/ERpqU1v1GGKgwqTRtDdAz0OD/eZgmkRq89jwMOeIOQ3s1Kdz4dLNU4T9llGyvzSG2Np4ocZxvMTdyr2cYXTh/+JMQXxD2oBNCpdwwV1RMZxgWrqzBRn8t8Q0CVQguQL8HBJwyx+xm4xwh8HmmmP8ICRLV4Vv4h9s38i9vysyp+bJ4kasbLm8DR90suCz11RfVvRp754YeRoQD74IMl6Qbv8Q+j3MJ/M+cDOIA3H7SfwiTH9o9ljMZKBNGRsF4a0zKO/YaqviD6Lu7jrHBQO1pnZyVbeCHuLa9RLJnlNC5zi+ygYPWTP4nCAIRFLEAbqleKQNTrOg51rQHcRwjJmxyq42lknEFNfae9WZWafzPf/zQceCbusURChQ7qX3A7Bx+Ciu3aEegl8lW2hDsMufwsjI2u2XFhPafHqL1znnZHMwS66cFdagcWbFNtj3jwiT5rwBoEzqUVbk6GMYySyYNKgbWHV0S5NjuiFKnddCEBiU8lwboCgpUG9sjqmmMHe+XikoeAb0xrV1GVfltAobRUJtz+QEq1B/+LL6RxyDNMck8i86pf9atAZPW3Zyfb4KXNQd9iDA9dJu4mw/lqxCUHGE06ijmCU= X-Microsoft-Exchange-Diagnostics: 1; CY1PR03MB2361; 6:bFMnoVjIZ8c+lXfGS998YByzGNHhuaoj4qBQ/1k5xu5xeZtLtYFnke9Oo1wh9WjZA8VG93qJQ4PLDuo8vKUR6ylmQ224MXcLPGSMH5AUImvs417xlTteLsX40klQTotjraLp8pGXy7j0oHHkQghnutEe8AMTnRiZQ/7wqeMh0wFEy7Ol2rXX/ew262Cqv+O/xqBVfFHPWhi4C/eB6FWPBozSb3eaiBZX6y7z33OGrQx8r9wgoaD32cliCWphkBbqDajChdhFx6ztums780XBjnjPqa9EZCOseSEw8Y4EXzD20Sbl97y10UhglxpTAZS938xG8rgtRHnZgJ9+NMjjIDS51xzXc9v01uPa5GvySBY=; 5:p2ASTbHC3zlrE2FBJCinhlSZwZVSKo8sRoY2ZxDuodw20Vvfk3cDGDyRNspEwT68XUeE5eWIUCJjmndOgZtplVkRjn2rjwgWiNcrpUGTA9EOgPwgkh9JJgQdBbFwBaI604T9dns6u1BlPfqV4wZ191lCxRGMO8tKJ3cuBKzU5r4=; 24:BAm2tpyxT3UVyMT/mgH+TJ5/gcRfGG9RR6kNIDChiCCtwkN1u50KYmbtIu5l2ezPxtajZ0/4un7UZLGsr3mvhQGjhcLgN8pZQIupm4usvwU=; 7:qslJ4Z6cOMYiochXZqww0ZfnFUL3fAplJlIUNeySjCJO0fj/UEWrft1eU608tqV+RrE+c84vhbUADxYaZV81VpMxxZs+YXcIY5LhUTPGwEINZbi93jCMC8oT3vhZn8oINDCyZ9uPd4Or6RG94uH88mAm25d7vuN1rDpYSRGezSoDq+/OnLeTCxyJh+xGxwRdJHz3Kn0TlAX/i4RWccP521HBA7CfNv4Pj0+w608znMkC7pL0p7rmQYhVWKhkVax0 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Dec 2017 14:00:36.6825 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 34858734-b976-49da-cf93-08d54231e2cc X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR03MB2361 Subject: [dpdk-dev] [PATCH 3/5] crypto/dpaa_sec: support ipsec protocol offload X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Akhil Goyal Acked-by: Hemant Agrawal --- doc/guides/cryptodevs/features/dpaa_sec.ini | 1 + drivers/crypto/dpaa_sec/dpaa_sec.c | 409 ++++++++++++++++++++++++++-- drivers/crypto/dpaa_sec/dpaa_sec.h | 62 ++++- 3 files changed, 449 insertions(+), 23 deletions(-) diff --git a/doc/guides/cryptodevs/features/dpaa_sec.ini b/doc/guides/cryptodevs/features/dpaa_sec.ini index 0e8f5b2..deab53a 100644 --- a/doc/guides/cryptodevs/features/dpaa_sec.ini +++ b/doc/guides/cryptodevs/features/dpaa_sec.ini @@ -7,6 +7,7 @@ Symmetric crypto = Y Sym operation chaining = Y HW Accelerated = Y +Protocol offload = Y ; ; Supported crypto algorithms of the 'dpaa_sec' crypto driver. diff --git a/drivers/crypto/dpaa_sec/dpaa_sec.c b/drivers/crypto/dpaa_sec/dpaa_sec.c index b51db83..ea744e6 100644 --- a/drivers/crypto/dpaa_sec/dpaa_sec.c +++ b/drivers/crypto/dpaa_sec/dpaa_sec.c @@ -41,6 +41,7 @@ #include #include #include +#include #include #include #include @@ -222,8 +223,19 @@ dqrr_out_fq_cb_rx(struct qman_portal *qm __always_unused, * sg[1] for input */ job = dpaa_mem_ptov(qm_fd_addr_get64(fd)); + ctx = container_of(job, struct dpaa_sec_op_ctx, job); ctx->fd_status = fd->status; + if (ctx->op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct qm_sg_entry *sg_out; + uint32_t len; + + sg_out = &job->sg[0]; + hw_sg_to_cpu(sg_out); + len = sg_out->length; + ctx->op->sym->m_src->pkt_len = len; + ctx->op->sym->m_src->data_len = len; + } dpaa_sec_ops[dpaa_sec_op_nb++] = ctx->op; dpaa_sec_op_ending(ctx); @@ -287,7 +299,13 @@ static inline int is_aead(dpaa_sec_session *ses) static inline int is_auth_cipher(dpaa_sec_session *ses) { return ((ses->cipher_alg != RTE_CRYPTO_CIPHER_NULL) && - (ses->auth_alg != RTE_CRYPTO_AUTH_NULL)); + (ses->auth_alg != RTE_CRYPTO_AUTH_NULL) && + (ses->proto_alg != RTE_SECURITY_PROTOCOL_IPSEC)); +} + +static inline int is_proto_ipsec(dpaa_sec_session *ses) +{ + return (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC); } static inline int is_encode(dpaa_sec_session *ses) @@ -308,27 +326,39 @@ caam_auth_alg(dpaa_sec_session *ses, struct alginfo *alginfo_a) ses->digest_length = 0; break; case RTE_CRYPTO_AUTH_MD5_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_MD5; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_MD5_96 : OP_ALG_ALGSEL_MD5; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; case RTE_CRYPTO_AUTH_SHA1_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_SHA1; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_SHA1_96 : OP_ALG_ALGSEL_SHA1; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; case RTE_CRYPTO_AUTH_SHA224_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_SHA224; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_SHA1_160 : OP_ALG_ALGSEL_SHA224; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; case RTE_CRYPTO_AUTH_SHA256_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_SHA256; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_SHA2_256_128 : OP_ALG_ALGSEL_SHA256; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; case RTE_CRYPTO_AUTH_SHA384_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_SHA384; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_SHA2_384_192 : OP_ALG_ALGSEL_SHA384; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; case RTE_CRYPTO_AUTH_SHA512_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_SHA512; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_SHA2_512_256 : OP_ALG_ALGSEL_SHA512; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; default: @@ -343,15 +373,21 @@ caam_cipher_alg(dpaa_sec_session *ses, struct alginfo *alginfo_c) case RTE_CRYPTO_CIPHER_NULL: break; case RTE_CRYPTO_CIPHER_AES_CBC: - alginfo_c->algtype = OP_ALG_ALGSEL_AES; + alginfo_c->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_AES_CBC : OP_ALG_ALGSEL_AES; alginfo_c->algmode = OP_ALG_AAI_CBC; break; case RTE_CRYPTO_CIPHER_3DES_CBC: - alginfo_c->algtype = OP_ALG_ALGSEL_3DES; + alginfo_c->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_3DES : OP_ALG_ALGSEL_3DES; alginfo_c->algmode = OP_ALG_AAI_CBC; break; case RTE_CRYPTO_CIPHER_AES_CTR: - alginfo_c->algtype = OP_ALG_ALGSEL_AES; + alginfo_c->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_AES_CTR : OP_ALG_ALGSEL_AES; alginfo_c->algmode = OP_ALG_AAI_CTR; break; default: @@ -497,14 +533,28 @@ dpaa_sec_prep_cdb(dpaa_sec_session *ses) cdb->sh_desc[0] = 0; cdb->sh_desc[1] = 0; cdb->sh_desc[2] = 0; - - /* Auth_only_len is set as 0 here and it will be overwritten - * in fd for each packet. - */ - shared_desc_len = cnstr_shdsc_authenc(cdb->sh_desc, - true, swap, &alginfo_c, &alginfo_a, - ses->iv.length, 0, - ses->digest_length, ses->dir); + if (is_proto_ipsec(ses)) { + if (ses->dir == DIR_ENC) { + shared_desc_len = cnstr_shdsc_ipsec_new_encap( + cdb->sh_desc, + true, swap, &ses->encap_pdb, + (uint8_t *)&ses->ip4_hdr, + &alginfo_c, &alginfo_a); + } else if (ses->dir == DIR_DEC) { + shared_desc_len = cnstr_shdsc_ipsec_new_decap( + cdb->sh_desc, + true, swap, &ses->decap_pdb, + &alginfo_c, &alginfo_a); + } + } else { + /* Auth_only_len is set as 0 here and it will be + * overwritten in fd for each packet. + */ + shared_desc_len = cnstr_shdsc_authenc(cdb->sh_desc, + true, swap, &alginfo_c, &alginfo_a, + ses->iv.length, 0, + ses->digest_length, ses->dir); + } } cdb->sh_hdr.hi.field.idlen = shared_desc_len; cdb->sh_hdr.hi.word = rte_cpu_to_be_32(cdb->sh_hdr.hi.word); @@ -886,6 +936,45 @@ build_cipher_auth(struct rte_crypto_op *op, dpaa_sec_session *ses) return cf; } +static inline struct dpaa_sec_job * +build_proto(struct rte_crypto_op *op, dpaa_sec_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct dpaa_sec_job *cf; + struct dpaa_sec_op_ctx *ctx; + struct qm_sg_entry *sg; + phys_addr_t src_start_addr, dst_start_addr; + + ctx = dpaa_sec_alloc_ctx(ses); + if (!ctx) + return NULL; + cf = &ctx->job; + ctx->op = op; + + src_start_addr = rte_pktmbuf_mtophys(sym->m_src); + + if (sym->m_dst) + dst_start_addr = rte_pktmbuf_mtophys(sym->m_dst); + else + dst_start_addr = src_start_addr; + + /* input */ + sg = &cf->sg[1]; + qm_sg_entry_set64(sg, src_start_addr); + sg->length = sym->m_src->pkt_len; + sg->final = 1; + cpu_to_hw_sg(sg); + + sym->m_src->packet_type &= ~RTE_PTYPE_L4_MASK; + /* output */ + sg = &cf->sg[0]; + qm_sg_entry_set64(sg, dst_start_addr); + sg->length = sym->m_src->buf_len - sym->m_src->data_off; + cpu_to_hw_sg(sg); + + return cf; +} + static int dpaa_sec_enqueue_op(struct rte_crypto_op *op, struct dpaa_sec_qp *qp) { @@ -896,8 +985,14 @@ dpaa_sec_enqueue_op(struct rte_crypto_op *op, struct dpaa_sec_qp *qp) uint32_t auth_only_len = op->sym->auth.data.length - op->sym->cipher.data.length; - ses = (dpaa_sec_session *)get_session_private_data(op->sym->session, - cryptodev_driver_id); + if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) + ses = (dpaa_sec_session *)get_session_private_data( + op->sym->session, cryptodev_driver_id); + else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) + ses = (dpaa_sec_session *)get_sec_session_private_data( + op->sym->sec_session); + else + return -ENOTSUP; if (unlikely(!ses->qp || ses->qp != qp)) { PMD_INIT_LOG(DEBUG, "sess->qp - %p qp %p", ses->qp, qp); @@ -921,6 +1016,8 @@ dpaa_sec_enqueue_op(struct rte_crypto_op *op, struct dpaa_sec_qp *qp) auth_only_len = ses->auth_only_len; } else if (is_auth_cipher(ses)) { cf = build_cipher_auth(op, ses); + } else if (is_proto_ipsec(ses)) { + cf = build_proto(op, ses); } else { PMD_TX_LOG(ERR, "not supported sec op"); return -ENOTSUP; @@ -959,7 +1056,7 @@ dpaa_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops, /*Prepare each packet which is to be sent*/ for (loop = 0; loop < nb_ops; loop++) { - if (ops[loop]->sess_type != RTE_CRYPTO_OP_WITH_SESSION) { + if (ops[loop]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { PMD_TX_LOG(ERR, "sessionless crypto op not supported"); return 0; } @@ -1369,6 +1466,235 @@ dpaa_sec_session_clear(struct rte_cryptodev *dev, } static int +dpaa_sec_set_ipsec_session(__rte_unused struct rte_cryptodev *dev, + struct rte_security_session_conf *conf, + void *sess) +{ + struct dpaa_sec_dev_private *internals = dev->data->dev_private; + struct rte_security_ipsec_xform *ipsec_xform = &conf->ipsec; + struct rte_crypto_auth_xform *auth_xform; + struct rte_crypto_cipher_xform *cipher_xform; + dpaa_sec_session *session = (dpaa_sec_session *)sess; + + PMD_INIT_FUNC_TRACE(); + + if (ipsec_xform->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { + cipher_xform = &conf->crypto_xform->cipher; + auth_xform = &conf->crypto_xform->next->auth; + } else { + auth_xform = &conf->crypto_xform->auth; + cipher_xform = &conf->crypto_xform->next->cipher; + } + session->proto_alg = conf->protocol; + session->cipher_key.data = rte_zmalloc(NULL, + cipher_xform->key.length, + RTE_CACHE_LINE_SIZE); + if (session->cipher_key.data == NULL && + cipher_xform->key.length > 0) { + RTE_LOG(ERR, PMD, "No Memory for cipher key\n"); + return -ENOMEM; + } + + session->cipher_key.length = cipher_xform->key.length; + session->auth_key.data = rte_zmalloc(NULL, + auth_xform->key.length, + RTE_CACHE_LINE_SIZE); + if (session->auth_key.data == NULL && + auth_xform->key.length > 0) { + RTE_LOG(ERR, PMD, "No Memory for auth key\n"); + rte_free(session->cipher_key.data); + return -ENOMEM; + } + session->auth_key.length = auth_xform->key.length; + memcpy(session->cipher_key.data, cipher_xform->key.data, + cipher_xform->key.length); + memcpy(session->auth_key.data, auth_xform->key.data, + auth_xform->key.length); + + switch (auth_xform->algo) { + case RTE_CRYPTO_AUTH_SHA1_HMAC: + session->auth_alg = RTE_CRYPTO_AUTH_SHA1_HMAC; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + session->auth_alg = RTE_CRYPTO_AUTH_MD5_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + session->auth_alg = RTE_CRYPTO_AUTH_SHA256_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + session->auth_alg = RTE_CRYPTO_AUTH_SHA384_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + session->auth_alg = RTE_CRYPTO_AUTH_SHA512_HMAC; + break; + case RTE_CRYPTO_AUTH_AES_CMAC: + session->auth_alg = RTE_CRYPTO_AUTH_AES_CMAC; + break; + case RTE_CRYPTO_AUTH_NULL: + session->auth_alg = RTE_CRYPTO_AUTH_NULL; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + case RTE_CRYPTO_AUTH_AES_XCBC_MAC: + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + case RTE_CRYPTO_AUTH_SHA1: + case RTE_CRYPTO_AUTH_SHA256: + case RTE_CRYPTO_AUTH_SHA512: + case RTE_CRYPTO_AUTH_SHA224: + case RTE_CRYPTO_AUTH_SHA384: + case RTE_CRYPTO_AUTH_MD5: + case RTE_CRYPTO_AUTH_AES_GMAC: + case RTE_CRYPTO_AUTH_KASUMI_F9: + case RTE_CRYPTO_AUTH_AES_CBC_MAC: + case RTE_CRYPTO_AUTH_ZUC_EIA3: + RTE_LOG(ERR, PMD, "Crypto: Unsupported auth alg %u\n", + auth_xform->algo); + goto out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Auth specified %u\n", + auth_xform->algo); + goto out; + } + + switch (cipher_xform->algo) { + case RTE_CRYPTO_CIPHER_AES_CBC: + session->cipher_alg = RTE_CRYPTO_CIPHER_AES_CBC; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + session->cipher_alg = RTE_CRYPTO_CIPHER_3DES_CBC; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + session->cipher_alg = RTE_CRYPTO_CIPHER_AES_CTR; + break; + case RTE_CRYPTO_CIPHER_NULL: + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + case RTE_CRYPTO_CIPHER_3DES_ECB: + case RTE_CRYPTO_CIPHER_AES_ECB: + case RTE_CRYPTO_CIPHER_KASUMI_F8: + RTE_LOG(ERR, PMD, "Crypto: Unsupported Cipher alg %u\n", + cipher_xform->algo); + goto out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Cipher specified %u\n", + cipher_xform->algo); + goto out; + } + + if (ipsec_xform->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { + memset(&session->encap_pdb, 0, sizeof(struct ipsec_encap_pdb) + + sizeof(session->ip4_hdr)); + session->ip4_hdr.ip_v = IPVERSION; + session->ip4_hdr.ip_hl = 5; + session->ip4_hdr.ip_len = rte_cpu_to_be_16( + sizeof(session->ip4_hdr)); + session->ip4_hdr.ip_tos = ipsec_xform->tunnel.ipv4.dscp; + session->ip4_hdr.ip_id = 0; + session->ip4_hdr.ip_off = 0; + session->ip4_hdr.ip_ttl = ipsec_xform->tunnel.ipv4.ttl; + session->ip4_hdr.ip_p = (ipsec_xform->proto == + RTE_SECURITY_IPSEC_SA_PROTO_ESP) ? 0x32 : 0x33; + session->ip4_hdr.ip_sum = 0; + session->ip4_hdr.ip_src = ipsec_xform->tunnel.ipv4.src_ip; + session->ip4_hdr.ip_dst = ipsec_xform->tunnel.ipv4.dst_ip; + session->ip4_hdr.ip_sum = calc_chksum((uint16_t *) + (void *)&session->ip4_hdr, + sizeof(struct ip)); + + session->encap_pdb.options = + (IPVERSION << PDBNH_ESP_ENCAP_SHIFT) | + PDBOPTS_ESP_OIHI_PDB_INL | + PDBOPTS_ESP_IVSRC | + PDBHMO_ESP_ENCAP_DTTL; + session->encap_pdb.spi = ipsec_xform->spi; + session->encap_pdb.ip_hdr_len = sizeof(struct ip); + + session->dir = DIR_ENC; + } else if (ipsec_xform->direction == + RTE_SECURITY_IPSEC_SA_DIR_INGRESS) { + memset(&session->decap_pdb, 0, sizeof(struct ipsec_decap_pdb)); + session->decap_pdb.options = sizeof(struct ip) << 16; + session->dir = DIR_DEC; + } else + goto out; + session->ctx_pool = internals->ctx_pool; + session->inq = dpaa_sec_attach_rxq(internals); + if (session->inq == NULL) { + PMD_DRV_LOG(ERR, "unable to attach sec queue"); + goto out; + } + + + return 0; +out: + rte_free(session->auth_key.data); + rte_free(session->cipher_key.data); + memset(session, 0, sizeof(dpaa_sec_session)); + return -1; +} + +static int +dpaa_sec_security_session_create(void *dev, + struct rte_security_session_conf *conf, + struct rte_security_session *sess, + struct rte_mempool *mempool) +{ + void *sess_private_data; + struct rte_cryptodev *cdev = (struct rte_cryptodev *)dev; + int ret; + + if (rte_mempool_get(mempool, &sess_private_data)) { + CDEV_LOG_ERR( + "Couldn't get object from session mempool"); + return -ENOMEM; + } + + switch (conf->protocol) { + case RTE_SECURITY_PROTOCOL_IPSEC: + ret = dpaa_sec_set_ipsec_session(cdev, conf, + sess_private_data); + break; + case RTE_SECURITY_PROTOCOL_MACSEC: + return -ENOTSUP; + default: + return -EINVAL; + } + if (ret != 0) { + PMD_DRV_LOG(ERR, + "DPAA2 PMD: failed to configure session parameters"); + + /* Return session to mempool */ + rte_mempool_put(mempool, sess_private_data); + return ret; + } + + set_sec_session_private_data(sess, sess_private_data); + + return ret; +} + +/** Clear the memory of session so it doesn't leave key material behind */ +static int +dpaa_sec_security_session_destroy(void *dev __rte_unused, + struct rte_security_session *sess) +{ + PMD_INIT_FUNC_TRACE(); + void *sess_priv = get_sec_session_private_data(sess); + + dpaa_sec_session *s = (dpaa_sec_session *)sess_priv; + + if (sess_priv) { + struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); + + rte_free(s->cipher_key.data); + rte_free(s->auth_key.data); + memset(sess, 0, sizeof(dpaa_sec_session)); + set_sec_session_private_data(sess, NULL); + rte_mempool_put(sess_mp, sess_priv); + } + return 0; +} + + +static int dpaa_sec_dev_configure(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_config *config __rte_unused) { @@ -1434,6 +1760,21 @@ static struct rte_cryptodev_ops crypto_ops = { .qp_detach_session = dpaa_sec_qp_detach_sess, }; +static const struct rte_security_capability * +dpaa_sec_capabilities_get(void *device __rte_unused) +{ + return dpaa_sec_security_cap; +} + +struct rte_security_ops dpaa_sec_security_ops = { + .session_create = dpaa_sec_security_session_create, + .session_update = NULL, + .session_stats_get = NULL, + .session_destroy = dpaa_sec_security_session_destroy, + .set_pkt_metadata = NULL, + .capabilities_get = dpaa_sec_capabilities_get +}; + static int dpaa_sec_uninit(struct rte_cryptodev *dev) { @@ -1442,6 +1783,8 @@ dpaa_sec_uninit(struct rte_cryptodev *dev) if (dev == NULL) return -ENODEV; + rte_free(dev->security_ctx); + rte_mempool_free(internals->ctx_pool); rte_free(internals); @@ -1455,6 +1798,7 @@ static int dpaa_sec_dev_init(struct rte_cryptodev *cryptodev) { struct dpaa_sec_dev_private *internals; + struct rte_security_ctx *security_instance; struct dpaa_sec_qp *qp; uint32_t i, flags; int ret; @@ -1469,12 +1813,33 @@ dpaa_sec_dev_init(struct rte_cryptodev *cryptodev) cryptodev->dequeue_burst = dpaa_sec_dequeue_burst; cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | - RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING; + RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | + RTE_CRYPTODEV_FF_SECURITY; internals = cryptodev->data->dev_private; internals->max_nb_queue_pairs = RTE_DPAA_MAX_NB_SEC_QPS; internals->max_nb_sessions = RTE_DPAA_SEC_PMD_MAX_NB_SESSIONS; + /* + * For secondary processes, we don't initialise any further as primary + * has already done this work. Only check we don't need a different + * RX function + */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + PMD_INIT_LOG(DEBUG, "Device already init by primary process"); + return 0; + } + + /* Initialize security_ctx only for primary process*/ + security_instance = rte_malloc("rte_security_instances_ops", + sizeof(struct rte_security_ctx), 0); + if (security_instance == NULL) + return -ENOMEM; + security_instance->device = (void *)cryptodev; + security_instance->ops = &dpaa_sec_security_ops; + security_instance->sess_cnt = 0; + cryptodev->security_ctx = security_instance; + for (i = 0; i < internals->max_nb_queue_pairs; i++) { /* init qman fq for queue pair */ qp = &internals->qps[i]; diff --git a/drivers/crypto/dpaa_sec/dpaa_sec.h b/drivers/crypto/dpaa_sec/dpaa_sec.h index 11407d3..295abf3 100644 --- a/drivers/crypto/dpaa_sec/dpaa_sec.h +++ b/drivers/crypto/dpaa_sec/dpaa_sec.h @@ -117,7 +117,8 @@ typedef struct dpaa_sec_session_entry { uint8_t dir; /*!< Operation Direction */ enum rte_crypto_cipher_algorithm cipher_alg; /*!< Cipher Algorithm*/ enum rte_crypto_auth_algorithm auth_alg; /*!< Authentication Algorithm*/ - enum rte_crypto_aead_algorithm aead_alg; /*!< Authentication Algorithm*/ + enum rte_crypto_aead_algorithm aead_alg; /*!< AEAD Algorithm*/ + enum rte_security_session_protocol proto_alg; /*!< Security Algorithm*/ union { struct { uint8_t *data; /**< pointer to key data */ @@ -140,6 +141,9 @@ typedef struct dpaa_sec_session_entry { } iv; /**< Initialisation vector parameters */ uint16_t auth_only_len; /*!< Length of data for Auth only */ uint32_t digest_length; + struct ipsec_encap_pdb encap_pdb; + struct ip ip4_hdr; + struct ipsec_decap_pdb decap_pdb; struct dpaa_sec_qp *qp; struct qman_fq *inq; struct sec_cdb cdb; /**< cmd block associated with qp */ @@ -404,4 +408,60 @@ static const struct rte_cryptodev_capabilities dpaa_sec_capabilities[] = { RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() }; +static const struct rte_security_capability dpaa_sec_security_cap[] = { + { /* IPsec Lookaside Protocol offload ESP Transport Egress */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, + .ipsec = { + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, + .mode = RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, + .direction = RTE_SECURITY_IPSEC_SA_DIR_EGRESS, + .options = { 0 } + }, + .crypto_capabilities = dpaa_sec_capabilities + }, + { /* IPsec Lookaside Protocol offload ESP Tunnel Ingress */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, + .ipsec = { + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, + .mode = RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, + .direction = RTE_SECURITY_IPSEC_SA_DIR_INGRESS, + .options = { 0 } + }, + .crypto_capabilities = dpaa_sec_capabilities + }, + { + .action = RTE_SECURITY_ACTION_TYPE_NONE + } +}; + +/** + * Checksum + * + * @param buffer calculate chksum for buffer + * @param len buffer length + * + * @return checksum value in host cpu order + */ +static inline uint16_t +calc_chksum(void *buffer, int len) +{ + uint16_t *buf = (uint16_t *)buffer; + uint32_t sum = 0; + uint16_t result; + + for (sum = 0; len > 1; len -= 2) + sum += *buf++; + + if (len == 1) + sum += *(unsigned char *)buf; + + sum = (sum >> 16) + (sum & 0xFFFF); + sum += (sum >> 16); + result = ~sum; + + return result; +} + #endif /* _DPAA_SEC_H_ */