From patchwork Fri Jan 20 14:05:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 19828 X-Patchwork-Delegate: pablo.de.lara.guarch@intel.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id B53BEF92F; Fri, 20 Jan 2017 09:40:30 +0100 (CET) Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0085.outbound.protection.outlook.com [104.47.33.85]) by dpdk.org (Postfix) with ESMTP id F29FD2B9B for ; Fri, 20 Jan 2017 09:39:49 +0100 (CET) Received: from BN6PR03CA0032.namprd03.prod.outlook.com (10.175.124.18) by MWHPR03MB2480.namprd03.prod.outlook.com (10.169.200.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.845.12; Fri, 20 Jan 2017 08:39:47 +0000 Received: from BN1AFFO11FD035.protection.gbl (2a01:111:f400:7c10::124) by BN6PR03CA0032.outlook.office365.com (2603:10b6:404:10c::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.860.13 via Frontend Transport; Fri, 20 Jan 2017 08:39:46 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; nxp.com; dkim=none (message not signed) header.d=none; nxp.com; dmarc=fail action=none header.from=nxp.com; nxp.com; dkim=none (message not signed) header.d=none; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BN1AFFO11FD035.mail.protection.outlook.com (10.58.52.159) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.803.8 via Frontend Transport; Fri, 20 Jan 2017 08:39:46 +0000 Received: from netperf2.ap.freescale.net ([10.232.133.164]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id v0K8dNVD021447; Fri, 20 Jan 2017 01:39:43 -0700 From: To: CC: , , , , , Akhil Goyal , Hemant Agrawal Date: Fri, 20 Jan 2017 19:35:05 +0530 Message-ID: <20170120140509.4495-7-akhil.goyal@nxp.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170120140509.4495-1-akhil.goyal@nxp.com> References: <20161222201700.20020-1-akhil.goyal@nxp.com> <20170120140509.4495-1-akhil.goyal@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131293751863234830; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(336005)(7916002)(39850400002)(39400400002)(39410400002)(39380400002)(39840400002)(39450400003)(39860400002)(2980300002)(1110001)(1109001)(339900001)(189002)(199003)(4326007)(92566002)(33646002)(2876002)(626004)(5660300001)(50226002)(97736004)(356003)(2906002)(86152003)(53936002)(6916009)(8936002)(305945005)(54906002)(47776003)(2950100002)(575784001)(86362001)(8656002)(68736007)(110136003)(105606002)(38730400001)(48376002)(81156014)(106466001)(36756003)(6666003)(81166006)(76176999)(5003940100001)(2351001)(77096006)(50466002)(1076002)(53946003)(50986999)(8676002)(85426001)(189998001)(104016004)(579004); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR03MB2480; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1AFFO11FD035; 1:kgraiDwDrXJEXhrG+TDKuEW75LcUHVnryQLqJx7AJXdHTrxOA3Lhbf4Lrip+XSvZe8aD6aXPsinctTs9bjx3DzW8xOfarI18TARtKKZ0u58kR4Y9sYe63oc3pR9xVM48ZkDhv5R1MytrOmADa2/vtit4D9I7xLBKGnD/N1jXvSjJpMi8MGrUX/rIbE8+Mp9YK3bUUMsK/xr0TLZ/nCkIQQmPeCw/Jzg4EgkDp61Y2Bzh2AVytqYksxQ4aVypWbLZx4bRdIQpmvDGr4g9d1OhDSgzqacXewlDl/tp7iTFOUb63fvA2OFtPy4xJNIhlZ7xsPhRD9TA4dByhVqMimUWqNIgZU21pKYYZVzIhuPKYaEgD2TYQCPJaslhCzxOwpcQ9l9RHdH+mLKTl1dmLtFxIfW54rZwMWeXv1PXgIEIPJRTbqVrBu3wXqVPJfdXF/7glcoKhq+rk/QiTpPkvoWXZFhesf2Gqolvu64rC0T8LqkxeZ2WP+B7982A1ue0s4wVXMqNXJ7BKC8nyLzjA1lBEuSyeUc9DGaMt8ddD2Mg3dsNuqp5Ca8GZBA6F3LdfU0QpBllwekcCVmDm3TWG9en0kgoczHoKvn2ZJzD+FuiCZod3eUR+w4UMSb8BXP19UdaiX8XwG7a+ADuaGcaER/61OMKHArlefMkdeMssvKX26SWrbWHa8vGHJk6sY+tEPLvSiQkHcTVtsH4cCX4aH3o774HVxLV2c2bwyejKpT90Eh5YIbiMAcC42lybeItJYLH MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: ae13f224-6842-4548-510e-08d4410fe380 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:MWHPR03MB2480; X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2480; 3:2fTIDnHFaQLYF79VWupzKGCXeX1tClxq/FbhQ3H+DuFMhNqAP+c5xT8pwDYtYvLCFAlKbQQJhAwUhgMzEI2ukqoOA/r7ngIOcpBpk09exr0z4tzHDdtTOcpemhQZanI7UQL0GbVBmgcpoY8SnLzEIPr2aHMjcLJhZPaP7cfjEk9dnc3vFGXI7XPTZhINwmrssaGNMiUdXxH0yijnMX6Ko63APP2eXUi3M4lnBTertjLjKTrrvkxHJsGxaIGM4qukmtuXm/woqK7oyoqrD16YlNiKlvEy8wrm9kxlfCoyrnhjrfEj6RXyRUPxrb9of+xRsYOvy1fhHcuXYdUKrrDZQUDkg/JjWgRr8aFCrPN7nEVHfRzchCgNEo2qtME+rvr7 X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2480; 25:23e+5fJhaLBiQKYcK8RezyOkTrSG2fvJk5Z0bJFmKkOg/J4dxv1Kl/rw3i0ydMCBtPKJS8nQAYpuguYl/rqsaBOUavCF0XjJapk8oyZve2eqlRDAcOKprdVxHt9RqT3lxG15wmCH1sOiQxjtQiJK6PT402GoiGL851T+W9/YPS4mTqRfpDm5vXJXD6sjIfeRMpi6XTbGxJTsMZgczm3PXS38KfXFD+a9r/Bovz3t5rqU28BIELAWL4Ykhc1f+98Jr6rmJSY9ZnNp8EvPGzugASsFxvcvQf42tFPHuvVv7YlAyPwOSn/PGwTRbMf5snVdWwwDbKEovQJQZguSbTJGsL7hzFEH8A95Ajcj2WTs5VU5amPwieDGZZYskrh2oAo6ESbPp7l50ABuk7LVHmr2w7gOK10iYRZcTnBAMS275JJLlxzNNOOklwxsRxcwlCL2bAsd4OBBKoz9hdMQdRMpWk4mhU8ydZtlKT9ZHFkqB1pKh62joe1Xj6EPBzzQ0I6yrDsSqwFXkN1wMgA774/OrFC8mQolJ2EWI2qckXrzxOQ9HalqSfz5i0g3AHNJGGZQr/0SAMc35VkHpZvdAEzPyOylCtX/uRAbNPedFI+UEp1URZ5IJ0ANZqg2PAXFxBF0H2QMdww3zhMkT779IJbUo7M0j+vXbt9hvkOIclA5lIfw8JEfuPaePQJJF0YxHZG3EoupbdlJpF+CTRb1k8vbFWnNP/NZji5mVqX0Fgb0QqsS0gaLJ514zWspmIchAHWYBFu2evvARVZ6QbDtEtYOdA== X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2480; 31:WQ9BR9AMZcDkgTdAU7kc/9AJdfNmMf+ZZ189koPyQ6cI7NpOB3gNgLbfCx9V4DjMwz0YDhBke5pChdHp9MORxQkoQG5QxoOTQg0nlg3WNUvtv9oHGF9gxWh/bHZFmiNPpxyNSBEOtItrdI4iWq8H1Z0k6dZQA36Owj45Xfz01pK/ZB/R7HkVT3gMM5bcSohfDZMC2849VYRpCvr+Yijtaw8AaAKc67Kz8snnWM9afthsc3cr23W/uothmEljaKqsBpKEB8c6+QsODkh05uevgh65gXzlEHzeQ+hFxr4Tip8= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(275809806118684); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095060)(601004)(2401047)(5005006)(8121501046)(13017025)(13024025)(13023025)(13015025)(13018025)(10201501046)(3002001)(6055026)(6096035)(20161123565025)(20161123559025)(20161123563025)(20161123556025)(20161123561025); SRVR:MWHPR03MB2480; BCL:0; PCL:0; RULEID:(400006); SRVR:MWHPR03MB2480; X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2480; 4:rB/9jQkImDwfu7HWujf1TfxDWf33GiP2ErcHvJnkmofBLvyV+ikOeLvUrNm69otwVIZg7aXPv0bOFmZKTiOs8UHmnewUnNtlay9vb0stHikUGrw3iRskV1kftX56VKPHyYxXkruMPNFmRMLnvLUnoh9xP013WdOLl3qPVa2lXXVjauzatpaXKjtu50XNaLPsUPZ3CqS9KVv8BVPKxRnmAzDGnQcvbX2tRvegpP9agRKZ6FOvHoJUoXJOzTp4NSsyFE350QvHIasIxBleQmOreQE1t8LvemGGVgyKZhJ3S0jv7u2EUvjNR5NiVrJZL+m+MMJ7fxBLIeD//7xO/OIkNo57EV0IW9JMytFXrif0iK3O3tZJQ6QLsWgdf/aPE/5cuTFWPtjUS29tpPthxYwKQV3uyx76SPZH/SRU5EMzAe7+XtobEJ++4SY/pJXULMN0fbKA6oEMgD2PPPYm2khdzJiBW635SD13nynCnMTQjmpvKxU9aeUMF1ybxkclvkrMEb1u8mrFTNZ2ycHGj7ulu0eTwjMeaLC3IMpBx+Ph3qEx5bBD3dJEnI5FgMBaTyOiCQw0UreKAJne3aB1z/EpFCEn5+r0BD5oGSy7X3pGs3eBQ4tK6IBE2tD2pfnCxfmqmFeapvxbjeG8/flz8JlWfnObhy17T53RmFp8vFwmcW5WIe24Mz3OsvSG2aSiGnt2U9VjnQB0fLEgwSV7jBVvXGK5WBaMeYC+9cyS4Jnrn4JQ8owoXNjr1MWfD0p1KoQFI9gLI2xmwqqjEJek98JgUA== X-Forefront-PRVS: 01930B2BA8 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR03MB2480; 23:K830sQ8sYgnr1XjGRV+KP/Sm7ZqQUqhS7Km5O5tne?= lwIxpQafvHCwlCc7tfJ8Vy4Twn8PdLurGyoeopsLZHhgwcoiJyQiflRRC4oh2+pgrb5OD/pGIHiQsoJEG1Wwm37fqyFbZrd7kgWC9xta/DnqrYRo5mfz/GzuqMud1x2bG0PiivpZyTQGVW9r2vW+EzldjLS86+ixiLa6s/xsL7TzagJUAfMj/H/kcn/wTwjwGt0uGerIFa6kPcxGnELahnT7AVqZFroMpPAOevFoPoEUMzpFcvipbZmyWxH3Bb1qH6Uv/u6t49nC2f2baMfCF0iagcCst16SgBMkJJWfRbqSbHYJq4WjpDmbeJKc7JgJ2P8yfdslFnsJGRwPRW7Gi6EjUbcSXmzAWVztfWWx8JJA+Ztz4xhwEXKkN7BGQsky9h/5/j2xrHCUvjjcYWZzHeK20JwnZ8HYV5lKF0klfrJhw5Tzi8/NMBQwGfafuJbalJfbJ5dKdPltwnQbewPCbKxy/U3c6J+4BkzLr81R/2qsJ+dn24jW8CPP9Eitqs6gC6FLYw2jvz1mfRpoYxXpJG5/coSiz+/4Wa7DCSFPPDrWt8s+z5LiFTLD2utgzUM+VeqT24gtYM1VIOtczEe1i4tkFlgk8RCXi0QLfSIDbS5oY12CeXNI3GETRUSNC4vm4YSPTh0qdPTQIyZ6KiZuvhj9pRke9Zvn6guXVTj3sxv0v0g8LkKTudELlPgSfMzx22ebhARgyQgK3hPkJwAPbkONqwiagCk4No4QDihGxAv2V/pGj/QCtqBvRcGFlq8EoaHTNlQWdZBkRbXw0V8Fnm6nJawwyzFTTs5YwJ3F0axBrDlSPkF+UIWrlK27s6k/6h7YdMj8dIyUg0d3kHSGYPCqFjILvwfKAieMMgDg+KvOGQGYkVAkiVzT9hw5XvBcrcw/28zcgOkANVgzEhGc/cNwqdAbpfdiAL3A1rbQwDu6VypJtRdGNIXJOnd21LGVl0/l6HuUpvs6Z3BxVIRgpAll1A60LaLvYyyF8gPDIbxiKQWKo8kl0Eh1giNgKql38mXaqvsfBu7O+jNgQpV/HE3WF/MJ+P3WqD2UNNMxBPZ9hE0VWQ1YFcdvlQPEr2739OxlM6Lhy/z4ZlFsCQGd9nBmXf1IvEAi2/ypVSSg9pwZsY4yOYxa6kPAyp2e/5U3ek2ymWll1++oN/IheDh5B0odR4M2M/lSFvoHuHSPrnGIxXVelLXybGZ+xSg3/VZehGn2RFjWImsWthHnw7kCGF5S15j3orpWCTNCoFpzjeZcoEyjIBTUHbmtYt4ROBG3scSuyxGBG9FzaiFjNmZmUyaukAlxDCAix7g0FFkxlD4eHNG9COUy4S+VJb06qglTzzFxLDQmSd3OaHeO5vN8gKVqORqOLT7FAjnOtl/42WxKQ== X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2480; 6:QHN77ewLPZwzggZXC8eD/VCGXMVfGE7FYDiW4DpGWsiJy9HTPmI1Da4easOeE+MXUVEPKFhlSpfcFfvPLqa1/9cKz3cdNfda0mK0zi2EeewrxwMCcBCNQj4cEz9ITdT3buh2NSR989LnMVix45yZ1Ktx31hajvAN3h7xURahF/n3GcagUmi17AUuZbT5D65PMa5jgRow6CxBhYPBb/GI0Ii0/O3Z2Kc33GsT7/1OSEEAvjOiJn44QD+5xXXyxBizgLoDcxHGyKaVfjgvf0bKkRbWrCA3KJJAmiBwiCbsF56enQ7UalkQoX3Cw71uuuRDQKQkW6Z8Mb9fHh/D76x0Z6/QTcxRJwybZrVcv2Db8bDxDi5lxQlTc0teCtI++xCxllUq5KzMxCr14NQF175liVCHS2kzRsGfbRkcvnXIikzTxv/dZxKt7OcSN2ljR520; 5:a+etCE/PAAhTrYp30OQs6N5E1OPQlU82Q9qbXBeX5UArFhQxPPkxGb1Ovkldvo9AqZ00uI5cXPP61w6pRh5jGa7rY454aJThYD1VQjzPj/GQIvSx4LGIqxRG317VYw0ESGCqvweDZGm4ow1W6UHZKp6QjwNUOkL3rmua/qZcAmP0pIfoxE64Otu6BqJBU97C; 24:zhhv7/bDPpTiR9EnuMXU8RkqxmbZIXuWSqErTFKpYyB5nJcJjKpfmN++f3QHQZ838mx6xDNkmJrdeLAhXSXCt+iLOI8WJ6W1QeoMU3zHR6o= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2480; 7:chUx2gbl1ogDbEZkPwsRz5Xrmigmf7Ldt4zHNz6uNAEhdj/th82hpRK5MzDbs97G0QrOYE4yR0fEelJGUI88MKSKkI0uIzOlu4l38TQrZ0X8rvCZ6ti1SCDBy83ub2azVYGs3mRCfzhU9DsxATCefg7VhETqBDRzMgPPJh88NE4Rj1fRE/pwEKcQqsOjI+LE+4lXX2VmmfX8C7L/wVAHJe3zhDRFydf9SYIeDDA5DVE5A46T5TvGOqDhxxmX9zYrx0bAB2ComPMP3ZGihKs/1oX+GEl1TV/9LIhYpYOooCEx6vu0PdRNz6+Cva46OA0hXfR5nyMOseqGB7NivgVwgIns8wNNkP5FHICRCCsmlUxqDBOHDzZT5qO8oaH1ZgiHBgIJ7oS68OMvSzskPMCHNZNQgq8LwU3YIIl0G5/euIsrqRyULOLtm9uWvftkztTBF51OlG7QGvQWFa/16KeaTw== X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jan 2017 08:39:46.1362 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB2480 Subject: [dpdk-dev] [PATCH v3 06/10] crypto/dpaa2_sec: add crypto operation support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Akhil Goyal Signed-off-by: Akhil Goyal Signed-off-by: Hemant Agrawal --- drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 25 + drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 1210 +++++++++++++++++++++++++++ drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 143 ++++ 3 files changed, 1378 insertions(+) diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h index 052a171..99a18de 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h +++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h @@ -146,8 +146,11 @@ struct qbman_fle { } while (0) #define DPAA2_SET_FD_LEN(fd, length) (fd)->simple.len = length #define DPAA2_SET_FD_BPID(fd, bpid) ((fd)->simple.bpid_offset |= bpid) +#define DPAA2_SET_FD_IVP(fd) ((fd->simple.bpid_offset |= 0x00004000)) #define DPAA2_SET_FD_OFFSET(fd, offset) \ ((fd->simple.bpid_offset |= (uint32_t)(offset) << 16)) +#define DPAA2_SET_FD_INTERNAL_JD(fd, len) fd->simple.frc = (0x80000000 | (len)) +#define DPAA2_SET_FD_FRC(fd, frc) fd->simple.frc = frc #define DPAA2_RESET_FD_CTRL(fd) (fd)->simple.ctrl = 0 #define DPAA2_SET_FD_ASAL(fd, asal) ((fd)->simple.ctrl |= (asal << 16)) @@ -155,12 +158,32 @@ struct qbman_fle { fd->simple.flc_lo = lower_32_bits((uint64_t)(addr)); \ fd->simple.flc_hi = upper_32_bits((uint64_t)(addr)); \ } while (0) +#define DPAA2_SET_FLE_INTERNAL_JD(fle, len) (fle->frc = (0x80000000 | (len))) +#define DPAA2_GET_FLE_ADDR(fle) \ + (uint64_t)((((uint64_t)(fle->addr_hi)) << 32) + fle->addr_lo) +#define DPAA2_SET_FLE_ADDR(fle, addr) do { \ + fle->addr_lo = lower_32_bits((uint64_t)addr); \ + fle->addr_hi = upper_32_bits((uint64_t)addr); \ +} while (0) +#define DPAA2_SET_FLE_OFFSET(fle, offset) \ + ((fle)->fin_bpid_offset |= (uint32_t)(offset) << 16) +#define DPAA2_SET_FLE_BPID(fle, bpid) ((fle)->fin_bpid_offset |= (uint64_t)bpid) +#define DPAA2_GET_FLE_BPID(fle, bpid) (fle->fin_bpid_offset & 0x000000ff) +#define DPAA2_SET_FLE_FIN(fle) (fle->fin_bpid_offset |= (uint64_t)1 << 31) +#define DPAA2_SET_FLE_IVP(fle) (((fle)->fin_bpid_offset |= 0x00004000)) +#define DPAA2_SET_FD_COMPOUND_FMT(fd) \ + (fd->simple.bpid_offset |= (uint32_t)1 << 28) #define DPAA2_GET_FD_ADDR(fd) \ ((uint64_t)((((uint64_t)((fd)->simple.addr_hi)) << 32) + (fd)->simple.addr_lo)) #define DPAA2_GET_FD_LEN(fd) ((fd)->simple.len) #define DPAA2_GET_FD_BPID(fd) (((fd)->simple.bpid_offset & 0x00003FFF)) +#define DPAA2_GET_FD_IVP(fd) ((fd->simple.bpid_offset & 0x00004000) >> 14) #define DPAA2_GET_FD_OFFSET(fd) (((fd)->simple.bpid_offset & 0x0FFF0000) >> 16) +#define DPAA2_SET_FLE_SG_EXT(fle) (fle->fin_bpid_offset |= (uint64_t)1 << 29) +#define DPAA2_IS_SET_FLE_SG_EXT(fle) \ + ((fle->fin_bpid_offset & ((uint64_t)1 << 29)) ? 1 : 0) + #define DPAA2_INLINE_MBUF_FROM_BUF(buf, meta_data_size) \ ((struct rte_mbuf *)((uint64_t)(buf) - (meta_data_size))) @@ -215,6 +238,7 @@ static phys_addr_t dpaa2_mem_vtop(uint64_t vaddr) */ #define DPAA2_MBUF_VADDR_TO_IOVA(mbuf) ((mbuf)->buf_physaddr) +#define DPAA2_OP_VADDR_TO_IOVA(op) (op->phys_addr) /** * macro to convert Virtual address to IOVA @@ -235,6 +259,7 @@ static phys_addr_t dpaa2_mem_vtop(uint64_t vaddr) #else /* RTE_LIBRTE_DPAA2_USE_PHYS_IOVA */ #define DPAA2_MBUF_VADDR_TO_IOVA(mbuf) ((mbuf)->buf_addr) +#define DPAA2_OP_VADDR_TO_IOVA(op) (op) #define DPAA2_VADDR_TO_IOVA(_vaddr) (_vaddr) #define DPAA2_IOVA_TO_VADDR(_iova) (_iova) #define DPAA2_MODIFY_IOVA_TO_VADDR(_mem, _type) diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index d6b6176..26ece22 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -48,16 +48,1215 @@ #include #include #include +#include #include #include "dpaa2_sec_priv.h" #include "dpaa2_sec_logs.h" +/* RTA header files */ +#include +#include + +/* Minimum job descriptor consists of a oneword job descriptor HEADER and + * a pointer to the shared descriptor + */ +#define MIN_JOB_DESC_SIZE (CAAM_CMD_SZ + CAAM_PTR_SZ) #define FSL_VENDOR_ID 0x1957 #define FSL_DEVICE_ID 0x410 #define FSL_SUBSYSTEM_SEC 1 #define FSL_MC_DPSECI_DEVID 3 +#define NO_PREFETCH 0 +#define TDES_CBC_IV_LEN 8 +#define AES_CBC_IV_LEN 16 +enum rta_sec_era rta_sec_era = RTA_SEC_ERA_8; + +static inline void +print_fd(const struct qbman_fd *fd) +{ + printf("addr_lo: %u\n", fd->simple.addr_lo); + printf("addr_hi: %u\n", fd->simple.addr_hi); + printf("len: %u\n", fd->simple.len); + printf("bpid: %u\n", DPAA2_GET_FD_BPID(fd)); + printf("fi_bpid_off: %u\n", fd->simple.bpid_offset); + printf("frc: %u\n", fd->simple.frc); + printf("ctrl: %u\n", fd->simple.ctrl); + printf("flc_lo: %u\n", fd->simple.flc_lo); + printf("flc_hi: %u\n\n", fd->simple.flc_hi); +} + +static inline void +print_fle(const struct qbman_fle *fle) +{ + printf("addr_lo: %u\n", fle->addr_lo); + printf("addr_hi: %u\n", fle->addr_hi); + printf("len: %u\n", fle->length); + printf("fi_bpid_off: %u\n", fle->fin_bpid_offset); + printf("frc: %u\n", fle->frc); +} + +static inline int +build_authenc_fd(dpaa2_sec_session *sess, + struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + struct rte_crypto_sym_op *sym_op = op->sym; + struct ctxt_priv *priv = sess->ctxt; + struct qbman_fle *fle, *sge; + struct sec_flow_context *flc; + uint32_t auth_only_len = sym_op->auth.data.length - + sym_op->cipher.data.length; + int icv_len = sym_op->auth.digest.length; + uint8_t *old_icv; + uint32_t mem_len = (7 * sizeof(struct qbman_fle)) + icv_len; + + PMD_INIT_FUNC_TRACE(); + + /* TODO we are using the first FLE entry to store Mbuf. + * Currently we donot know which FLE has the mbuf stored. + * So while retreiving we can go back 1 FLE from the FD -ADDR + * to get the MBUF Addr from the previous FLE. + * We can have a better approach to use the inline Mbuf + */ + /* TODO - we can use some mempool to avoid malloc here */ + fle = rte_zmalloc(NULL, mem_len, RTE_CACHE_LINE_SIZE); + if (!fle) { + RTE_LOG(ERR, PMD, "Memory alloc failed for SGE\n"); + return -1; + } + DPAA2_SET_FLE_ADDR(fle, DPAA2_OP_VADDR_TO_IOVA(op)); + fle = fle + 1; + sge = fle + 2; + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FD_BPID(fd, bpid); + DPAA2_SET_FLE_BPID(fle, bpid); + DPAA2_SET_FLE_BPID(fle + 1, bpid); + DPAA2_SET_FLE_BPID(sge, bpid); + DPAA2_SET_FLE_BPID(sge + 1, bpid); + DPAA2_SET_FLE_BPID(sge + 2, bpid); + DPAA2_SET_FLE_BPID(sge + 3, bpid); + } else { + DPAA2_SET_FD_IVP(fd); + DPAA2_SET_FLE_IVP(fle); + DPAA2_SET_FLE_IVP((fle + 1)); + DPAA2_SET_FLE_IVP(sge); + DPAA2_SET_FLE_IVP((sge + 1)); + DPAA2_SET_FLE_IVP((sge + 2)); + DPAA2_SET_FLE_IVP((sge + 3)); + } + + /* Save the shared descriptor */ + flc = &priv->flc_desc[0].flc; + /* Configure FD as a FRAME LIST */ + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(fle)); + DPAA2_SET_FD_COMPOUND_FMT(fd); + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + PMD_TX_LOG(DEBUG, "auth_off: 0x%x/length %d, digest-len=%d\n" + "cipher_off: 0x%x/length %d, iv-len=%d data_off: 0x%x\n", + sym_op->auth.data.offset, + sym_op->auth.data.length, + sym_op->auth.digest.length, + sym_op->cipher.data.offset, + sym_op->cipher.data.length, + sym_op->cipher.iv.length, + sym_op->m_src->data_off); + + /* Configure Output FLE with Scatter/Gather Entry */ + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + if (auth_only_len) + DPAA2_SET_FLE_INTERNAL_JD(fle, auth_only_len); + fle->length = (sess->dir == DIR_ENC) ? + (sym_op->cipher.data.length + icv_len) : + sym_op->cipher.data.length; + + DPAA2_SET_FLE_SG_EXT(fle); + + /* Configure Output SGE for Encap/Decap */ + DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->cipher.data.offset + + sym_op->m_src->data_off); + sge->length = sym_op->cipher.data.length; + + if (sess->dir == DIR_ENC) { + sge++; + DPAA2_SET_FLE_ADDR(sge, + DPAA2_VADDR_TO_IOVA(sym_op->auth.digest.data)); + sge->length = sym_op->auth.digest.length; + DPAA2_SET_FD_LEN(fd, (sym_op->auth.data.length + + sym_op->cipher.iv.length)); + } + DPAA2_SET_FLE_FIN(sge); + + sge++; + fle++; + + /* Configure Input FLE with Scatter/Gather Entry */ + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + DPAA2_SET_FLE_SG_EXT(fle); + DPAA2_SET_FLE_FIN(fle); + fle->length = (sess->dir == DIR_ENC) ? + (sym_op->auth.data.length + sym_op->cipher.iv.length) : + (sym_op->auth.data.length + sym_op->cipher.iv.length + + sym_op->auth.digest.length); + + /* Configure Input SGE for Encap/Decap */ + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(sym_op->cipher.iv.data)); + sge->length = sym_op->cipher.iv.length; + sge++; + + DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->auth.data.offset + + sym_op->m_src->data_off); + sge->length = sym_op->auth.data.length; + if (sess->dir == DIR_DEC) { + sge++; + old_icv = (uint8_t *)(sge + 1); + memcpy(old_icv, sym_op->auth.digest.data, + sym_op->auth.digest.length); + memset(sym_op->auth.digest.data, 0, sym_op->auth.digest.length); + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(old_icv)); + sge->length = sym_op->auth.digest.length; + DPAA2_SET_FD_LEN(fd, (sym_op->auth.data.length + + sym_op->auth.digest.length + + sym_op->cipher.iv.length)); + } + DPAA2_SET_FLE_FIN(sge); + if (auth_only_len) { + DPAA2_SET_FLE_INTERNAL_JD(fle, auth_only_len); + DPAA2_SET_FD_INTERNAL_JD(fd, auth_only_len); + } + return 0; +} + +static inline int +build_auth_fd(dpaa2_sec_session *sess, struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + struct rte_crypto_sym_op *sym_op = op->sym; + struct qbman_fle *fle, *sge; + uint32_t mem_len = (sess->dir == DIR_ENC) ? + (3 * sizeof(struct qbman_fle)) : + (5 * sizeof(struct qbman_fle) + + sym_op->auth.digest.length); + struct sec_flow_context *flc; + struct ctxt_priv *priv = sess->ctxt; + uint8_t *old_digest; + + PMD_INIT_FUNC_TRACE(); + + fle = rte_zmalloc(NULL, mem_len, RTE_CACHE_LINE_SIZE); + if (!fle) { + RTE_LOG(ERR, PMD, "Memory alloc failed for FLE\n"); + return -1; + } + /* TODO we are using the first FLE entry to store Mbuf. + * Currently we donot know which FLE has the mbuf stored. + * So while retreiving we can go back 1 FLE from the FD -ADDR + * to get the MBUF Addr from the previous FLE. + * We can have a better approach to use the inline Mbuf + */ + DPAA2_SET_FLE_ADDR(fle, DPAA2_OP_VADDR_TO_IOVA(op)); + fle = fle + 1; + + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FD_BPID(fd, bpid); + DPAA2_SET_FLE_BPID(fle, bpid); + DPAA2_SET_FLE_BPID(fle + 1, bpid); + } else { + DPAA2_SET_FD_IVP(fd); + DPAA2_SET_FLE_IVP(fle); + DPAA2_SET_FLE_IVP((fle + 1)); + } + flc = &priv->flc_desc[DESC_INITFINAL].flc; + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sym_op->auth.digest.data)); + fle->length = sym_op->auth.digest.length; + + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(fle)); + DPAA2_SET_FD_COMPOUND_FMT(fd); + fle++; + + if (sess->dir == DIR_ENC) { + DPAA2_SET_FLE_ADDR(fle, + DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(fle, sym_op->auth.data.offset + + sym_op->m_src->data_off); + DPAA2_SET_FD_LEN(fd, sym_op->auth.data.length); + fle->length = sym_op->auth.data.length; + } else { + sge = fle + 2; + DPAA2_SET_FLE_SG_EXT(fle); + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FLE_BPID(sge, bpid); + DPAA2_SET_FLE_BPID(sge + 1, bpid); + } else { + DPAA2_SET_FLE_IVP(sge); + DPAA2_SET_FLE_IVP((sge + 1)); + } + DPAA2_SET_FLE_ADDR(sge, + DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->auth.data.offset + + sym_op->m_src->data_off); + + DPAA2_SET_FD_LEN(fd, sym_op->auth.data.length + + sym_op->auth.digest.length); + sge->length = sym_op->auth.data.length; + sge++; + old_digest = (uint8_t *)(sge + 1); + rte_memcpy(old_digest, sym_op->auth.digest.data, + sym_op->auth.digest.length); + memset(sym_op->auth.digest.data, 0, sym_op->auth.digest.length); + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(old_digest)); + sge->length = sym_op->auth.digest.length; + fle->length = sym_op->auth.data.length + + sym_op->auth.digest.length; + DPAA2_SET_FLE_FIN(sge); + } + DPAA2_SET_FLE_FIN(fle); + + return 0; +} + +static int +build_cipher_fd(dpaa2_sec_session *sess, struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + struct rte_crypto_sym_op *sym_op = op->sym; + struct qbman_fle *fle, *sge; + uint32_t mem_len = (5 * sizeof(struct qbman_fle)); + struct sec_flow_context *flc; + struct ctxt_priv *priv = sess->ctxt; + + PMD_INIT_FUNC_TRACE(); + + /* todo - we can use some mempool to avoid malloc here */ + fle = rte_zmalloc(NULL, mem_len, RTE_CACHE_LINE_SIZE); + if (!fle) { + RTE_LOG(ERR, PMD, "Memory alloc failed for SGE\n"); + return -1; + } + /* TODO we are using the first FLE entry to store Mbuf. + * Currently we donot know which FLE has the mbuf stored. + * So while retreiving we can go back 1 FLE from the FD -ADDR + * to get the MBUF Addr from the previous FLE. + * We can have a better approach to use the inline Mbuf + */ + DPAA2_SET_FLE_ADDR(fle, DPAA2_OP_VADDR_TO_IOVA(op)); + fle = fle + 1; + sge = fle + 2; + + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FD_BPID(fd, bpid); + DPAA2_SET_FLE_BPID(fle, bpid); + DPAA2_SET_FLE_BPID(fle + 1, bpid); + DPAA2_SET_FLE_BPID(sge, bpid); + DPAA2_SET_FLE_BPID(sge + 1, bpid); + } else { + DPAA2_SET_FD_IVP(fd); + DPAA2_SET_FLE_IVP(fle); + DPAA2_SET_FLE_IVP((fle + 1)); + DPAA2_SET_FLE_IVP(sge); + DPAA2_SET_FLE_IVP((sge + 1)); + } + + flc = &priv->flc_desc[0].flc; + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(fle)); + DPAA2_SET_FD_LEN(fd, sym_op->cipher.data.length + + sym_op->cipher.iv.length); + DPAA2_SET_FD_COMPOUND_FMT(fd); + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + PMD_TX_LOG(DEBUG, "cipher_off: 0x%x/length %d,ivlen=%d data_off: 0x%x", + sym_op->cipher.data.offset, + sym_op->cipher.data.length, + sym_op->cipher.iv.length, + sym_op->m_src->data_off); + + DPAA2_SET_FLE_ADDR(fle, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(fle, sym_op->cipher.data.offset + + sym_op->m_src->data_off); + + fle->length = sym_op->cipher.data.length + sym_op->cipher.iv.length; + + PMD_TX_LOG(DEBUG, "1 - flc = %p, fle = %p FLEaddr = %x-%x, length %d", + flc, fle, fle->addr_hi, fle->addr_lo, fle->length); + + fle++; + + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + fle->length = sym_op->cipher.data.length + sym_op->cipher.iv.length; + + DPAA2_SET_FLE_SG_EXT(fle); + + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(sym_op->cipher.iv.data)); + sge->length = sym_op->cipher.iv.length; + + sge++; + DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->cipher.data.offset + + sym_op->m_src->data_off); + + sge->length = sym_op->cipher.data.length; + DPAA2_SET_FLE_FIN(sge); + DPAA2_SET_FLE_FIN(fle); + + PMD_TX_LOG(DEBUG, "fdaddr =%p bpid =%d meta =%d off =%d, len =%d", + (void *)DPAA2_GET_FD_ADDR(fd), + DPAA2_GET_FD_BPID(fd), + bpid_info[bpid].meta_data_size, + DPAA2_GET_FD_OFFSET(fd), + DPAA2_GET_FD_LEN(fd)); + + return 0; +} + +static inline int +build_sec_fd(dpaa2_sec_session *sess, struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + int ret = -1; + + PMD_INIT_FUNC_TRACE(); + + switch (sess->ctxt_type) { + case DPAA2_SEC_CIPHER: + ret = build_cipher_fd(sess, op, fd, bpid); + break; + case DPAA2_SEC_AUTH: + ret = build_auth_fd(sess, op, fd, bpid); + break; + case DPAA2_SEC_CIPHER_HASH: + ret = build_authenc_fd(sess, op, fd, bpid); + break; + case DPAA2_SEC_HASH_CIPHER: + default: + RTE_LOG(ERR, PMD, "error: Unsupported session\n"); + } + return ret; +} + +static uint16_t +dpaa2_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function to transmit the frames to given device and VQ*/ + uint32_t loop; + int32_t ret; + struct qbman_fd fd_arr[MAX_TX_RING_SLOTS]; + uint32_t frames_to_send; + struct qbman_eq_desc eqdesc; + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp; + struct qbman_swp *swp; + uint16_t num_tx = 0; + /*todo - need to support multiple buffer pools */ + uint16_t bpid; + struct rte_mempool *mb_pool; + dpaa2_sec_session *sess; + + if (unlikely(nb_ops == 0)) + return 0; + + if (ops[0]->sym->sess_type != RTE_CRYPTO_SYM_OP_WITH_SESSION) { + RTE_LOG(ERR, PMD, "sessionless crypto op not supported\n"); + return 0; + } + /*Prepare enqueue descriptor*/ + qbman_eq_desc_clear(&eqdesc); + qbman_eq_desc_set_no_orp(&eqdesc, DPAA2_EQ_RESP_ERR_FQ); + qbman_eq_desc_set_response(&eqdesc, 0, 0); + qbman_eq_desc_set_fq(&eqdesc, dpaa2_qp->tx_vq.fqid); + + if (!DPAA2_PER_LCORE_SEC_DPIO) { + ret = dpaa2_affine_qbman_swp_sec(); + if (ret) { + RTE_LOG(ERR, PMD, "Failure in affining portal\n"); + return 0; + } + } + swp = DPAA2_PER_LCORE_SEC_PORTAL; + + while (nb_ops) { + frames_to_send = (nb_ops >> 3) ? MAX_TX_RING_SLOTS : nb_ops; + + for (loop = 0; loop < frames_to_send; loop++) { + /*Clear the unused FD fields before sending*/ + memset(&fd_arr[loop], 0, sizeof(struct qbman_fd)); + sess = (dpaa2_sec_session *) + (*ops)->sym->session->_private; + mb_pool = (*ops)->sym->m_src->pool; + bpid = mempool_to_bpid(mb_pool); + ret = build_sec_fd(sess, *ops, &fd_arr[loop], bpid); + if (ret) { + PMD_DRV_LOG(ERR, "error: Improper packet" + " contents for crypto operation\n"); + goto skip_tx; + } + ops++; + } + loop = 0; + while (loop < frames_to_send) { + loop += qbman_swp_send_multiple(swp, &eqdesc, + &fd_arr[loop], + frames_to_send - loop); + } + + num_tx += frames_to_send; + nb_ops -= frames_to_send; + } +skip_tx: + dpaa2_qp->tx_vq.tx_pkts += num_tx; + dpaa2_qp->tx_vq.err_pkts += nb_ops; + return num_tx; +} + +static inline struct rte_crypto_op * +sec_fd_to_mbuf(const struct qbman_fd *fd) +{ + struct qbman_fle *fle; + struct rte_crypto_op *op; + + fle = (struct qbman_fle *)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd)); + + PMD_RX_LOG(DEBUG, "FLE addr = %x - %x, offset = %x", + fle->addr_hi, fle->addr_lo, fle->fin_bpid_offset); + + /* TODO we are using the first FLE entry to store Mbuf. + * Currently we donot know which FLE has the mbuf stored. + * So while retreiving we can go back 1 FLE from the FD -ADDR + * to get the MBUF Addr from the previous FLE. + * We can have a better approach to use the inline Mbuf + */ + + if (unlikely(DPAA2_GET_FD_IVP(fd))) { + /* TODO complete it. */ + RTE_LOG(ERR, PMD, "error: Non inline buffer - WHAT to DO?"); + return NULL; + } + op = (struct rte_crypto_op *)DPAA2_IOVA_TO_VADDR( + DPAA2_GET_FLE_ADDR((fle - 1))); + + /* Prefeth op */ + rte_prefetch0(op->sym->m_src); + + PMD_RX_LOG(DEBUG, "mbuf %p BMAN buf addr %p", + (void *)op->sym->m_src, op->sym->m_src->buf_addr); + + PMD_RX_LOG(DEBUG, "fdaddr =%p bpid =%d meta =%d off =%d, len =%d", + (void *)DPAA2_GET_FD_ADDR(fd), + DPAA2_GET_FD_BPID(fd), + bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size, + DPAA2_GET_FD_OFFSET(fd), + DPAA2_GET_FD_LEN(fd)); + + /* free the fle memory */ + rte_free(fle - 1); + + return op; +} + +static uint16_t +dpaa2_sec_dequeue_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function is responsible to receive frames for a given device and VQ*/ + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp; + struct qbman_result *dq_storage; + uint32_t fqid = dpaa2_qp->rx_vq.fqid; + int ret, num_rx = 0; + uint8_t is_last = 0, status; + struct qbman_swp *swp; + const struct qbman_fd *fd; + struct qbman_pull_desc pulldesc; + + if (!DPAA2_PER_LCORE_SEC_DPIO) { + ret = dpaa2_affine_qbman_swp_sec(); + if (ret) { + RTE_LOG(ERR, PMD, "Failure in affining portal\n"); + return 0; + } + } + swp = DPAA2_PER_LCORE_SEC_PORTAL; + dq_storage = dpaa2_qp->rx_vq.q_storage->dq_storage[0]; + + qbman_pull_desc_clear(&pulldesc); + qbman_pull_desc_set_numframes(&pulldesc, + (nb_ops > DPAA2_DQRR_RING_SIZE) ? + DPAA2_DQRR_RING_SIZE : nb_ops); + qbman_pull_desc_set_fq(&pulldesc, fqid); + qbman_pull_desc_set_storage(&pulldesc, dq_storage, + (dma_addr_t)DPAA2_VADDR_TO_IOVA(dq_storage), + 1); + + /*Issue a volatile dequeue command. */ + while (1) { + if (qbman_swp_pull(swp, &pulldesc)) { + RTE_LOG(WARNING, PMD, "SEC VDQ command is not issued." + "QBMAN is busy\n"); + /* Portal was busy, try again */ + continue; + } + break; + }; + + /* Receive the packets till Last Dequeue entry is found with + * respect to the above issues PULL command. + */ + while (!is_last) { + /* Check if the previous issued command is completed. + * Also seems like the SWP is shared between the Ethernet Driver + * and the SEC driver. + */ + while (!qbman_check_command_complete(swp, dq_storage)) + ; + + /* Loop until the dq_storage is updated with + * new token by QBMAN + */ + while (!qbman_result_has_new_result(swp, dq_storage)) + ; + /* Check whether Last Pull command is Expired and + * setting Condition for Loop termination + */ + if (qbman_result_DQ_is_pull_complete(dq_storage)) { + is_last = 1; + /* Check for valid frame. */ + status = (uint8_t)qbman_result_DQ_flags(dq_storage); + if ((status & QBMAN_DQ_STAT_VALIDFRAME) == 0) { + PMD_RX_LOG(DEBUG, "No frame is delivered"); + continue; + } + } + + fd = qbman_result_DQ_fd(dq_storage); + ops[num_rx] = sec_fd_to_mbuf(fd); + + if (unlikely(fd->simple.frc)) { + /* TODO Parse SEC errors */ + RTE_LOG(ERR, PMD, "SEC returned Error - %x\n", + fd->simple.frc); + ops[num_rx]->status = RTE_CRYPTO_OP_STATUS_ERROR; + } else { + ops[num_rx]->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + } + + num_rx++; + dq_storage++; + } /* End of Packet Rx loop */ + + dpaa2_qp->rx_vq.rx_pkts += num_rx; + + PMD_RX_LOG(DEBUG, "SEC Received %d Packets", num_rx); + /*Return the total number of packets received to DPAA2 app*/ + return num_rx; +} +/** Release queue pair */ +static int +dpaa2_sec_queue_pair_release(struct rte_cryptodev *dev, uint16_t queue_pair_id) +{ + struct dpaa2_sec_qp *qp = + (struct dpaa2_sec_qp *)dev->data->queue_pairs[queue_pair_id]; + + PMD_INIT_FUNC_TRACE(); + + if (qp->rx_vq.q_storage) { + dpaa2_free_dq_storage(qp->rx_vq.q_storage); + rte_free(qp->rx_vq.q_storage); + } + rte_free(qp); + + dev->data->queue_pairs[queue_pair_id] = NULL; + + return 0; +} + +/** Setup a queue pair */ +static int +dpaa2_sec_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, + __rte_unused const struct rte_cryptodev_qp_conf *qp_conf, + __rte_unused int socket_id) +{ + struct dpaa2_sec_dev_private *priv = dev->data->dev_private; + struct dpaa2_sec_qp *qp; + struct fsl_mc_io *dpseci = (struct fsl_mc_io *)priv->hw; + struct dpseci_rx_queue_cfg cfg; + int32_t retcode; + + PMD_INIT_FUNC_TRACE(); + + /* If qp is already in use free ring memory and qp metadata. */ + if (dev->data->queue_pairs[qp_id] != NULL) { + PMD_DRV_LOG(INFO, "QP already setup"); + return 0; + } + + PMD_DRV_LOG(DEBUG, "dev =%p, queue =%d, conf =%p", + dev, qp_id, qp_conf); + + memset(&cfg, 0, sizeof(struct dpseci_rx_queue_cfg)); + + qp = rte_malloc(NULL, sizeof(struct dpaa2_sec_qp), + RTE_CACHE_LINE_SIZE); + if (!qp) { + RTE_LOG(ERR, PMD, "malloc failed for rx/tx queues\n"); + return -1; + } + + qp->rx_vq.dev = dev; + qp->tx_vq.dev = dev; + qp->rx_vq.q_storage = rte_malloc("sec dq storage", + sizeof(struct queue_storage_info_t), + RTE_CACHE_LINE_SIZE); + if (!qp->rx_vq.q_storage) { + RTE_LOG(ERR, PMD, "malloc failed for q_storage\n"); + return -1; + } + memset(qp->rx_vq.q_storage, 0, sizeof(struct queue_storage_info_t)); + + if (dpaa2_alloc_dq_storage(qp->rx_vq.q_storage)) { + RTE_LOG(ERR, PMD, "dpaa2_alloc_dq_storage failed\n"); + return -1; + } + + dev->data->queue_pairs[qp_id] = qp; + + cfg.options = cfg.options | DPSECI_QUEUE_OPT_USER_CTX; + cfg.user_ctx = (uint64_t)(&qp->rx_vq); + retcode = dpseci_set_rx_queue(dpseci, CMD_PRI_LOW, priv->token, + qp_id, &cfg); + return retcode; +} + +/** Start queue pair */ +static int +dpaa2_sec_queue_pair_start(__rte_unused struct rte_cryptodev *dev, + __rte_unused uint16_t queue_pair_id) +{ + PMD_INIT_FUNC_TRACE(); + + return 0; +} + +/** Stop queue pair */ +static int +dpaa2_sec_queue_pair_stop(__rte_unused struct rte_cryptodev *dev, + __rte_unused uint16_t queue_pair_id) +{ + PMD_INIT_FUNC_TRACE(); + + return 0; +} + +/** Return the number of allocated queue pairs */ +static uint32_t +dpaa2_sec_queue_pair_count(struct rte_cryptodev *dev) +{ + PMD_INIT_FUNC_TRACE(); + + return dev->data->nb_queue_pairs; +} + +/** Returns the size of the aesni gcm session structure */ +static unsigned int +dpaa2_sec_session_get_size(struct rte_cryptodev *dev __rte_unused) +{ + PMD_INIT_FUNC_TRACE(); + + return sizeof(dpaa2_sec_session); +} + +static void +dpaa2_sec_session_initialize(struct rte_mempool *mp __rte_unused, + void *sess __rte_unused) +{ + PMD_INIT_FUNC_TRACE(); +} + +static int +dpaa2_sec_cipher_init(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, + dpaa2_sec_session *session) +{ + struct dpaa2_sec_cipher_ctxt *ctxt = &session->ext_params.cipher_ctxt; + struct alginfo cipherdata; + unsigned int bufsize, i; + struct ctxt_priv *priv; + struct sec_flow_context *flc; + + PMD_INIT_FUNC_TRACE(); + + /* For SEC CIPHER only one descriptor is required. */ + priv = (struct ctxt_priv *)rte_zmalloc(NULL, + sizeof(struct ctxt_priv) + sizeof(struct sec_flc_desc), + RTE_CACHE_LINE_SIZE); + if (priv == NULL) { + RTE_LOG(ERR, PMD, "No Memory for priv CTXT"); + return -1; + } + + flc = &priv->flc_desc[0].flc; + + session->cipher_key.data = rte_zmalloc(NULL, xform->cipher.key.length, + RTE_CACHE_LINE_SIZE); + if (session->cipher_key.data == NULL) { + RTE_LOG(ERR, PMD, "No Memory for cipher key"); + return -1; + } + session->cipher_key.length = xform->cipher.key.length; + + memcpy(session->cipher_key.data, xform->cipher.key.data, + xform->cipher.key.length); + cipherdata.key = (uint64_t)session->cipher_key.data; + cipherdata.keylen = session->cipher_key.length; + cipherdata.key_enc_flags = 0; + cipherdata.key_type = RTA_DATA_IMM; + + switch (xform->cipher.algo) { + case RTE_CRYPTO_CIPHER_AES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_AES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_AES_CBC; + ctxt->iv.length = AES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_3DES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_3DES_CBC; + ctxt->iv.length = TDES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + case RTE_CRYPTO_CIPHER_3DES_CTR: + case RTE_CRYPTO_CIPHER_AES_GCM: + case RTE_CRYPTO_CIPHER_AES_CCM: + case RTE_CRYPTO_CIPHER_AES_ECB: + case RTE_CRYPTO_CIPHER_3DES_ECB: + case RTE_CRYPTO_CIPHER_AES_XTS: + case RTE_CRYPTO_CIPHER_AES_F8: + case RTE_CRYPTO_CIPHER_ARC4: + case RTE_CRYPTO_CIPHER_KASUMI_F8: + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + case RTE_CRYPTO_CIPHER_ZUC_EEA3: + case RTE_CRYPTO_CIPHER_NULL: + RTE_LOG(ERR, PMD, "Crypto: Unsupported Cipher alg %u", + xform->cipher.algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Cipher specified %u\n", + xform->cipher.algo); + goto error_out; + } + session->dir = (xform->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DIR_ENC : DIR_DEC; + + bufsize = cnstr_shdsc_blkcipher(priv->flc_desc[0].desc, 1, 0, + &cipherdata, NULL, ctxt->iv.length, + session->dir); + flc->dhr = 0; + flc->bpv0 = 0x1; + flc->mode_bits = 0x8000; + + flc->word1_sdl = (uint8_t)bufsize; + flc->word2_rflc_31_0 = lower_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + flc->word3_rflc_63_32 = upper_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + session->ctxt = priv; + + for (i = 0; i < bufsize; i++) + PMD_DRV_LOG(DEBUG, "DESC[%d]:0x%x\n", + i, priv->flc_desc[0].desc[i]); + + return 0; + +error_out: + rte_free(session->cipher_key.data); + return -1; +} + +static int +dpaa2_sec_auth_init(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, + dpaa2_sec_session *session) +{ + struct dpaa2_sec_auth_ctxt *ctxt = &session->ext_params.auth_ctxt; + struct alginfo authdata; + unsigned int bufsize; + struct ctxt_priv *priv; + struct sec_flow_context *flc; + + PMD_INIT_FUNC_TRACE(); + + /* For SEC AUTH three descriptors are required for various stages */ + priv = (struct ctxt_priv *)rte_zmalloc(NULL, + sizeof(struct ctxt_priv) + 3 * + sizeof(struct sec_flc_desc), + RTE_CACHE_LINE_SIZE); + if (priv == NULL) { + RTE_LOG(ERR, PMD, "No Memory for priv CTXT"); + return -1; + } + + flc = &priv->flc_desc[DESC_INITFINAL].flc; + + session->auth_key.data = rte_zmalloc(NULL, xform->auth.key.length, + RTE_CACHE_LINE_SIZE); + session->auth_key.length = xform->auth.key.length; + + memcpy(session->auth_key.data, xform->auth.key.data, + xform->auth.key.length); + authdata.key = (uint64_t)session->auth_key.data; + authdata.keylen = session->auth_key.length; + authdata.key_enc_flags = 0; + authdata.key_type = RTA_DATA_IMM; + + switch (xform->auth.algo) { + case RTE_CRYPTO_AUTH_SHA1_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA1; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA1_HMAC; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + authdata.algtype = OP_ALG_ALGSEL_MD5; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_MD5_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA256; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA256_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA384; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA384_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA512; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA512_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA224; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA224_HMAC; + break; + case RTE_CRYPTO_AUTH_AES_XCBC_MAC: + case RTE_CRYPTO_AUTH_AES_GCM: + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + case RTE_CRYPTO_AUTH_NULL: + case RTE_CRYPTO_AUTH_SHA1: + case RTE_CRYPTO_AUTH_SHA256: + case RTE_CRYPTO_AUTH_SHA512: + case RTE_CRYPTO_AUTH_SHA224: + case RTE_CRYPTO_AUTH_SHA384: + case RTE_CRYPTO_AUTH_MD5: + case RTE_CRYPTO_AUTH_AES_CCM: + case RTE_CRYPTO_AUTH_AES_GMAC: + case RTE_CRYPTO_AUTH_KASUMI_F9: + case RTE_CRYPTO_AUTH_AES_CMAC: + case RTE_CRYPTO_AUTH_AES_CBC_MAC: + case RTE_CRYPTO_AUTH_ZUC_EIA3: + RTE_LOG(ERR, PMD, "Crypto: Unsupported auth alg %u", + xform->auth.algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Auth specified %u\n", + xform->auth.algo); + goto error_out; + } + session->dir = (xform->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE) ? + DIR_ENC : DIR_DEC; + + bufsize = cnstr_shdsc_hmac(priv->flc_desc[DESC_INITFINAL].desc, + 1, 0, &authdata, !session->dir, + ctxt->trunc_len); + + flc->word1_sdl = (uint8_t)bufsize; + flc->word2_rflc_31_0 = lower_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + flc->word3_rflc_63_32 = upper_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + session->ctxt = priv; + + return 0; + +error_out: + rte_free(session->auth_key.data); + return -1; +} + +static int +dpaa2_sec_aead_init(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, + dpaa2_sec_session *session) +{ + struct dpaa2_sec_aead_ctxt *ctxt = &session->ext_params.aead_ctxt; + struct alginfo authdata, cipherdata; + unsigned int bufsize; + struct ctxt_priv *priv; + struct sec_flow_context *flc; + struct rte_crypto_cipher_xform *cipher_xform; + struct rte_crypto_auth_xform *auth_xform; + int err; + + PMD_INIT_FUNC_TRACE(); + + if (session->ext_params.aead_ctxt.auth_cipher_text) { + cipher_xform = &xform->cipher; + auth_xform = &xform->next->auth; + session->ctxt_type = + (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DPAA2_SEC_CIPHER_HASH : DPAA2_SEC_HASH_CIPHER; + } else { + cipher_xform = &xform->next->cipher; + auth_xform = &xform->auth; + session->ctxt_type = + (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DPAA2_SEC_HASH_CIPHER : DPAA2_SEC_CIPHER_HASH; + } + /* For SEC AEAD only one descriptor is required */ + priv = (struct ctxt_priv *)rte_zmalloc(NULL, + sizeof(struct ctxt_priv) + sizeof(struct sec_flc_desc), + RTE_CACHE_LINE_SIZE); + if (priv == NULL) { + RTE_LOG(ERR, PMD, "No Memory for priv CTXT"); + return -1; + } + + flc = &priv->flc_desc[0].flc; + + session->cipher_key.data = rte_zmalloc(NULL, cipher_xform->key.length, + RTE_CACHE_LINE_SIZE); + if (session->cipher_key.data == NULL) { + RTE_LOG(ERR, PMD, "No Memory for cipher key"); + return -1; + } + session->cipher_key.length = cipher_xform->key.length; + session->auth_key.data = rte_zmalloc(NULL, auth_xform->key.length, + RTE_CACHE_LINE_SIZE); + if (session->auth_key.data == NULL) { + RTE_LOG(ERR, PMD, "No Memory for auth key"); + goto error_out; + } + session->auth_key.length = auth_xform->key.length; + memcpy(session->cipher_key.data, cipher_xform->key.data, + cipher_xform->key.length); + memcpy(session->auth_key.data, auth_xform->key.data, + auth_xform->key.length); + + ctxt->trunc_len = auth_xform->digest_length; + authdata.key = (uint64_t)session->auth_key.data; + authdata.keylen = session->auth_key.length; + authdata.key_enc_flags = 0; + authdata.key_type = RTA_DATA_IMM; + + switch (auth_xform->algo) { + case RTE_CRYPTO_AUTH_SHA1_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA1; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA1_HMAC; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + authdata.algtype = OP_ALG_ALGSEL_MD5; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_MD5_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA224; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA224_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA256; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA256_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA384; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA384_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA512; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA512_HMAC; + break; + case RTE_CRYPTO_AUTH_AES_XCBC_MAC: + case RTE_CRYPTO_AUTH_AES_GCM: + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + case RTE_CRYPTO_AUTH_NULL: + case RTE_CRYPTO_AUTH_SHA1: + case RTE_CRYPTO_AUTH_SHA256: + case RTE_CRYPTO_AUTH_SHA512: + case RTE_CRYPTO_AUTH_SHA224: + case RTE_CRYPTO_AUTH_SHA384: + case RTE_CRYPTO_AUTH_MD5: + case RTE_CRYPTO_AUTH_AES_CCM: + case RTE_CRYPTO_AUTH_AES_GMAC: + case RTE_CRYPTO_AUTH_KASUMI_F9: + case RTE_CRYPTO_AUTH_AES_CMAC: + case RTE_CRYPTO_AUTH_AES_CBC_MAC: + case RTE_CRYPTO_AUTH_ZUC_EIA3: + RTE_LOG(ERR, PMD, "Crypto: Unsupported auth alg %u", + auth_xform->algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Auth specified %u\n", + auth_xform->algo); + goto error_out; + } + cipherdata.key = (uint64_t)session->cipher_key.data; + cipherdata.keylen = session->cipher_key.length; + cipherdata.key_enc_flags = 0; + cipherdata.key_type = RTA_DATA_IMM; + + switch (cipher_xform->algo) { + case RTE_CRYPTO_CIPHER_AES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_AES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_AES_CBC; + ctxt->iv.length = AES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_3DES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_3DES_CBC; + ctxt->iv.length = TDES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_AES_GCM: + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + case RTE_CRYPTO_CIPHER_NULL: + case RTE_CRYPTO_CIPHER_3DES_ECB: + case RTE_CRYPTO_CIPHER_AES_ECB: + case RTE_CRYPTO_CIPHER_AES_CTR: + case RTE_CRYPTO_CIPHER_AES_CCM: + case RTE_CRYPTO_CIPHER_KASUMI_F8: + RTE_LOG(ERR, PMD, "Crypto: Unsupported Cipher alg %u", + cipher_xform->algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Cipher specified %u\n", + cipher_xform->algo); + goto error_out; + } + session->dir = (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DIR_ENC : DIR_DEC; + + priv->flc_desc[0].desc[0] = cipherdata.keylen; + priv->flc_desc[0].desc[1] = authdata.keylen; + err = rta_inline_query(IPSEC_AUTH_VAR_AES_DEC_BASE_DESC_LEN, + MIN_JOB_DESC_SIZE, + (unsigned int *)priv->flc_desc[0].desc, + &priv->flc_desc[0].desc[2], 2); + + if (err < 0) { + PMD_DRV_LOG(ERR, "Crypto: Incorrect key lengths"); + goto error_out; + } + if (priv->flc_desc[0].desc[2] & 1) + cipherdata.key_type = RTA_DATA_IMM; + else { + cipherdata.key = DPAA2_VADDR_TO_IOVA(cipherdata.key); + cipherdata.key_type = RTA_DATA_PTR; + } + if (priv->flc_desc[0].desc[2] & (1<<1)) + authdata.key_type = RTA_DATA_IMM; + else { + authdata.key = DPAA2_VADDR_TO_IOVA(authdata.key); + authdata.key_type = RTA_DATA_PTR; + } + priv->flc_desc[0].desc[0] = 0; + priv->flc_desc[0].desc[1] = 0; + priv->flc_desc[0].desc[2] = 0; + + if (session->ctxt_type == DPAA2_SEC_CIPHER_HASH) { + bufsize = cnstr_shdsc_authenc(priv->flc_desc[0].desc, 1, + 0, &cipherdata, &authdata, + ctxt->iv.length, + ctxt->auth_only_len, + ctxt->trunc_len, + session->dir); + } else { + RTE_LOG(ERR, PMD, "Hash before cipher not supported"); + goto error_out; + } + + flc->word1_sdl = (uint8_t)bufsize; + flc->word2_rflc_31_0 = lower_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + flc->word3_rflc_63_32 = upper_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + session->ctxt = priv; + + return 0; + +error_out: + rte_free(session->cipher_key.data); + rte_free(session->auth_key.data); + return -1; +} + +static void * +dpaa2_sec_session_configure(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, void *sess) +{ + dpaa2_sec_session *session = sess; + + PMD_INIT_FUNC_TRACE(); + + if (unlikely(sess == NULL)) { + RTE_LOG(ERR, PMD, "invalid session struct"); + return NULL; + } + /* Cipher Only */ + if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL) { + session->ctxt_type = DPAA2_SEC_CIPHER; + dpaa2_sec_cipher_init(dev, xform, session); + + /* Authentication Only */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && + xform->next == NULL) { + session->ctxt_type = DPAA2_SEC_AUTH; + dpaa2_sec_auth_init(dev, xform, session); + + /* Cipher then Authenticate */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && + xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + session->ext_params.aead_ctxt.auth_cipher_text = true; + dpaa2_sec_aead_init(dev, xform, session); + + /* Authenticate then Cipher */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && + xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + session->ext_params.aead_ctxt.auth_cipher_text = false; + dpaa2_sec_aead_init(dev, xform, session); + } else { + RTE_LOG(ERR, PMD, "Invalid crypto type"); + return NULL; + } + + return session; +} + +/** Clear the memory of session so it doesn't leave key material behind */ +static void +dpaa2_sec_session_clear(struct rte_cryptodev *dev __rte_unused, void *sess) +{ + PMD_INIT_FUNC_TRACE(); + + if (sess) + memset(sess, 0, sizeof(dpaa2_sec_session)); +} static int dpaa2_sec_dev_configure(struct rte_cryptodev *dev __rte_unused) @@ -194,6 +1393,15 @@ static struct rte_cryptodev_ops crypto_ops = { .dev_stop = dpaa2_sec_dev_stop, .dev_close = dpaa2_sec_dev_close, .dev_infos_get = dpaa2_sec_dev_infos_get, + .queue_pair_setup = dpaa2_sec_queue_pair_setup, + .queue_pair_release = dpaa2_sec_queue_pair_release, + .queue_pair_start = dpaa2_sec_queue_pair_start, + .queue_pair_stop = dpaa2_sec_queue_pair_stop, + .queue_pair_count = dpaa2_sec_queue_pair_count, + .session_get_size = dpaa2_sec_session_get_size, + .session_initialize = dpaa2_sec_session_initialize, + .session_configure = dpaa2_sec_session_configure, + .session_clear = dpaa2_sec_session_clear, }; static int @@ -232,6 +1440,8 @@ dpaa2_sec_dev_init(struct rte_cryptodev *cryptodev) cryptodev->dev_type = RTE_CRYPTODEV_DPAA2_SEC_PMD; cryptodev->dev_ops = &crypto_ops; + cryptodev->enqueue_burst = dpaa2_sec_enqueue_burst; + cryptodev->dequeue_burst = dpaa2_sec_dequeue_burst; cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING; diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h index e0d6148..f2529fe 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h @@ -34,6 +34,8 @@ #ifndef _RTE_DPAA2_SEC_PMD_PRIVATE_H_ #define _RTE_DPAA2_SEC_PMD_PRIVATE_H_ +#define MAX_QUEUES 64 +#define MAX_DESC_SIZE 64 /** private data structure for each DPAA2_SEC device */ struct dpaa2_sec_dev_private { void *mc_portal; /**< MC Portal for configuring this device */ @@ -52,6 +54,147 @@ struct dpaa2_sec_qp { struct dpaa2_queue tx_vq; }; +enum shr_desc_type { + DESC_UPDATE, + DESC_FINAL, + DESC_INITFINAL, +}; + +#define DIR_ENC 1 +#define DIR_DEC 0 + +/* SEC Flow Context Descriptor */ +struct sec_flow_context { + /* word 0 */ + uint16_t word0_sdid; /* 11-0 SDID */ + uint16_t word0_res; /* 31-12 reserved */ + + /* word 1 */ + uint8_t word1_sdl; /* 5-0 SDL */ + /* 7-6 reserved */ + + uint8_t word1_bits_15_8; /* 11-8 CRID */ + /* 14-12 reserved */ + /* 15 CRJD */ + + uint8_t word1_bits23_16; /* 16 EWS */ + /* 17 DAC */ + /* 18,19,20 ? */ + /* 23-21 reserved */ + + uint8_t word1_bits31_24; /* 24 RSC */ + /* 25 RBMT */ + /* 31-26 reserved */ + + /* word 2 RFLC[31-0] */ + uint32_t word2_rflc_31_0; + + /* word 3 RFLC[63-32] */ + uint32_t word3_rflc_63_32; + + /* word 4 */ + uint16_t word4_iicid; /* 15-0 IICID */ + uint16_t word4_oicid; /* 31-16 OICID */ + + /* word 5 */ + uint32_t word5_ofqid:24; /* 23-0 OFQID */ + uint32_t word5_31_24:8; + /* 24 OSC */ + /* 25 OBMT */ + /* 29-26 reserved */ + /* 31-30 ICR */ + + /* word 6 */ + uint32_t word6_oflc_31_0; + + /* word 7 */ + uint32_t word7_oflc_63_32; + + /* Word 8-15 storage profiles */ + uint16_t dl; /**< DataLength(correction) */ + uint16_t reserved; /**< reserved */ + uint16_t dhr; /**< DataHeadRoom(correction) */ + uint16_t mode_bits; /**< mode bits */ + uint16_t bpv0; /**< buffer pool0 valid */ + uint16_t bpid0; /**< Bypass Memory Translation */ + uint16_t bpv1; /**< buffer pool1 valid */ + uint16_t bpid1; /**< Bypass Memory Translation */ + uint64_t word_12_15[2]; /**< word 12-15 are reserved */ +}; + +struct sec_flc_desc { + struct sec_flow_context flc; + uint32_t desc[MAX_DESC_SIZE]; +}; + +struct ctxt_priv { + struct sec_flc_desc flc_desc[0]; +}; + +enum dpaa2_sec_op_type { + DPAA2_SEC_NONE, /*!< No Cipher operations*/ + DPAA2_SEC_CIPHER,/*!< CIPHER operations */ + DPAA2_SEC_AUTH, /*!< Authentication Operations */ + DPAA2_SEC_CIPHER_HASH, /*!< Authenticated Encryption with + * associated data + */ + DPAA2_SEC_HASH_CIPHER, /*!< Encryption with Authenticated + * associated data + */ + DPAA2_SEC_IPSEC, /*!< IPSEC protocol operations*/ + DPAA2_SEC_PDCP, /*!< PDCP protocol operations*/ + DPAA2_SEC_PKC, /*!< Public Key Cryptographic Operations */ + DPAA2_SEC_MAX +}; + +struct dpaa2_sec_cipher_ctxt { + struct { + uint8_t *data; + uint16_t length; + } iv; /**< Initialisation vector parameters */ + uint8_t *init_counter; /*!< Set initial counter for CTR mode */ +}; + +struct dpaa2_sec_auth_ctxt { + uint8_t trunc_len; /*!< Length for output ICV, should + * be 0 if no truncation required + */ +}; + +struct dpaa2_sec_aead_ctxt { + struct { + uint8_t *data; + uint16_t length; + } iv; /**< Initialisation vector parameters */ + uint16_t auth_only_len; /*!< Length of data for Auth only */ + uint8_t auth_cipher_text; /**< Authenticate/cipher ordering */ + uint8_t trunc_len; /*!< Length for output ICV, should + * be 0 if no truncation required + */ +}; + +typedef struct dpaa2_sec_session_entry { + void *ctxt; + uint8_t ctxt_type; + uint8_t dir; /*!< Operation Direction */ + enum rte_crypto_cipher_algorithm cipher_alg; /*!< Cipher Algorithm*/ + enum rte_crypto_auth_algorithm auth_alg; /*!< Authentication Algorithm*/ + struct { + uint8_t *data; /**< pointer to key data */ + size_t length; /**< key length in bytes */ + } cipher_key; + struct { + uint8_t *data; /**< pointer to key data */ + size_t length; /**< key length in bytes */ + } auth_key; + uint8_t status; + union { + struct dpaa2_sec_cipher_ctxt cipher_ctxt; + struct dpaa2_sec_auth_ctxt auth_ctxt; + struct dpaa2_sec_aead_ctxt aead_ctxt; + } ext_params; +} dpaa2_sec_session; + static const struct rte_cryptodev_capabilities dpaa2_sec_capabilities[] = { { /* MD5 HMAC */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,