From patchwork Fri Mar 3 19:36:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 21306 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 8BD97F614; Fri, 3 Mar 2017 15:12:16 +0100 (CET) Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0046.outbound.protection.outlook.com [104.47.37.46]) by dpdk.org (Postfix) with ESMTP id B8DB369C3 for ; Fri, 3 Mar 2017 15:11:38 +0100 (CET) Received: from DM2PR03CA0049.namprd03.prod.outlook.com (10.141.96.48) by MWHPR03MB2957.namprd03.prod.outlook.com (10.175.136.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.947.12; Fri, 3 Mar 2017 14:11:37 +0000 Received: from BL2FFO11FD018.protection.gbl (2a01:111:f400:7c09::141) by DM2PR03CA0049.outlook.office365.com (2a01:111:e400:2428::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.947.12 via Frontend Transport; Fri, 3 Mar 2017 14:11:37 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; nxp.com; dkim=none (message not signed) header.d=none;nxp.com; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BL2FFO11FD018.mail.protection.outlook.com (10.173.161.36) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.933.11 via Frontend Transport; Fri, 3 Mar 2017 14:11:36 +0000 Received: from netperf2.ap.freescale.net ([10.232.133.164]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id v23EB3la010625; Fri, 3 Mar 2017 07:11:33 -0700 From: Akhil Goyal To: CC: , , , , , Akhil Goyal , Hemant Agrawal Date: Sat, 4 Mar 2017 01:06:39 +0530 Message-ID: <20170303193648.30665-10-akhil.goyal@nxp.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170303193648.30665-1-akhil.goyal@nxp.com> References: <20170120140509.4495-1-akhil.goyal@nxp.com> <20170303193648.30665-1-akhil.goyal@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131330238966234897; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(336005)(7916002)(39840400002)(39860400002)(39400400002)(39850400002)(39450400003)(39410400002)(39380400002)(2980300002)(1109001)(1110001)(339900001)(199003)(189002)(9170700003)(33646002)(5660300001)(48376002)(8656002)(8676002)(2950100002)(50226002)(85426001)(2906002)(305945005)(2351001)(50466002)(6916009)(92566002)(106466001)(8936002)(104016004)(50986999)(76176999)(53936002)(54906002)(1076002)(38730400002)(53946003)(189998001)(110136004)(47776003)(36756003)(77096006)(81166006)(105606002)(356003)(626004)(86362001)(575784001)(4326008)(5003940100001)(579004); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR03MB2957; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; MLV:ovrnspm; A:1; MX:1; PTR:InfoDomainNonexistent; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD018; 1:pAluIZ937zS9MutC/3tfFGc/n7iQfWTOow+ehJwQi6kW4evga5j0EiXkk/0rkflV164pPjQAgKFpXHlIESvVG09bs8WnEsyK9iE9Ie0YEZjna8hRZq1HXMams4ZlclbdOtHII3ZaK12pMyYS31/1XxLpPyth6tx15JBdkD/Gp5Rv/jpI/7Yn1/GR281W1MIithiiEhBxds5yRaTv6Jc0ZWXJ1kj9W2k4M4KIhAh5h4N/ihPM/oCG43RSr/40zAKl6tU/nW2tDCzMs8rvhyyJpbKMmq4CDpSGa7dPuKSZ0CXTnRxE7IcPBkVVwELvZ2bsHtqdkO654OX0GboRzSEpXnyvdz0y3bbwYF1hfh8wu3PGdNTcOhixdFJC1g9+eCTc4O03t4hTqefJe4W+CqzWMJBpqFtxwSQ2B0uyWWqbns6b1VdQrttUxwydc3AmXcj9dWNB/SXrNoXmtgq3EHVPFyNEt8phCobH1zSkDsof4iTzmBHhlQIQ32wQgiUzzZR4QbsYpvLWbQ6WpqGkjFYfwEXQ+W2OyBEb5n+oQ+HApfk1Z5LQLfr4889Y0IEvHkIxipcPHMLk22TdJlZdRjQR6HxSFJdTx6UJRB5VYJ1LP9hJlEhcgf5i3CzFLETvKA6f3e3//4eG4aJO+kzvHCvFE180v5HjAAGY8c+jdUI1JBGjp9hSUEXIl2pQ0rDWEk+nii+vpHrw8rRZQ0dsro0NoPA+zLiRBeqNBk0b0HcpwzqLjqoMecNjNxdhLwqZwMaz MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 8cdaa080-7388-4e6a-1db0-08d4623f344e X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:MWHPR03MB2957; X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2957; 3:kVHxYAVrVWq0Ak9HbtbT1XJH6tVXYqtIHkVCka8/VeSFHvoA/B0MN+dJKhJqqrA8NKbBZiFi9o6PRkqr8wZ91TZNPXuU00wO4B0lVYIPxbfifhcJ/D5ZITg9UVumyTG/UlF5gUxU/geurr69iqmcBT6AkSbyF0cylDnyvSw1uViyQCIJWcRWFKTEhqtxtvvKLscfIdEOm5ywMvKWyDpx/xQcoEtLPQ+EpNPg1C7QKPFLOMBYR2xhtyTfbvZUV5eI9Vgj8+zyLI1NAPxYpz1Ljxlxfjbwec2aVuPyaVK4R/cvO7n794ICwC0+rPeRa4w9CQ+3xFyEuPOiXfMckEolqLcQecpMkD2NZWFYWANU8OafLwTk7g7Kb2CcHo+AiTjF; 25:rXTCuKlOcyukTiw6/EG9D4pttfWNnyxWPTFu8m/g6huW90N9th+47W9cbywNWDh3BCVFgbQyFHZ8EdFvFmaHRTwhGqCgQiZfijEz+73HH9sEvb84yUc7yAVTBiAx4BR1hAhPuio0Zbn4JNURhohjFov0cpuVMkLukRmnMUUOMe/rawpuzvkzeXvdx83gKe3PPZM7qi6w+fgpwQ+lIkxQJhnleTrqV7nt3bhrMfPOhIBGizMoyk9OsdKLsk4HGjGCQ9+0aqpl5h8cnc8I72NzaYkeUVfYJdjk8s+HC02zkCefsYoIa75Pk55YoeajgKma3MDuQrD1E+PHzySPaqaxoEXYHNLSPyXf5FwUyMwnNL5fGdSL1D1xSd0fqK4Xg1UE+HwYJ8OPs5yEiEym5vivx8HMd3IruK+1mE3n7TCj3xNjcBLcchTvUaH8jXJTG2QjBn3sVe32/TWIJDDxCfNRCg== X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2957; 31:IjuXTBKycPEir3kt+MirHXBAUivFcEc3eiIOpd57zetOM7SYMa1HVNsM+IYYYxvnbWWS1/XNcPRdhtPpBDYoj/eamXLwsQGUu87LFMuiNgVGKFTmQizA35QbSnTr/XQe9FX90bBlyvqn1V+54+J9kunPIMNKP3OrTEoQprDSwVSTQLNxS83CtZdu38/MLLgrdIW7R5hX/VAzAnFQQ4CRSOcC5HJpAJtr50jjoUiPiFy1NKnM1XaB9bqoYm0KVHmazqAFS8tCNW5q7vj5DRO20A== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(275809806118684); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095060)(601004)(2401047)(13015025)(13017025)(8121501046)(5005006)(13023025)(13018025)(13024025)(3002001)(10201501046)(6055026)(6096035)(20161123563025)(20161123556025)(20161123561025)(20161123559025)(20161123565025); SRVR:MWHPR03MB2957; BCL:0; PCL:0; RULEID:(400006); SRVR:MWHPR03MB2957; X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2957; 4:FvLHKw/zjBcwEO+7mxmILimo0Gz5OFaW073gcS1EIFHOCN28CiQHYlYhLyqvltafI3d+lljdGzIzOuNHxKMd3sYTjSc0ih/gLoaK1TJoyb5GfLlZd9OlBtyCzaWtLc8F79PDr/U6yg0oBECjzMqdyc+4ZF29AtGTgY9X7fbSOFbWgWtBjcCQQ2XRyRq9yQxJIMTLuKpS/MSJCePYe9RF4rK1Swn2vrDNchlgwSJKq/+6yrsMdpNVBB3a+nagNEc2vFy7uU5GtjXS6DoRntwTRYD3u1JdLGBsfpLhmnqiKfLufEyFi+fxwSdCvnByXUZYSWSb9XxJtkTu7ejZD3x9o41MHSVqmAsjoWKQXsGo1GLMQRGBb1Vzgbu31CsRBnLavFI+8WMdcWYDidTAb/So+O+nlTu2gnvNZ1CzrhYR9usQI3WyzezKvGrsEPjumHEpupsCaPjZjj8bz2L2lPidnpcqcoUMZ0ijgq7x92XTQMqeoXgwzWuy3mR3EHaUTNI8DjMtxU/lTPpoV7lmgKoCyKFmMcMAsm2uFOxs2E0G7K6VBDFXCeoi0X76jfBsWsPU3lJklAceinpcXx9aHEUkM3ZhLyQrswA/rVAHPVCTY/lY+f3s8S0XSoMWg6fuGl43tREOc4Dm/z8Cr7uJx9ex2jwiX7ekPTl4Uyh6nAKxEZHUfpijyirQ6wg3t7nBuxSHDvqSGE5BCsTu6kchR5igG43lHj79opPmhd6ovwNsaUHvbFcKfa7BWDPz2bX/rJqFX2bvXcw3Fd+U7EELD4awNQ== X-Forefront-PRVS: 0235CBE7D0 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR03MB2957; 23:De8VJ7+L8bvNtN//Tt63vlXl6AdGGouYVqoSMILAk?= OCsYegOjX0MI9GDZKIMDAU44SzydFqjPNlfmRXHmZZBANU7J7zyvE5+wYNEDBeqPZyOxLfCDnDALiGXJMZ3fwVZWZ6y4jFio6qtFWpCh5sHdD6OCp4VwAQH5bVIo7RPk4gjrtC89TrwZ2Qddp2A++R4hQMOvvixkUdpJao3D98E5bl/t+7ZzrT5lSAwayh9dWbtt6AupwUW1NZuE6T5bQKyXFC3l3tfd1rdGJB3E0KRaxmHNjica6myFcuTlTWfGYgdRY0mzYaBTWru9YldR8CDNmxOQxRpx10c0PwDbjiZcNPr7kyYpGuQv2D6lqUQz8G+g/ucsIY1FrXKqeaHpUlwMhaUDM8B4JS7lSGKXhrfNAGgNzEQ4GDqCsaLUdg/Itw9+5Wo0KwtdmaZAfKy+KO1xDWiJtucabXTFFcKvRhssYlL9CTPLSn6ZTiuFvj9QrsjvCOKS/PC0HlIrCBFBGnUvrMxyEPQlCH7dg29uwuaIXEWdd9qbz7iQ2jhxySFZNtwcODsva4YyxzYnuQnIU4NHrbcS2KWDgN6hEyv7/q8wNlnHg71IvYcxmsUh6b7BwB4OthkdUPz4eL1c15v8leRFmOgxBQ54LASe6ONEkSHndkMdu980CIPbOmYlPG4RhJHKcFvrxqiS0tOHX2AVWb7fEnQVdHK+Cp0F4r11Bi2X+3biet2Hxzzwgd/MJ8fwXuR4eRtMZeF6U+6vtNxd6b1ol5Ygsdmj/TE1K6M7z7vQUseJ16uYzqoWWkH6RAvNFV8lj5tih1lEwElCXE1Dcg2r4x4FjXpwnLPtz68+hGIuo0/GACAz+PszixI507CUT9+wpMamqAuKBsTjlRw47aykp+1mGLfKGq97iWRJ4NtMr0O9DhpwoWjyWtds7LEb88uJRt+J06ok39x1WPhj+giKzeMhwlZMXJOlumPi70N+AWw5RyyL9XuEI7KC3fJ/ntyBvOEFqSJRlimRGZvV5raar6VzlvqEhIxbkwB/zfqolczBk+FgUYEEIZWE0HgZRRFEtQeFXGVElePiGa1wLoDjThD9Gj/NXYQz82EuLe5XSyJvuoQTTNwsj4u8TW8ZI9MZn3jOfelltYUquFAgpg5+8LD3UUTnzUS6aljIWtF+KKKpS9Az8mfE4PTrcZeZ9ueAN/7gc2tuhcsK/NkT4cG3NdFH11AtlC5/zB7wbdeGHeci1AoJjl43fGcyGxTn+oq6QRlsWG18e18erKGzUCODAd9pnjHfn0OZbNiCeZ1Rv7erkC1MTpp7RNcw396lgT5wiTBuLXnm2FYnH9NG5we X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2957; 6:cYP69WsyhbEq92Ab0dDE/ThAVVyLRcNZIysXY77T47RwZW4NjCNCl6CXE9gl8N75tbxD+cwe+D/3jDbV8TySm4o+xOzAXsiqHTe9PnGaaBO03v13zUpAhcAPbfC7H4LpmI77gkn2z7ItUA9eaDPfjzCEI5raUnRvhX0x/Mku2DNznXCM2BeQim7uQdveV/G3ouMwVhidKN1SEwwCVEpjjG705gGETIdFFSEYNxmNaYtYVY+ghNaHfM9/J/RgAuFPbzI95l8Iq2Cxy0XzfJpgQU7PLG2KV4rPRgfKvHpGtBv1b9R0uox1xwjmEuUuhgNKWbYHE1J8QakCknCDLEWw0VG1UD2gONt5kZJ7iwTTnmrekXJHW0aLSXAubm7t0rwwlYLavSTfold+IQwJxzUrxZ2MCbrnpYwg1tYGyD25j1U=; 5:UkM5AAx/DwApk9EstVdJmTdksBZERPJGpuKHojSszO4VWD+zZHfNGWsi2c4I0VwONxftddocZ5YDnnAn5yT0PjSlzZMWGCf1W5H8xpTIsIDMSHRV/Hv19+rrF1CDRPpDaau/75t7FlQgvb9TuFJ1qUA8bsRAsWdxqvifmfz6/X4PZyvrFlgu94/K+biIvb3l; 24:FHQBHjCO+fD46SQDtgbsA22N1iXw1DdOTZ35CMn2xBYAutadvXD3FHUSdgFDR+CiiGB/kOYEQUcIAvHuwCIPs2767FDLfCh5MZ75hme36yw= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2957; 7:I7IVjDo3QNufEizaWfZPn5nb9uLwSoSSChIPCEHsvoKBq3d1d7xrWuAp8Ac9cJDQGrc8w71/5slQmPZRUvS1SSfQl4B/ZdKcHXt7d04ZLF7A4tM/7LtOsZeVOqu9nybSGN2C/wBHnyWqyX4jXCNAC1IGSp8jGWvhZxHYhS7V3ckTrPJ+Zm+NY8c6O2gQw+BqfT6a76+VInRtkD/kKoWtDarbvEZ1x21E5dhqXC6gwcIwjQo0upps6MnRdDmlHBMDAI2cq8+Jv1Ba+1pLYVE8X1veS5bGhR3TTwnj8NvCEHG/pBdYhJz2ArYjd92KNxBOtRUJ4ifGLT4SMAn3va1UOg== X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Mar 2017 14:11:36.4362 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB2957 Subject: [dpdk-dev] [PATCH v4 07/12] crypto/dpaa2_sec: add crypto operation support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Akhil Goyal Signed-off-by: Hemant Agrawal --- drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 25 + drivers/bus/fslmc/rte_bus_fslmc_version.map | 1 + drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 1210 +++++++++++++++++++++++++++ drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 143 ++++ 4 files changed, 1379 insertions(+) diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h index ad8a22f..dd6ad5b 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h +++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h @@ -146,8 +146,11 @@ struct qbman_fle { } while (0) #define DPAA2_SET_FD_LEN(fd, length) (fd)->simple.len = length #define DPAA2_SET_FD_BPID(fd, bpid) ((fd)->simple.bpid_offset |= bpid) +#define DPAA2_SET_FD_IVP(fd) ((fd->simple.bpid_offset |= 0x00004000)) #define DPAA2_SET_FD_OFFSET(fd, offset) \ ((fd->simple.bpid_offset |= (uint32_t)(offset) << 16)) +#define DPAA2_SET_FD_INTERNAL_JD(fd, len) fd->simple.frc = (0x80000000 | (len)) +#define DPAA2_SET_FD_FRC(fd, frc) fd->simple.frc = frc #define DPAA2_RESET_FD_CTRL(fd) (fd)->simple.ctrl = 0 #define DPAA2_SET_FD_ASAL(fd, asal) ((fd)->simple.ctrl |= (asal << 16)) @@ -155,12 +158,32 @@ struct qbman_fle { fd->simple.flc_lo = lower_32_bits((uint64_t)(addr)); \ fd->simple.flc_hi = upper_32_bits((uint64_t)(addr)); \ } while (0) +#define DPAA2_SET_FLE_INTERNAL_JD(fle, len) (fle->frc = (0x80000000 | (len))) +#define DPAA2_GET_FLE_ADDR(fle) \ + (uint64_t)((((uint64_t)(fle->addr_hi)) << 32) + fle->addr_lo) +#define DPAA2_SET_FLE_ADDR(fle, addr) do { \ + fle->addr_lo = lower_32_bits((uint64_t)addr); \ + fle->addr_hi = upper_32_bits((uint64_t)addr); \ +} while (0) +#define DPAA2_SET_FLE_OFFSET(fle, offset) \ + ((fle)->fin_bpid_offset |= (uint32_t)(offset) << 16) +#define DPAA2_SET_FLE_BPID(fle, bpid) ((fle)->fin_bpid_offset |= (uint64_t)bpid) +#define DPAA2_GET_FLE_BPID(fle, bpid) (fle->fin_bpid_offset & 0x000000ff) +#define DPAA2_SET_FLE_FIN(fle) (fle->fin_bpid_offset |= (uint64_t)1 << 31) +#define DPAA2_SET_FLE_IVP(fle) (((fle)->fin_bpid_offset |= 0x00004000)) +#define DPAA2_SET_FD_COMPOUND_FMT(fd) \ + (fd->simple.bpid_offset |= (uint32_t)1 << 28) #define DPAA2_GET_FD_ADDR(fd) \ ((uint64_t)((((uint64_t)((fd)->simple.addr_hi)) << 32) + (fd)->simple.addr_lo)) #define DPAA2_GET_FD_LEN(fd) ((fd)->simple.len) #define DPAA2_GET_FD_BPID(fd) (((fd)->simple.bpid_offset & 0x00003FFF)) +#define DPAA2_GET_FD_IVP(fd) ((fd->simple.bpid_offset & 0x00004000) >> 14) #define DPAA2_GET_FD_OFFSET(fd) (((fd)->simple.bpid_offset & 0x0FFF0000) >> 16) +#define DPAA2_SET_FLE_SG_EXT(fle) (fle->fin_bpid_offset |= (uint64_t)1 << 29) +#define DPAA2_IS_SET_FLE_SG_EXT(fle) \ + ((fle->fin_bpid_offset & ((uint64_t)1 << 29)) ? 1 : 0) + #define DPAA2_INLINE_MBUF_FROM_BUF(buf, meta_data_size) \ ((struct rte_mbuf *)((uint64_t)(buf) - (meta_data_size))) @@ -215,6 +238,7 @@ static phys_addr_t dpaa2_mem_vtop(uint64_t vaddr) */ #define DPAA2_MBUF_VADDR_TO_IOVA(mbuf) ((mbuf)->buf_physaddr) +#define DPAA2_OP_VADDR_TO_IOVA(op) (op->phys_addr) /** * macro to convert Virtual address to IOVA @@ -235,6 +259,7 @@ static phys_addr_t dpaa2_mem_vtop(uint64_t vaddr) #else /* RTE_LIBRTE_DPAA2_USE_PHYS_IOVA */ #define DPAA2_MBUF_VADDR_TO_IOVA(mbuf) ((mbuf)->buf_addr) +#define DPAA2_OP_VADDR_TO_IOVA(op) (op) #define DPAA2_VADDR_TO_IOVA(_vaddr) (_vaddr) #define DPAA2_IOVA_TO_VADDR(_iova) (_iova) #define DPAA2_MODIFY_IOVA_TO_VADDR(_mem, _type) diff --git a/drivers/bus/fslmc/rte_bus_fslmc_version.map b/drivers/bus/fslmc/rte_bus_fslmc_version.map index 97d6b15..d792af2 100644 --- a/drivers/bus/fslmc/rte_bus_fslmc_version.map +++ b/drivers/bus/fslmc/rte_bus_fslmc_version.map @@ -23,6 +23,7 @@ DPDK_17.05 { per_lcore__dpaa2_io; qbman_check_command_complete; qbman_eq_desc_clear; + qbman_eq_desc_set_fq; qbman_eq_desc_set_no_orp; qbman_eq_desc_set_qd; qbman_eq_desc_set_response; diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index 7287c53..3f517f4 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -47,17 +47,1216 @@ #include #include #include +#include #include #include #include "dpaa2_sec_priv.h" #include "dpaa2_sec_logs.h" +/* RTA header files */ +#include +#include + +/* Minimum job descriptor consists of a oneword job descriptor HEADER and + * a pointer to the shared descriptor + */ +#define MIN_JOB_DESC_SIZE (CAAM_CMD_SZ + CAAM_PTR_SZ) #define FSL_VENDOR_ID 0x1957 #define FSL_DEVICE_ID 0x410 #define FSL_SUBSYSTEM_SEC 1 #define FSL_MC_DPSECI_DEVID 3 +#define NO_PREFETCH 0 +#define TDES_CBC_IV_LEN 8 +#define AES_CBC_IV_LEN 16 +enum rta_sec_era rta_sec_era = RTA_SEC_ERA_8; + +static inline void +print_fd(const struct qbman_fd *fd) +{ + printf("addr_lo: %u\n", fd->simple.addr_lo); + printf("addr_hi: %u\n", fd->simple.addr_hi); + printf("len: %u\n", fd->simple.len); + printf("bpid: %u\n", DPAA2_GET_FD_BPID(fd)); + printf("fi_bpid_off: %u\n", fd->simple.bpid_offset); + printf("frc: %u\n", fd->simple.frc); + printf("ctrl: %u\n", fd->simple.ctrl); + printf("flc_lo: %u\n", fd->simple.flc_lo); + printf("flc_hi: %u\n\n", fd->simple.flc_hi); +} + +static inline void +print_fle(const struct qbman_fle *fle) +{ + printf("addr_lo: %u\n", fle->addr_lo); + printf("addr_hi: %u\n", fle->addr_hi); + printf("len: %u\n", fle->length); + printf("fi_bpid_off: %u\n", fle->fin_bpid_offset); + printf("frc: %u\n", fle->frc); +} + +static inline int +build_authenc_fd(dpaa2_sec_session *sess, + struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + struct rte_crypto_sym_op *sym_op = op->sym; + struct ctxt_priv *priv = sess->ctxt; + struct qbman_fle *fle, *sge; + struct sec_flow_context *flc; + uint32_t auth_only_len = sym_op->auth.data.length - + sym_op->cipher.data.length; + int icv_len = sym_op->auth.digest.length; + uint8_t *old_icv; + uint32_t mem_len = (7 * sizeof(struct qbman_fle)) + icv_len; + + PMD_INIT_FUNC_TRACE(); + + /* TODO we are using the first FLE entry to store Mbuf. + * Currently we donot know which FLE has the mbuf stored. + * So while retreiving we can go back 1 FLE from the FD -ADDR + * to get the MBUF Addr from the previous FLE. + * We can have a better approach to use the inline Mbuf + */ + /* TODO - we can use some mempool to avoid malloc here */ + fle = rte_zmalloc(NULL, mem_len, RTE_CACHE_LINE_SIZE); + if (!fle) { + RTE_LOG(ERR, PMD, "Memory alloc failed for SGE\n"); + return -1; + } + DPAA2_SET_FLE_ADDR(fle, DPAA2_OP_VADDR_TO_IOVA(op)); + fle = fle + 1; + sge = fle + 2; + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FD_BPID(fd, bpid); + DPAA2_SET_FLE_BPID(fle, bpid); + DPAA2_SET_FLE_BPID(fle + 1, bpid); + DPAA2_SET_FLE_BPID(sge, bpid); + DPAA2_SET_FLE_BPID(sge + 1, bpid); + DPAA2_SET_FLE_BPID(sge + 2, bpid); + DPAA2_SET_FLE_BPID(sge + 3, bpid); + } else { + DPAA2_SET_FD_IVP(fd); + DPAA2_SET_FLE_IVP(fle); + DPAA2_SET_FLE_IVP((fle + 1)); + DPAA2_SET_FLE_IVP(sge); + DPAA2_SET_FLE_IVP((sge + 1)); + DPAA2_SET_FLE_IVP((sge + 2)); + DPAA2_SET_FLE_IVP((sge + 3)); + } + + /* Save the shared descriptor */ + flc = &priv->flc_desc[0].flc; + /* Configure FD as a FRAME LIST */ + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(fle)); + DPAA2_SET_FD_COMPOUND_FMT(fd); + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + PMD_TX_LOG(DEBUG, "auth_off: 0x%x/length %d, digest-len=%d\n" + "cipher_off: 0x%x/length %d, iv-len=%d data_off: 0x%x\n", + sym_op->auth.data.offset, + sym_op->auth.data.length, + sym_op->auth.digest.length, + sym_op->cipher.data.offset, + sym_op->cipher.data.length, + sym_op->cipher.iv.length, + sym_op->m_src->data_off); + + /* Configure Output FLE with Scatter/Gather Entry */ + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + if (auth_only_len) + DPAA2_SET_FLE_INTERNAL_JD(fle, auth_only_len); + fle->length = (sess->dir == DIR_ENC) ? + (sym_op->cipher.data.length + icv_len) : + sym_op->cipher.data.length; + + DPAA2_SET_FLE_SG_EXT(fle); + + /* Configure Output SGE for Encap/Decap */ + DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->cipher.data.offset + + sym_op->m_src->data_off); + sge->length = sym_op->cipher.data.length; + + if (sess->dir == DIR_ENC) { + sge++; + DPAA2_SET_FLE_ADDR(sge, + DPAA2_VADDR_TO_IOVA(sym_op->auth.digest.data)); + sge->length = sym_op->auth.digest.length; + DPAA2_SET_FD_LEN(fd, (sym_op->auth.data.length + + sym_op->cipher.iv.length)); + } + DPAA2_SET_FLE_FIN(sge); + + sge++; + fle++; + + /* Configure Input FLE with Scatter/Gather Entry */ + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + DPAA2_SET_FLE_SG_EXT(fle); + DPAA2_SET_FLE_FIN(fle); + fle->length = (sess->dir == DIR_ENC) ? + (sym_op->auth.data.length + sym_op->cipher.iv.length) : + (sym_op->auth.data.length + sym_op->cipher.iv.length + + sym_op->auth.digest.length); + + /* Configure Input SGE for Encap/Decap */ + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(sym_op->cipher.iv.data)); + sge->length = sym_op->cipher.iv.length; + sge++; + + DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->auth.data.offset + + sym_op->m_src->data_off); + sge->length = sym_op->auth.data.length; + if (sess->dir == DIR_DEC) { + sge++; + old_icv = (uint8_t *)(sge + 1); + memcpy(old_icv, sym_op->auth.digest.data, + sym_op->auth.digest.length); + memset(sym_op->auth.digest.data, 0, sym_op->auth.digest.length); + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(old_icv)); + sge->length = sym_op->auth.digest.length; + DPAA2_SET_FD_LEN(fd, (sym_op->auth.data.length + + sym_op->auth.digest.length + + sym_op->cipher.iv.length)); + } + DPAA2_SET_FLE_FIN(sge); + if (auth_only_len) { + DPAA2_SET_FLE_INTERNAL_JD(fle, auth_only_len); + DPAA2_SET_FD_INTERNAL_JD(fd, auth_only_len); + } + return 0; +} + +static inline int +build_auth_fd(dpaa2_sec_session *sess, struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + struct rte_crypto_sym_op *sym_op = op->sym; + struct qbman_fle *fle, *sge; + uint32_t mem_len = (sess->dir == DIR_ENC) ? + (3 * sizeof(struct qbman_fle)) : + (5 * sizeof(struct qbman_fle) + + sym_op->auth.digest.length); + struct sec_flow_context *flc; + struct ctxt_priv *priv = sess->ctxt; + uint8_t *old_digest; + + PMD_INIT_FUNC_TRACE(); + + fle = rte_zmalloc(NULL, mem_len, RTE_CACHE_LINE_SIZE); + if (!fle) { + RTE_LOG(ERR, PMD, "Memory alloc failed for FLE\n"); + return -1; + } + /* TODO we are using the first FLE entry to store Mbuf. + * Currently we donot know which FLE has the mbuf stored. + * So while retreiving we can go back 1 FLE from the FD -ADDR + * to get the MBUF Addr from the previous FLE. + * We can have a better approach to use the inline Mbuf + */ + DPAA2_SET_FLE_ADDR(fle, DPAA2_OP_VADDR_TO_IOVA(op)); + fle = fle + 1; + + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FD_BPID(fd, bpid); + DPAA2_SET_FLE_BPID(fle, bpid); + DPAA2_SET_FLE_BPID(fle + 1, bpid); + } else { + DPAA2_SET_FD_IVP(fd); + DPAA2_SET_FLE_IVP(fle); + DPAA2_SET_FLE_IVP((fle + 1)); + } + flc = &priv->flc_desc[DESC_INITFINAL].flc; + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sym_op->auth.digest.data)); + fle->length = sym_op->auth.digest.length; + + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(fle)); + DPAA2_SET_FD_COMPOUND_FMT(fd); + fle++; + + if (sess->dir == DIR_ENC) { + DPAA2_SET_FLE_ADDR(fle, + DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(fle, sym_op->auth.data.offset + + sym_op->m_src->data_off); + DPAA2_SET_FD_LEN(fd, sym_op->auth.data.length); + fle->length = sym_op->auth.data.length; + } else { + sge = fle + 2; + DPAA2_SET_FLE_SG_EXT(fle); + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FLE_BPID(sge, bpid); + DPAA2_SET_FLE_BPID(sge + 1, bpid); + } else { + DPAA2_SET_FLE_IVP(sge); + DPAA2_SET_FLE_IVP((sge + 1)); + } + DPAA2_SET_FLE_ADDR(sge, + DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->auth.data.offset + + sym_op->m_src->data_off); + + DPAA2_SET_FD_LEN(fd, sym_op->auth.data.length + + sym_op->auth.digest.length); + sge->length = sym_op->auth.data.length; + sge++; + old_digest = (uint8_t *)(sge + 1); + rte_memcpy(old_digest, sym_op->auth.digest.data, + sym_op->auth.digest.length); + memset(sym_op->auth.digest.data, 0, sym_op->auth.digest.length); + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(old_digest)); + sge->length = sym_op->auth.digest.length; + fle->length = sym_op->auth.data.length + + sym_op->auth.digest.length; + DPAA2_SET_FLE_FIN(sge); + } + DPAA2_SET_FLE_FIN(fle); + + return 0; +} + +static int +build_cipher_fd(dpaa2_sec_session *sess, struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + struct rte_crypto_sym_op *sym_op = op->sym; + struct qbman_fle *fle, *sge; + uint32_t mem_len = (5 * sizeof(struct qbman_fle)); + struct sec_flow_context *flc; + struct ctxt_priv *priv = sess->ctxt; + + PMD_INIT_FUNC_TRACE(); + + /* todo - we can use some mempool to avoid malloc here */ + fle = rte_zmalloc(NULL, mem_len, RTE_CACHE_LINE_SIZE); + if (!fle) { + RTE_LOG(ERR, PMD, "Memory alloc failed for SGE\n"); + return -1; + } + /* TODO we are using the first FLE entry to store Mbuf. + * Currently we donot know which FLE has the mbuf stored. + * So while retreiving we can go back 1 FLE from the FD -ADDR + * to get the MBUF Addr from the previous FLE. + * We can have a better approach to use the inline Mbuf + */ + DPAA2_SET_FLE_ADDR(fle, DPAA2_OP_VADDR_TO_IOVA(op)); + fle = fle + 1; + sge = fle + 2; + + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FD_BPID(fd, bpid); + DPAA2_SET_FLE_BPID(fle, bpid); + DPAA2_SET_FLE_BPID(fle + 1, bpid); + DPAA2_SET_FLE_BPID(sge, bpid); + DPAA2_SET_FLE_BPID(sge + 1, bpid); + } else { + DPAA2_SET_FD_IVP(fd); + DPAA2_SET_FLE_IVP(fle); + DPAA2_SET_FLE_IVP((fle + 1)); + DPAA2_SET_FLE_IVP(sge); + DPAA2_SET_FLE_IVP((sge + 1)); + } + + flc = &priv->flc_desc[0].flc; + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(fle)); + DPAA2_SET_FD_LEN(fd, sym_op->cipher.data.length + + sym_op->cipher.iv.length); + DPAA2_SET_FD_COMPOUND_FMT(fd); + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + PMD_TX_LOG(DEBUG, "cipher_off: 0x%x/length %d,ivlen=%d data_off: 0x%x", + sym_op->cipher.data.offset, + sym_op->cipher.data.length, + sym_op->cipher.iv.length, + sym_op->m_src->data_off); + + DPAA2_SET_FLE_ADDR(fle, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(fle, sym_op->cipher.data.offset + + sym_op->m_src->data_off); + + fle->length = sym_op->cipher.data.length + sym_op->cipher.iv.length; + + PMD_TX_LOG(DEBUG, "1 - flc = %p, fle = %p FLEaddr = %x-%x, length %d", + flc, fle, fle->addr_hi, fle->addr_lo, fle->length); + + fle++; + + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + fle->length = sym_op->cipher.data.length + sym_op->cipher.iv.length; + + DPAA2_SET_FLE_SG_EXT(fle); + + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(sym_op->cipher.iv.data)); + sge->length = sym_op->cipher.iv.length; + + sge++; + DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->cipher.data.offset + + sym_op->m_src->data_off); + + sge->length = sym_op->cipher.data.length; + DPAA2_SET_FLE_FIN(sge); + DPAA2_SET_FLE_FIN(fle); + + PMD_TX_LOG(DEBUG, "fdaddr =%p bpid =%d meta =%d off =%d, len =%d", + (void *)DPAA2_GET_FD_ADDR(fd), + DPAA2_GET_FD_BPID(fd), + bpid_info[bpid].meta_data_size, + DPAA2_GET_FD_OFFSET(fd), + DPAA2_GET_FD_LEN(fd)); + + return 0; +} + +static inline int +build_sec_fd(dpaa2_sec_session *sess, struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + int ret = -1; + + PMD_INIT_FUNC_TRACE(); + + switch (sess->ctxt_type) { + case DPAA2_SEC_CIPHER: + ret = build_cipher_fd(sess, op, fd, bpid); + break; + case DPAA2_SEC_AUTH: + ret = build_auth_fd(sess, op, fd, bpid); + break; + case DPAA2_SEC_CIPHER_HASH: + ret = build_authenc_fd(sess, op, fd, bpid); + break; + case DPAA2_SEC_HASH_CIPHER: + default: + RTE_LOG(ERR, PMD, "error: Unsupported session\n"); + } + return ret; +} + +static uint16_t +dpaa2_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function to transmit the frames to given device and VQ*/ + uint32_t loop; + int32_t ret; + struct qbman_fd fd_arr[MAX_TX_RING_SLOTS]; + uint32_t frames_to_send; + struct qbman_eq_desc eqdesc; + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp; + struct qbman_swp *swp; + uint16_t num_tx = 0; + /*todo - need to support multiple buffer pools */ + uint16_t bpid; + struct rte_mempool *mb_pool; + dpaa2_sec_session *sess; + + if (unlikely(nb_ops == 0)) + return 0; + + if (ops[0]->sym->sess_type != RTE_CRYPTO_SYM_OP_WITH_SESSION) { + RTE_LOG(ERR, PMD, "sessionless crypto op not supported\n"); + return 0; + } + /*Prepare enqueue descriptor*/ + qbman_eq_desc_clear(&eqdesc); + qbman_eq_desc_set_no_orp(&eqdesc, DPAA2_EQ_RESP_ERR_FQ); + qbman_eq_desc_set_response(&eqdesc, 0, 0); + qbman_eq_desc_set_fq(&eqdesc, dpaa2_qp->tx_vq.fqid); + + if (!DPAA2_PER_LCORE_SEC_DPIO) { + ret = dpaa2_affine_qbman_swp_sec(); + if (ret) { + RTE_LOG(ERR, PMD, "Failure in affining portal\n"); + return 0; + } + } + swp = DPAA2_PER_LCORE_SEC_PORTAL; + + while (nb_ops) { + frames_to_send = (nb_ops >> 3) ? MAX_TX_RING_SLOTS : nb_ops; + + for (loop = 0; loop < frames_to_send; loop++) { + /*Clear the unused FD fields before sending*/ + memset(&fd_arr[loop], 0, sizeof(struct qbman_fd)); + sess = (dpaa2_sec_session *) + (*ops)->sym->session->_private; + mb_pool = (*ops)->sym->m_src->pool; + bpid = mempool_to_bpid(mb_pool); + ret = build_sec_fd(sess, *ops, &fd_arr[loop], bpid); + if (ret) { + PMD_DRV_LOG(ERR, "error: Improper packet" + " contents for crypto operation\n"); + goto skip_tx; + } + ops++; + } + loop = 0; + while (loop < frames_to_send) { + loop += qbman_swp_send_multiple(swp, &eqdesc, + &fd_arr[loop], + frames_to_send - loop); + } + + num_tx += frames_to_send; + nb_ops -= frames_to_send; + } +skip_tx: + dpaa2_qp->tx_vq.tx_pkts += num_tx; + dpaa2_qp->tx_vq.err_pkts += nb_ops; + return num_tx; +} + +static inline struct rte_crypto_op * +sec_fd_to_mbuf(const struct qbman_fd *fd) +{ + struct qbman_fle *fle; + struct rte_crypto_op *op; + + fle = (struct qbman_fle *)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd)); + + PMD_RX_LOG(DEBUG, "FLE addr = %x - %x, offset = %x", + fle->addr_hi, fle->addr_lo, fle->fin_bpid_offset); + + /* TODO we are using the first FLE entry to store Mbuf. + * Currently we donot know which FLE has the mbuf stored. + * So while retreiving we can go back 1 FLE from the FD -ADDR + * to get the MBUF Addr from the previous FLE. + * We can have a better approach to use the inline Mbuf + */ + + if (unlikely(DPAA2_GET_FD_IVP(fd))) { + /* TODO complete it. */ + RTE_LOG(ERR, PMD, "error: Non inline buffer - WHAT to DO?"); + return NULL; + } + op = (struct rte_crypto_op *)DPAA2_IOVA_TO_VADDR( + DPAA2_GET_FLE_ADDR((fle - 1))); + + /* Prefeth op */ + rte_prefetch0(op->sym->m_src); + + PMD_RX_LOG(DEBUG, "mbuf %p BMAN buf addr %p", + (void *)op->sym->m_src, op->sym->m_src->buf_addr); + + PMD_RX_LOG(DEBUG, "fdaddr =%p bpid =%d meta =%d off =%d, len =%d", + (void *)DPAA2_GET_FD_ADDR(fd), + DPAA2_GET_FD_BPID(fd), + bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size, + DPAA2_GET_FD_OFFSET(fd), + DPAA2_GET_FD_LEN(fd)); + + /* free the fle memory */ + rte_free(fle - 1); + + return op; +} + +static uint16_t +dpaa2_sec_dequeue_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function is responsible to receive frames for a given device and VQ*/ + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp; + struct qbman_result *dq_storage; + uint32_t fqid = dpaa2_qp->rx_vq.fqid; + int ret, num_rx = 0; + uint8_t is_last = 0, status; + struct qbman_swp *swp; + const struct qbman_fd *fd; + struct qbman_pull_desc pulldesc; + + if (!DPAA2_PER_LCORE_SEC_DPIO) { + ret = dpaa2_affine_qbman_swp_sec(); + if (ret) { + RTE_LOG(ERR, PMD, "Failure in affining portal\n"); + return 0; + } + } + swp = DPAA2_PER_LCORE_SEC_PORTAL; + dq_storage = dpaa2_qp->rx_vq.q_storage->dq_storage[0]; + + qbman_pull_desc_clear(&pulldesc); + qbman_pull_desc_set_numframes(&pulldesc, + (nb_ops > DPAA2_DQRR_RING_SIZE) ? + DPAA2_DQRR_RING_SIZE : nb_ops); + qbman_pull_desc_set_fq(&pulldesc, fqid); + qbman_pull_desc_set_storage(&pulldesc, dq_storage, + (dma_addr_t)DPAA2_VADDR_TO_IOVA(dq_storage), + 1); + + /*Issue a volatile dequeue command. */ + while (1) { + if (qbman_swp_pull(swp, &pulldesc)) { + RTE_LOG(WARNING, PMD, "SEC VDQ command is not issued." + "QBMAN is busy\n"); + /* Portal was busy, try again */ + continue; + } + break; + }; + + /* Receive the packets till Last Dequeue entry is found with + * respect to the above issues PULL command. + */ + while (!is_last) { + /* Check if the previous issued command is completed. + * Also seems like the SWP is shared between the Ethernet Driver + * and the SEC driver. + */ + while (!qbman_check_command_complete(swp, dq_storage)) + ; + + /* Loop until the dq_storage is updated with + * new token by QBMAN + */ + while (!qbman_result_has_new_result(swp, dq_storage)) + ; + /* Check whether Last Pull command is Expired and + * setting Condition for Loop termination + */ + if (qbman_result_DQ_is_pull_complete(dq_storage)) { + is_last = 1; + /* Check for valid frame. */ + status = (uint8_t)qbman_result_DQ_flags(dq_storage); + if ((status & QBMAN_DQ_STAT_VALIDFRAME) == 0) { + PMD_RX_LOG(DEBUG, "No frame is delivered"); + continue; + } + } + + fd = qbman_result_DQ_fd(dq_storage); + ops[num_rx] = sec_fd_to_mbuf(fd); + + if (unlikely(fd->simple.frc)) { + /* TODO Parse SEC errors */ + RTE_LOG(ERR, PMD, "SEC returned Error - %x\n", + fd->simple.frc); + ops[num_rx]->status = RTE_CRYPTO_OP_STATUS_ERROR; + } else { + ops[num_rx]->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + } + + num_rx++; + dq_storage++; + } /* End of Packet Rx loop */ + + dpaa2_qp->rx_vq.rx_pkts += num_rx; + + PMD_RX_LOG(DEBUG, "SEC Received %d Packets", num_rx); + /*Return the total number of packets received to DPAA2 app*/ + return num_rx; +} +/** Release queue pair */ +static int +dpaa2_sec_queue_pair_release(struct rte_cryptodev *dev, uint16_t queue_pair_id) +{ + struct dpaa2_sec_qp *qp = + (struct dpaa2_sec_qp *)dev->data->queue_pairs[queue_pair_id]; + + PMD_INIT_FUNC_TRACE(); + + if (qp->rx_vq.q_storage) { + dpaa2_free_dq_storage(qp->rx_vq.q_storage); + rte_free(qp->rx_vq.q_storage); + } + rte_free(qp); + + dev->data->queue_pairs[queue_pair_id] = NULL; + + return 0; +} + +/** Setup a queue pair */ +static int +dpaa2_sec_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, + __rte_unused const struct rte_cryptodev_qp_conf *qp_conf, + __rte_unused int socket_id) +{ + struct dpaa2_sec_dev_private *priv = dev->data->dev_private; + struct dpaa2_sec_qp *qp; + struct fsl_mc_io *dpseci = (struct fsl_mc_io *)priv->hw; + struct dpseci_rx_queue_cfg cfg; + int32_t retcode; + + PMD_INIT_FUNC_TRACE(); + + /* If qp is already in use free ring memory and qp metadata. */ + if (dev->data->queue_pairs[qp_id] != NULL) { + PMD_DRV_LOG(INFO, "QP already setup"); + return 0; + } + + PMD_DRV_LOG(DEBUG, "dev =%p, queue =%d, conf =%p", + dev, qp_id, qp_conf); + + memset(&cfg, 0, sizeof(struct dpseci_rx_queue_cfg)); + + qp = rte_malloc(NULL, sizeof(struct dpaa2_sec_qp), + RTE_CACHE_LINE_SIZE); + if (!qp) { + RTE_LOG(ERR, PMD, "malloc failed for rx/tx queues\n"); + return -1; + } + + qp->rx_vq.dev = dev; + qp->tx_vq.dev = dev; + qp->rx_vq.q_storage = rte_malloc("sec dq storage", + sizeof(struct queue_storage_info_t), + RTE_CACHE_LINE_SIZE); + if (!qp->rx_vq.q_storage) { + RTE_LOG(ERR, PMD, "malloc failed for q_storage\n"); + return -1; + } + memset(qp->rx_vq.q_storage, 0, sizeof(struct queue_storage_info_t)); + + if (dpaa2_alloc_dq_storage(qp->rx_vq.q_storage)) { + RTE_LOG(ERR, PMD, "dpaa2_alloc_dq_storage failed\n"); + return -1; + } + + dev->data->queue_pairs[qp_id] = qp; + + cfg.options = cfg.options | DPSECI_QUEUE_OPT_USER_CTX; + cfg.user_ctx = (uint64_t)(&qp->rx_vq); + retcode = dpseci_set_rx_queue(dpseci, CMD_PRI_LOW, priv->token, + qp_id, &cfg); + return retcode; +} + +/** Start queue pair */ +static int +dpaa2_sec_queue_pair_start(__rte_unused struct rte_cryptodev *dev, + __rte_unused uint16_t queue_pair_id) +{ + PMD_INIT_FUNC_TRACE(); + + return 0; +} + +/** Stop queue pair */ +static int +dpaa2_sec_queue_pair_stop(__rte_unused struct rte_cryptodev *dev, + __rte_unused uint16_t queue_pair_id) +{ + PMD_INIT_FUNC_TRACE(); + + return 0; +} + +/** Return the number of allocated queue pairs */ +static uint32_t +dpaa2_sec_queue_pair_count(struct rte_cryptodev *dev) +{ + PMD_INIT_FUNC_TRACE(); + + return dev->data->nb_queue_pairs; +} + +/** Returns the size of the aesni gcm session structure */ +static unsigned int +dpaa2_sec_session_get_size(struct rte_cryptodev *dev __rte_unused) +{ + PMD_INIT_FUNC_TRACE(); + + return sizeof(dpaa2_sec_session); +} + +static void +dpaa2_sec_session_initialize(struct rte_mempool *mp __rte_unused, + void *sess __rte_unused) +{ + PMD_INIT_FUNC_TRACE(); +} + +static int +dpaa2_sec_cipher_init(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, + dpaa2_sec_session *session) +{ + struct dpaa2_sec_cipher_ctxt *ctxt = &session->ext_params.cipher_ctxt; + struct alginfo cipherdata; + unsigned int bufsize, i; + struct ctxt_priv *priv; + struct sec_flow_context *flc; + + PMD_INIT_FUNC_TRACE(); + + /* For SEC CIPHER only one descriptor is required. */ + priv = (struct ctxt_priv *)rte_zmalloc(NULL, + sizeof(struct ctxt_priv) + sizeof(struct sec_flc_desc), + RTE_CACHE_LINE_SIZE); + if (priv == NULL) { + RTE_LOG(ERR, PMD, "No Memory for priv CTXT"); + return -1; + } + + flc = &priv->flc_desc[0].flc; + + session->cipher_key.data = rte_zmalloc(NULL, xform->cipher.key.length, + RTE_CACHE_LINE_SIZE); + if (session->cipher_key.data == NULL) { + RTE_LOG(ERR, PMD, "No Memory for cipher key"); + return -1; + } + session->cipher_key.length = xform->cipher.key.length; + + memcpy(session->cipher_key.data, xform->cipher.key.data, + xform->cipher.key.length); + cipherdata.key = (uint64_t)session->cipher_key.data; + cipherdata.keylen = session->cipher_key.length; + cipherdata.key_enc_flags = 0; + cipherdata.key_type = RTA_DATA_IMM; + + switch (xform->cipher.algo) { + case RTE_CRYPTO_CIPHER_AES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_AES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_AES_CBC; + ctxt->iv.length = AES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_3DES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_3DES_CBC; + ctxt->iv.length = TDES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + case RTE_CRYPTO_CIPHER_3DES_CTR: + case RTE_CRYPTO_CIPHER_AES_GCM: + case RTE_CRYPTO_CIPHER_AES_CCM: + case RTE_CRYPTO_CIPHER_AES_ECB: + case RTE_CRYPTO_CIPHER_3DES_ECB: + case RTE_CRYPTO_CIPHER_AES_XTS: + case RTE_CRYPTO_CIPHER_AES_F8: + case RTE_CRYPTO_CIPHER_ARC4: + case RTE_CRYPTO_CIPHER_KASUMI_F8: + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + case RTE_CRYPTO_CIPHER_ZUC_EEA3: + case RTE_CRYPTO_CIPHER_NULL: + RTE_LOG(ERR, PMD, "Crypto: Unsupported Cipher alg %u", + xform->cipher.algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Cipher specified %u\n", + xform->cipher.algo); + goto error_out; + } + session->dir = (xform->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DIR_ENC : DIR_DEC; + + bufsize = cnstr_shdsc_blkcipher(priv->flc_desc[0].desc, 1, 0, + &cipherdata, NULL, ctxt->iv.length, + session->dir); + flc->dhr = 0; + flc->bpv0 = 0x1; + flc->mode_bits = 0x8000; + + flc->word1_sdl = (uint8_t)bufsize; + flc->word2_rflc_31_0 = lower_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + flc->word3_rflc_63_32 = upper_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + session->ctxt = priv; + + for (i = 0; i < bufsize; i++) + PMD_DRV_LOG(DEBUG, "DESC[%d]:0x%x\n", + i, priv->flc_desc[0].desc[i]); + + return 0; + +error_out: + rte_free(session->cipher_key.data); + return -1; +} + +static int +dpaa2_sec_auth_init(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, + dpaa2_sec_session *session) +{ + struct dpaa2_sec_auth_ctxt *ctxt = &session->ext_params.auth_ctxt; + struct alginfo authdata; + unsigned int bufsize; + struct ctxt_priv *priv; + struct sec_flow_context *flc; + + PMD_INIT_FUNC_TRACE(); + + /* For SEC AUTH three descriptors are required for various stages */ + priv = (struct ctxt_priv *)rte_zmalloc(NULL, + sizeof(struct ctxt_priv) + 3 * + sizeof(struct sec_flc_desc), + RTE_CACHE_LINE_SIZE); + if (priv == NULL) { + RTE_LOG(ERR, PMD, "No Memory for priv CTXT"); + return -1; + } + + flc = &priv->flc_desc[DESC_INITFINAL].flc; + + session->auth_key.data = rte_zmalloc(NULL, xform->auth.key.length, + RTE_CACHE_LINE_SIZE); + session->auth_key.length = xform->auth.key.length; + + memcpy(session->auth_key.data, xform->auth.key.data, + xform->auth.key.length); + authdata.key = (uint64_t)session->auth_key.data; + authdata.keylen = session->auth_key.length; + authdata.key_enc_flags = 0; + authdata.key_type = RTA_DATA_IMM; + + switch (xform->auth.algo) { + case RTE_CRYPTO_AUTH_SHA1_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA1; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA1_HMAC; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + authdata.algtype = OP_ALG_ALGSEL_MD5; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_MD5_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA256; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA256_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA384; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA384_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA512; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA512_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA224; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA224_HMAC; + break; + case RTE_CRYPTO_AUTH_AES_XCBC_MAC: + case RTE_CRYPTO_AUTH_AES_GCM: + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + case RTE_CRYPTO_AUTH_NULL: + case RTE_CRYPTO_AUTH_SHA1: + case RTE_CRYPTO_AUTH_SHA256: + case RTE_CRYPTO_AUTH_SHA512: + case RTE_CRYPTO_AUTH_SHA224: + case RTE_CRYPTO_AUTH_SHA384: + case RTE_CRYPTO_AUTH_MD5: + case RTE_CRYPTO_AUTH_AES_CCM: + case RTE_CRYPTO_AUTH_AES_GMAC: + case RTE_CRYPTO_AUTH_KASUMI_F9: + case RTE_CRYPTO_AUTH_AES_CMAC: + case RTE_CRYPTO_AUTH_AES_CBC_MAC: + case RTE_CRYPTO_AUTH_ZUC_EIA3: + RTE_LOG(ERR, PMD, "Crypto: Unsupported auth alg %u", + xform->auth.algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Auth specified %u\n", + xform->auth.algo); + goto error_out; + } + session->dir = (xform->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE) ? + DIR_ENC : DIR_DEC; + + bufsize = cnstr_shdsc_hmac(priv->flc_desc[DESC_INITFINAL].desc, + 1, 0, &authdata, !session->dir, + ctxt->trunc_len); + + flc->word1_sdl = (uint8_t)bufsize; + flc->word2_rflc_31_0 = lower_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + flc->word3_rflc_63_32 = upper_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + session->ctxt = priv; + + return 0; + +error_out: + rte_free(session->auth_key.data); + return -1; +} + +static int +dpaa2_sec_aead_init(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, + dpaa2_sec_session *session) +{ + struct dpaa2_sec_aead_ctxt *ctxt = &session->ext_params.aead_ctxt; + struct alginfo authdata, cipherdata; + unsigned int bufsize; + struct ctxt_priv *priv; + struct sec_flow_context *flc; + struct rte_crypto_cipher_xform *cipher_xform; + struct rte_crypto_auth_xform *auth_xform; + int err; + + PMD_INIT_FUNC_TRACE(); + + if (session->ext_params.aead_ctxt.auth_cipher_text) { + cipher_xform = &xform->cipher; + auth_xform = &xform->next->auth; + session->ctxt_type = + (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DPAA2_SEC_CIPHER_HASH : DPAA2_SEC_HASH_CIPHER; + } else { + cipher_xform = &xform->next->cipher; + auth_xform = &xform->auth; + session->ctxt_type = + (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DPAA2_SEC_HASH_CIPHER : DPAA2_SEC_CIPHER_HASH; + } + /* For SEC AEAD only one descriptor is required */ + priv = (struct ctxt_priv *)rte_zmalloc(NULL, + sizeof(struct ctxt_priv) + sizeof(struct sec_flc_desc), + RTE_CACHE_LINE_SIZE); + if (priv == NULL) { + RTE_LOG(ERR, PMD, "No Memory for priv CTXT"); + return -1; + } + + flc = &priv->flc_desc[0].flc; + + session->cipher_key.data = rte_zmalloc(NULL, cipher_xform->key.length, + RTE_CACHE_LINE_SIZE); + if (session->cipher_key.data == NULL) { + RTE_LOG(ERR, PMD, "No Memory for cipher key"); + return -1; + } + session->cipher_key.length = cipher_xform->key.length; + session->auth_key.data = rte_zmalloc(NULL, auth_xform->key.length, + RTE_CACHE_LINE_SIZE); + if (session->auth_key.data == NULL) { + RTE_LOG(ERR, PMD, "No Memory for auth key"); + goto error_out; + } + session->auth_key.length = auth_xform->key.length; + memcpy(session->cipher_key.data, cipher_xform->key.data, + cipher_xform->key.length); + memcpy(session->auth_key.data, auth_xform->key.data, + auth_xform->key.length); + + ctxt->trunc_len = auth_xform->digest_length; + authdata.key = (uint64_t)session->auth_key.data; + authdata.keylen = session->auth_key.length; + authdata.key_enc_flags = 0; + authdata.key_type = RTA_DATA_IMM; + + switch (auth_xform->algo) { + case RTE_CRYPTO_AUTH_SHA1_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA1; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA1_HMAC; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + authdata.algtype = OP_ALG_ALGSEL_MD5; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_MD5_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA224; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA224_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA256; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA256_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA384; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA384_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA512; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA512_HMAC; + break; + case RTE_CRYPTO_AUTH_AES_XCBC_MAC: + case RTE_CRYPTO_AUTH_AES_GCM: + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + case RTE_CRYPTO_AUTH_NULL: + case RTE_CRYPTO_AUTH_SHA1: + case RTE_CRYPTO_AUTH_SHA256: + case RTE_CRYPTO_AUTH_SHA512: + case RTE_CRYPTO_AUTH_SHA224: + case RTE_CRYPTO_AUTH_SHA384: + case RTE_CRYPTO_AUTH_MD5: + case RTE_CRYPTO_AUTH_AES_CCM: + case RTE_CRYPTO_AUTH_AES_GMAC: + case RTE_CRYPTO_AUTH_KASUMI_F9: + case RTE_CRYPTO_AUTH_AES_CMAC: + case RTE_CRYPTO_AUTH_AES_CBC_MAC: + case RTE_CRYPTO_AUTH_ZUC_EIA3: + RTE_LOG(ERR, PMD, "Crypto: Unsupported auth alg %u", + auth_xform->algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Auth specified %u\n", + auth_xform->algo); + goto error_out; + } + cipherdata.key = (uint64_t)session->cipher_key.data; + cipherdata.keylen = session->cipher_key.length; + cipherdata.key_enc_flags = 0; + cipherdata.key_type = RTA_DATA_IMM; + + switch (cipher_xform->algo) { + case RTE_CRYPTO_CIPHER_AES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_AES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_AES_CBC; + ctxt->iv.length = AES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_3DES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_3DES_CBC; + ctxt->iv.length = TDES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_AES_GCM: + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + case RTE_CRYPTO_CIPHER_NULL: + case RTE_CRYPTO_CIPHER_3DES_ECB: + case RTE_CRYPTO_CIPHER_AES_ECB: + case RTE_CRYPTO_CIPHER_AES_CTR: + case RTE_CRYPTO_CIPHER_AES_CCM: + case RTE_CRYPTO_CIPHER_KASUMI_F8: + RTE_LOG(ERR, PMD, "Crypto: Unsupported Cipher alg %u", + cipher_xform->algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Cipher specified %u\n", + cipher_xform->algo); + goto error_out; + } + session->dir = (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DIR_ENC : DIR_DEC; + + priv->flc_desc[0].desc[0] = cipherdata.keylen; + priv->flc_desc[0].desc[1] = authdata.keylen; + err = rta_inline_query(IPSEC_AUTH_VAR_AES_DEC_BASE_DESC_LEN, + MIN_JOB_DESC_SIZE, + (unsigned int *)priv->flc_desc[0].desc, + &priv->flc_desc[0].desc[2], 2); + + if (err < 0) { + PMD_DRV_LOG(ERR, "Crypto: Incorrect key lengths"); + goto error_out; + } + if (priv->flc_desc[0].desc[2] & 1) + cipherdata.key_type = RTA_DATA_IMM; + else { + cipherdata.key = DPAA2_VADDR_TO_IOVA(cipherdata.key); + cipherdata.key_type = RTA_DATA_PTR; + } + if (priv->flc_desc[0].desc[2] & (1<<1)) + authdata.key_type = RTA_DATA_IMM; + else { + authdata.key = DPAA2_VADDR_TO_IOVA(authdata.key); + authdata.key_type = RTA_DATA_PTR; + } + priv->flc_desc[0].desc[0] = 0; + priv->flc_desc[0].desc[1] = 0; + priv->flc_desc[0].desc[2] = 0; + + if (session->ctxt_type == DPAA2_SEC_CIPHER_HASH) { + bufsize = cnstr_shdsc_authenc(priv->flc_desc[0].desc, 1, + 0, &cipherdata, &authdata, + ctxt->iv.length, + ctxt->auth_only_len, + ctxt->trunc_len, + session->dir); + } else { + RTE_LOG(ERR, PMD, "Hash before cipher not supported"); + goto error_out; + } + + flc->word1_sdl = (uint8_t)bufsize; + flc->word2_rflc_31_0 = lower_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + flc->word3_rflc_63_32 = upper_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + session->ctxt = priv; + + return 0; + +error_out: + rte_free(session->cipher_key.data); + rte_free(session->auth_key.data); + return -1; +} + +static void * +dpaa2_sec_session_configure(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, void *sess) +{ + dpaa2_sec_session *session = sess; + + PMD_INIT_FUNC_TRACE(); + + if (unlikely(sess == NULL)) { + RTE_LOG(ERR, PMD, "invalid session struct"); + return NULL; + } + /* Cipher Only */ + if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL) { + session->ctxt_type = DPAA2_SEC_CIPHER; + dpaa2_sec_cipher_init(dev, xform, session); + + /* Authentication Only */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && + xform->next == NULL) { + session->ctxt_type = DPAA2_SEC_AUTH; + dpaa2_sec_auth_init(dev, xform, session); + + /* Cipher then Authenticate */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && + xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + session->ext_params.aead_ctxt.auth_cipher_text = true; + dpaa2_sec_aead_init(dev, xform, session); + + /* Authenticate then Cipher */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && + xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + session->ext_params.aead_ctxt.auth_cipher_text = false; + dpaa2_sec_aead_init(dev, xform, session); + } else { + RTE_LOG(ERR, PMD, "Invalid crypto type"); + return NULL; + } + + return session; +} + +/** Clear the memory of session so it doesn't leave key material behind */ +static void +dpaa2_sec_session_clear(struct rte_cryptodev *dev __rte_unused, void *sess) +{ + PMD_INIT_FUNC_TRACE(); + + if (sess) + memset(sess, 0, sizeof(dpaa2_sec_session)); +} static int dpaa2_sec_dev_configure(struct rte_cryptodev *dev __rte_unused) @@ -194,6 +1393,15 @@ static struct rte_cryptodev_ops crypto_ops = { .dev_stop = dpaa2_sec_dev_stop, .dev_close = dpaa2_sec_dev_close, .dev_infos_get = dpaa2_sec_dev_infos_get, + .queue_pair_setup = dpaa2_sec_queue_pair_setup, + .queue_pair_release = dpaa2_sec_queue_pair_release, + .queue_pair_start = dpaa2_sec_queue_pair_start, + .queue_pair_stop = dpaa2_sec_queue_pair_stop, + .queue_pair_count = dpaa2_sec_queue_pair_count, + .session_get_size = dpaa2_sec_session_get_size, + .session_initialize = dpaa2_sec_session_initialize, + .session_configure = dpaa2_sec_session_configure, + .session_clear = dpaa2_sec_session_clear, }; static int @@ -232,6 +1440,8 @@ dpaa2_sec_dev_init(struct rte_cryptodev *cryptodev) cryptodev->dev_type = RTE_CRYPTODEV_DPAA2_SEC_PMD; cryptodev->dev_ops = &crypto_ops; + cryptodev->enqueue_burst = dpaa2_sec_enqueue_burst; + cryptodev->dequeue_burst = dpaa2_sec_dequeue_burst; cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING; diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h index e0d6148..f2529fe 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h @@ -34,6 +34,8 @@ #ifndef _RTE_DPAA2_SEC_PMD_PRIVATE_H_ #define _RTE_DPAA2_SEC_PMD_PRIVATE_H_ +#define MAX_QUEUES 64 +#define MAX_DESC_SIZE 64 /** private data structure for each DPAA2_SEC device */ struct dpaa2_sec_dev_private { void *mc_portal; /**< MC Portal for configuring this device */ @@ -52,6 +54,147 @@ struct dpaa2_sec_qp { struct dpaa2_queue tx_vq; }; +enum shr_desc_type { + DESC_UPDATE, + DESC_FINAL, + DESC_INITFINAL, +}; + +#define DIR_ENC 1 +#define DIR_DEC 0 + +/* SEC Flow Context Descriptor */ +struct sec_flow_context { + /* word 0 */ + uint16_t word0_sdid; /* 11-0 SDID */ + uint16_t word0_res; /* 31-12 reserved */ + + /* word 1 */ + uint8_t word1_sdl; /* 5-0 SDL */ + /* 7-6 reserved */ + + uint8_t word1_bits_15_8; /* 11-8 CRID */ + /* 14-12 reserved */ + /* 15 CRJD */ + + uint8_t word1_bits23_16; /* 16 EWS */ + /* 17 DAC */ + /* 18,19,20 ? */ + /* 23-21 reserved */ + + uint8_t word1_bits31_24; /* 24 RSC */ + /* 25 RBMT */ + /* 31-26 reserved */ + + /* word 2 RFLC[31-0] */ + uint32_t word2_rflc_31_0; + + /* word 3 RFLC[63-32] */ + uint32_t word3_rflc_63_32; + + /* word 4 */ + uint16_t word4_iicid; /* 15-0 IICID */ + uint16_t word4_oicid; /* 31-16 OICID */ + + /* word 5 */ + uint32_t word5_ofqid:24; /* 23-0 OFQID */ + uint32_t word5_31_24:8; + /* 24 OSC */ + /* 25 OBMT */ + /* 29-26 reserved */ + /* 31-30 ICR */ + + /* word 6 */ + uint32_t word6_oflc_31_0; + + /* word 7 */ + uint32_t word7_oflc_63_32; + + /* Word 8-15 storage profiles */ + uint16_t dl; /**< DataLength(correction) */ + uint16_t reserved; /**< reserved */ + uint16_t dhr; /**< DataHeadRoom(correction) */ + uint16_t mode_bits; /**< mode bits */ + uint16_t bpv0; /**< buffer pool0 valid */ + uint16_t bpid0; /**< Bypass Memory Translation */ + uint16_t bpv1; /**< buffer pool1 valid */ + uint16_t bpid1; /**< Bypass Memory Translation */ + uint64_t word_12_15[2]; /**< word 12-15 are reserved */ +}; + +struct sec_flc_desc { + struct sec_flow_context flc; + uint32_t desc[MAX_DESC_SIZE]; +}; + +struct ctxt_priv { + struct sec_flc_desc flc_desc[0]; +}; + +enum dpaa2_sec_op_type { + DPAA2_SEC_NONE, /*!< No Cipher operations*/ + DPAA2_SEC_CIPHER,/*!< CIPHER operations */ + DPAA2_SEC_AUTH, /*!< Authentication Operations */ + DPAA2_SEC_CIPHER_HASH, /*!< Authenticated Encryption with + * associated data + */ + DPAA2_SEC_HASH_CIPHER, /*!< Encryption with Authenticated + * associated data + */ + DPAA2_SEC_IPSEC, /*!< IPSEC protocol operations*/ + DPAA2_SEC_PDCP, /*!< PDCP protocol operations*/ + DPAA2_SEC_PKC, /*!< Public Key Cryptographic Operations */ + DPAA2_SEC_MAX +}; + +struct dpaa2_sec_cipher_ctxt { + struct { + uint8_t *data; + uint16_t length; + } iv; /**< Initialisation vector parameters */ + uint8_t *init_counter; /*!< Set initial counter for CTR mode */ +}; + +struct dpaa2_sec_auth_ctxt { + uint8_t trunc_len; /*!< Length for output ICV, should + * be 0 if no truncation required + */ +}; + +struct dpaa2_sec_aead_ctxt { + struct { + uint8_t *data; + uint16_t length; + } iv; /**< Initialisation vector parameters */ + uint16_t auth_only_len; /*!< Length of data for Auth only */ + uint8_t auth_cipher_text; /**< Authenticate/cipher ordering */ + uint8_t trunc_len; /*!< Length for output ICV, should + * be 0 if no truncation required + */ +}; + +typedef struct dpaa2_sec_session_entry { + void *ctxt; + uint8_t ctxt_type; + uint8_t dir; /*!< Operation Direction */ + enum rte_crypto_cipher_algorithm cipher_alg; /*!< Cipher Algorithm*/ + enum rte_crypto_auth_algorithm auth_alg; /*!< Authentication Algorithm*/ + struct { + uint8_t *data; /**< pointer to key data */ + size_t length; /**< key length in bytes */ + } cipher_key; + struct { + uint8_t *data; /**< pointer to key data */ + size_t length; /**< key length in bytes */ + } auth_key; + uint8_t status; + union { + struct dpaa2_sec_cipher_ctxt cipher_ctxt; + struct dpaa2_sec_auth_ctxt auth_ctxt; + struct dpaa2_sec_aead_ctxt aead_ctxt; + } ext_params; +} dpaa2_sec_session; + static const struct rte_cryptodev_capabilities dpaa2_sec_capabilities[] = { { /* MD5 HMAC */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,