From patchwork Thu Jul 1 13:25:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95132 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D3229A0A0C; Thu, 1 Jul 2021 15:27:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A89EE412E1; Thu, 1 Jul 2021 15:26:58 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2055.outbound.protection.outlook.com [40.107.94.55]) by mails.dpdk.org (Postfix) with ESMTP id 8A289412DE for ; Thu, 1 Jul 2021 15:26:57 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e8yH69p+PIQ9gFXVhVURreruM3XnJdxMReMZ7waaFqEWioHVMSycIVuPebvOx+YcuQRp6ahe5ePLlZ6rdk8ISgD/Sil0UbLA82EfULAlqduS28NKkBOrDSCLdZWQTJ8NynmczNZwmQBWhEGrHvG7NzYM1twrBvPZEI66mLog9Rgu/ehW4h3JDttKX7RaHraMT8d8kVy7URAqTyDRH0r569iWVkyYdVgEuZhCPibvdLuN1jIle999yqUr7kUAGqn+YBA+ViDecZ/lvzPKN3qAbI0nt60U73LJTkviq9420Y74Kz8z0ua4eL74l1MSfsG9XRPoUj7qAZJBHJeELtGrJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hWj3mTRBNmqEvBf3udHAt1DCnW5ciY0oH9zHYOjHpMo=; b=E0LRhkG2zWSlrQXRPoJNGDblaqaOFEMS48s2UdDd54cv4d3ZYXDoGfX/An/7Z2483pRzxCdp0+LXo4ZezGxKPB6S6P/T3OsUXFeOwCJFVpxttbor/obnmJVzWLIyJYY2RYNv4Jk/LtOLR5GZnYJv9MUeIU8nQdFUuiSaL7wHU860tx5tIcN53uMEYUUhbQW46HxybPMYA9Wct2IA60RdJb/YI76w8zxrAneM4RwDip6/QdEs7msvOBWDZiKyLPBr8HvvEjPj8F78Hnh+B5H6naDZs2OOCFsK5riYa6MrDH4LSyae4TMu2SwLp2EhQKaxf7BByUMHKGJoW050FaRNqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hWj3mTRBNmqEvBf3udHAt1DCnW5ciY0oH9zHYOjHpMo=; b=acD4Dhb3VWzyZukdsZ7iXfNqE0JQNL2IXllIPgdH/uH+bwEo8SLyaBT3gm8Kja88BWIKBFaetnuvthnr2bQ7B53Za2K8qd4x3nfQJ4aCs8ugBwSEE803wDjB4mu6lmjpXSTC+Ft+MV+S1YqB4ifGLu5I6ir+qi6oN1v4VlMY2ftIFRTm12cnUdCdyMfkoqz4/U8tRK7KGpW3vkiqA5e7cBXSo5gd9+MvK/ojdjGNwytmdNVlNGAncyeuwivahfL+Ovu6xhdXtnHxzZD2Dgxpo4Uj2Fn6TmggpU+Ie9zVCuWc59OIOxlrh5HgoEFyWbv6X3gzOCSw+NZUdBayijMjqg== Received: from BN8PR12CA0033.namprd12.prod.outlook.com (2603:10b6:408:60::46) by DM6PR12MB3275.namprd12.prod.outlook.com (2603:10b6:5:185::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.23; Thu, 1 Jul 2021 13:26:55 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::d8) by BN8PR12CA0033.outlook.office365.com (2603:10b6:408:60::46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:26:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:26:54 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:26:51 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:25:55 +0300 Message-ID: <20210701132609.53727-2-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: beef8d2b-06a1-46b5-5044-08d93c93e508 X-MS-TrafficTypeDiagnostic: DM6PR12MB3275: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:138; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: k94EHCToHkpUBXO/m+IvlFDbcLrkEFdwjLcPJs16D7HTvSwjnzh2EAW0Q43yLW4pRkrLGOS+up7o2/3qlx8kGf7UnyYEIP8OJMYO+SaQ+/tgC+Jr4GlHnvzr+a+JME08GI3Dv8OPR8jpjxCcpwuCFAIkyvoTnBmeUn2Mv60wJRlbsrnbJ+WCePTf0pjKVXh1uxtoIMR3Y+TWf49czRwTdiLxfCnEzZFroVXH2TLb5uQffzIktTSK/SzES0JACpwP1dQViXiwzapVaRxKN8mglzL2cBvTO5wv6KRJUzvoTJrw70pNTTChMjzSk6/n7gFdliE6DKRWkKuDWYYPS73i3I//E6K7A7HmUJ8D5Wd6SoYiCcic6olAIPOK/mwkDEY8gP4SBg7KkPbVOawQjNAXDZ2I1A3Ztt891N/HorHEIt166rvmg+3VPyRTzSjD0R9hyR0qb9QuLlSJGrCzZtGJm8qc7SJIzpq8teXiNrV7yVpQQx0DKCBfvPlJmTiegLGgxJ6I6S443JppXraHVZj4gZov3QnmXGazuE7a8WfqRAJrswwTKMpLNMqCdgPTXFCYkBGvomgj2hM/YhrjSORO5BD0aswLXeyEgP65rsJOBrc9jpR9V0wrR3wmtAHPn676JSStaNsYjbCBfL93q/3DmLRiV5NUG/Hh5e9qZgcmbIlpmYNmQYla11RKYUEftWjGgiyWbFefhCr/S+V/ZaE/Vw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(136003)(376002)(39860400002)(396003)(346002)(36840700001)(46966006)(186003)(2616005)(70206006)(7696005)(82310400003)(426003)(4326008)(55016002)(6286002)(1076003)(82740400003)(86362001)(107886003)(70586007)(47076005)(16526019)(478600001)(26005)(356005)(36756003)(54906003)(36860700001)(83380400001)(8676002)(2906002)(30864003)(8936002)(316002)(5660300002)(6916009)(336012)(7636003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:26:54.8033 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: beef8d2b-06a1-46b5-5044-08d93c93e508 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3275 Subject: [dpdk-dev] [PATCH v5 01/15] drivers: introduce mlx5 crypto PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a new PMD for Mellanox devices- crypto PMD. The crypto PMD will be supported starting Nvidia ConnectX6 and BlueField2. The crypto PMD will add the support of encryption and decryption using the AES-XTS symmetric algorithm. The crypto PMD requires rdma-core and uses mlx5 DevX. This patch adds the PCI probing, basic functions, build files and log utility. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- MAINTAINERS | 4 + doc/guides/cryptodevs/features/mlx5.ini | 27 +++ doc/guides/cryptodevs/index.rst | 1 + doc/guides/cryptodevs/mlx5.rst | 63 ++++++ doc/guides/rel_notes/release_21_08.rst | 5 + drivers/common/mlx5/mlx5_common.h | 1 + drivers/common/mlx5/mlx5_common_pci.c | 14 ++ drivers/common/mlx5/mlx5_common_pci.h | 21 +- drivers/crypto/meson.build | 1 + drivers/crypto/mlx5/meson.build | 26 +++ drivers/crypto/mlx5/mlx5_crypto.c | 275 ++++++++++++++++++++++++ drivers/crypto/mlx5/mlx5_crypto_utils.h | 19 ++ drivers/crypto/mlx5/version.map | 3 + 13 files changed, 450 insertions(+), 10 deletions(-) create mode 100644 doc/guides/cryptodevs/features/mlx5.ini create mode 100644 doc/guides/cryptodevs/mlx5.rst create mode 100644 drivers/crypto/mlx5/meson.build create mode 100644 drivers/crypto/mlx5/mlx5_crypto.c create mode 100644 drivers/crypto/mlx5/mlx5_crypto_utils.h create mode 100644 drivers/crypto/mlx5/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 5877a16971..c0e2aa1f9f 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1080,6 +1080,10 @@ F: drivers/crypto/octeontx2/ F: doc/guides/cryptodevs/octeontx2.rst F: doc/guides/cryptodevs/features/octeontx2.ini +Mellanox mlx5 +M: Matan Azrad +F: drivers/crypto/mlx5/ + Null Crypto M: Declan Doherty F: drivers/crypto/null/ diff --git a/doc/guides/cryptodevs/features/mlx5.ini b/doc/guides/cryptodevs/features/mlx5.ini new file mode 100644 index 0000000000..ceadd967b6 --- /dev/null +++ b/doc/guides/cryptodevs/features/mlx5.ini @@ -0,0 +1,27 @@ +; +; Features of a mlx5 crypto driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +HW Accelerated = Y + +; +; Supported crypto algorithms of a mlx5 crypto driver. +; +[Cipher] + +; +; Supported authentication algorithms of a mlx5 crypto driver. +; +[Auth] + +; +; Supported AEAD algorithms of a mlx5 crypto driver. +; +[AEAD] + +; +; Supported Asymmetric algorithms of a mlx5 crypto driver. +; +[Asymmetric] diff --git a/doc/guides/cryptodevs/index.rst b/doc/guides/cryptodevs/index.rst index 279f56a002..747409c441 100644 --- a/doc/guides/cryptodevs/index.rst +++ b/doc/guides/cryptodevs/index.rst @@ -22,6 +22,7 @@ Crypto Device Drivers octeontx octeontx2 openssl + mlx5 mvsam nitrox null diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst new file mode 100644 index 0000000000..05a0a449e2 --- /dev/null +++ b/doc/guides/cryptodevs/mlx5.rst @@ -0,0 +1,63 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright (c) 2021 NVIDIA Corporation & Affiliates + +.. include:: + +MLX5 Crypto Driver +================== + +The MLX5 crypto driver library +(**librte_crypto_mlx5**) provides support for **Mellanox ConnectX-6** +family adapters. + +Overview +-------- + +The device can provide disk encryption services, allowing data encryption +and decryption towards a disk. Having all encryption/decryption +operations done in a single device can reduce cost and overheads of the related +FIPS certification, as ConnectX-6 is FIPS 140-2 level-2 ready. +The encryption cipher is AES-XTS of 256/512 bit key size. + +MKEY is a memory region object in the hardware, that holds address translation information and +attributes per memory area. Its ID must be tied to addresses provided to the hardware. +The encryption operations are performed with MKEY read/write transactions, when +the MKEY is configured to perform crypto operations. + +The encryption does not require text to be aligned to the AES block size (128b). + +The PMD uses libibverbs and libmlx5 to access the device firmware or to +access the hardware components directly. +There are different levels of objects and bypassing abilities. +To get the best performances: + +- Verbs is a complete high-level generic API. +- Direct Verbs is a device-specific API. +- DevX allows to access firmware objects. + +Enabling librte_crypto_mlx5 causes DPDK applications to be linked against +libibverbs. + + +Driver options +-------------- + +- ``class`` parameter [string] + + Select the class of the driver that should probe the device. + `crypto` for the mlx5 crypto driver. + + +Supported NICs +-------------- + +* Mellanox\ |reg| ConnectX\ |reg|-6 200G MCX654106A-HCAT (2x200G) + +Prerequisites +------------- + +- Mellanox OFED version: **5.3** + see :doc:`../../nics/mlx5` guide for more Mellanox OFED details. + +- Compilation can be done also with rdma-core v15+. + see :doc:`../../nics/mlx5` guide for more rdma-core details. diff --git a/doc/guides/rel_notes/release_21_08.rst b/doc/guides/rel_notes/release_21_08.rst index a6ecfdf3ce..e12341fe55 100644 --- a/doc/guides/rel_notes/release_21_08.rst +++ b/doc/guides/rel_notes/release_21_08.rst @@ -55,6 +55,11 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Added support for Nvidia crypto device driver.** + + * Added mlx5 crypto driver to support AES-XTS cipher operations. + The first device to support it is ConnectX-6. + Removed Items ------------- diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 1fbefe0fa6..89aca32305 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -216,6 +216,7 @@ enum mlx5_class { MLX5_CLASS_VDPA = RTE_BIT64(1), MLX5_CLASS_REGEX = RTE_BIT64(2), MLX5_CLASS_COMPRESS = RTE_BIT64(3), + MLX5_CLASS_CRYPTO = RTE_BIT64(4), }; #define MLX5_DBR_SIZE RTE_CACHE_LINE_SIZE diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 3f16cd21cf..8a47afee20 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -31,6 +31,7 @@ static const struct { { .name = "net", .driver_class = MLX5_CLASS_NET }, { .name = "regex", .driver_class = MLX5_CLASS_REGEX }, { .name = "compress", .driver_class = MLX5_CLASS_COMPRESS }, + { .name = "crypto", .driver_class = MLX5_CLASS_CRYPTO }, }; static const unsigned int mlx5_class_combinations[] = { @@ -38,13 +39,26 @@ static const unsigned int mlx5_class_combinations[] = { MLX5_CLASS_VDPA, MLX5_CLASS_REGEX, MLX5_CLASS_COMPRESS, + MLX5_CLASS_CRYPTO, MLX5_CLASS_NET | MLX5_CLASS_REGEX, MLX5_CLASS_VDPA | MLX5_CLASS_REGEX, MLX5_CLASS_NET | MLX5_CLASS_COMPRESS, MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS, MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, + MLX5_CLASS_NET | MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_CRYPTO, + MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, + MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, + MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, + MLX5_CLASS_NET | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, + MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | + MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | + MLX5_CLASS_CRYPTO, /* New class combination should be added here. */ }; diff --git a/drivers/common/mlx5/mlx5_common_pci.h b/drivers/common/mlx5/mlx5_common_pci.h index de89bb98bc..cb8d2f5f87 100644 --- a/drivers/common/mlx5/mlx5_common_pci.h +++ b/drivers/common/mlx5/mlx5_common_pci.h @@ -9,17 +9,18 @@ * @file * * RTE Mellanox PCI Driver Interface - * Mellanox ConnectX PCI device supports multiple class: net,vdpa,regex and - * compress devices. This layer enables creating such multiple class of devices - * on a single PCI device by allowing to bind multiple class specific device - * driver to attach to mlx5_pci driver. + * Mellanox ConnectX PCI device supports multiple class: net,vdpa,regex,compress + * and crypto devices. This layer enables creating such multiple class of + * devices on a single PCI device by allowing to bind multiple class specific + * device driver to attach to mlx5_pci driver. * - * ----------- ------------ ------------- ---------------- - * | mlx5 | | mlx5 | | mlx5 | | mlx5 | - * | net pmd | | vdpa pmd | | regex pmd | | compress pmd | - * ----------- ------------ ------------- ---------------- - * \ \ / / - * \ \ / / + * -------- -------- --------- ------------ ---------- + * | mlx5 | | mlx5 | | mlx5 | | mlx5 | | mlx5 | + * | net | | vdpa | | regex | | compress | | crypto | + * | pmd | | pmd | | pmd | | pmd | | pmd | + * -------- -------- --------- ------------ ---------- + * \ \ | / / + * \ \ | / / * \ \_--------------_/ / * \_______________| mlx5 |_______________/ * | pci common | diff --git a/drivers/crypto/meson.build b/drivers/crypto/meson.build index b9fdf9392f..6951607def 100644 --- a/drivers/crypto/meson.build +++ b/drivers/crypto/meson.build @@ -15,6 +15,7 @@ drivers = [ 'dpaa_sec', 'dpaa2_sec', 'kasumi', + 'mlx5', 'mvsam', 'nitrox', 'null', diff --git a/drivers/crypto/mlx5/meson.build b/drivers/crypto/mlx5/meson.build new file mode 100644 index 0000000000..6fd70bc477 --- /dev/null +++ b/drivers/crypto/mlx5/meson.build @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2021 NVIDIA Corporation & Affiliates + +if not is_linux + build = false + reason = 'only supported on Linux' + subdir_done() +endif + +fmt_name = 'mlx5_crypto' +deps += ['common_mlx5', 'eal', 'cryptodev'] +sources = files( + 'mlx5_crypto.c', +) +cflags_options = [ + '-std=c11', + '-Wno-strict-prototypes', + '-D_BSD_SOURCE', + '-D_DEFAULT_SOURCE', + '-D_XOPEN_SOURCE=600', +] +foreach option:cflags_options + if cc.has_argument(option) + cflags += option + endif +endforeach diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c new file mode 100644 index 0000000000..f89d508a7e --- /dev/null +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -0,0 +1,275 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "mlx5_crypto_utils.h" + +#define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto +#define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5 + +#define MLX5_CRYPTO_FEATURE_FLAGS \ + RTE_CRYPTODEV_FF_HW_ACCELERATED + +struct mlx5_crypto_priv { + TAILQ_ENTRY(mlx5_crypto_priv) next; + struct ibv_context *ctx; /* Device context. */ + struct rte_pci_device *pci_dev; + struct rte_cryptodev *crypto_dev; + void *uar; /* User Access Region. */ + uint32_t pdn; /* Protection Domain number. */ + struct ibv_pd *pd; +}; + +TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list = + TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list); +static pthread_mutex_t priv_list_lock = PTHREAD_MUTEX_INITIALIZER; + +int mlx5_crypto_logtype; + +uint8_t mlx5_crypto_driver_id; + +static const char mlx5_crypto_drv_name[] = RTE_STR(MLX5_CRYPTO_DRIVER_NAME); + +static const struct rte_driver mlx5_drv = { + .name = mlx5_crypto_drv_name, + .alias = mlx5_crypto_drv_name +}; + +static struct cryptodev_driver mlx5_cryptodev_driver; + +static struct rte_cryptodev_ops mlx5_crypto_ops = { + .dev_configure = NULL, + .dev_start = NULL, + .dev_stop = NULL, + .dev_close = NULL, + .dev_infos_get = NULL, + .stats_get = NULL, + .stats_reset = NULL, + .queue_pair_setup = NULL, + .queue_pair_release = NULL, + .sym_session_get_size = NULL, + .sym_session_configure = NULL, + .sym_session_clear = NULL, + .sym_get_raw_dp_ctx_size = NULL, + .sym_configure_raw_dp_ctx = NULL, +}; + +static void +mlx5_crypto_hw_global_release(struct mlx5_crypto_priv *priv) +{ + if (priv->pd != NULL) { + claim_zero(mlx5_glue->dealloc_pd(priv->pd)); + priv->pd = NULL; + } + if (priv->uar != NULL) { + mlx5_glue->devx_free_uar(priv->uar); + priv->uar = NULL; + } +} + +static int +mlx5_crypto_pd_create(struct mlx5_crypto_priv *priv) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT + struct mlx5dv_obj obj; + struct mlx5dv_pd pd_info; + int ret; + + priv->pd = mlx5_glue->alloc_pd(priv->ctx); + if (priv->pd == NULL) { + DRV_LOG(ERR, "Failed to allocate PD."); + return errno ? -errno : -ENOMEM; + } + obj.pd.in = priv->pd; + obj.pd.out = &pd_info; + ret = mlx5_glue->dv_init_obj(&obj, MLX5DV_OBJ_PD); + if (ret != 0) { + DRV_LOG(ERR, "Fail to get PD object info."); + mlx5_glue->dealloc_pd(priv->pd); + priv->pd = NULL; + return -errno; + } + priv->pdn = pd_info.pdn; + return 0; +#else + (void)priv; + DRV_LOG(ERR, "Cannot get pdn - no DV support."); + return -ENOTSUP; +#endif /* HAVE_IBV_FLOW_DV_SUPPORT */ +} + +static int +mlx5_crypto_hw_global_prepare(struct mlx5_crypto_priv *priv) +{ + if (mlx5_crypto_pd_create(priv) != 0) + return -1; + priv->uar = mlx5_devx_alloc_uar(priv->ctx, -1); + if (priv->uar == NULL || mlx5_os_get_devx_uar_reg_addr(priv->uar) == + NULL) { + rte_errno = errno; + claim_zero(mlx5_glue->dealloc_pd(priv->pd)); + DRV_LOG(ERR, "Failed to allocate UAR."); + return -1; + } + return 0; +} + +/** + * DPDK callback to register a PCI device. + * + * This function spawns crypto device out of a given PCI device. + * + * @param[in] pci_drv + * PCI driver structure (mlx5_crypto_driver). + * @param[in] pci_dev + * PCI device information. + * + * @return + * 0 on success, 1 to skip this driver, a negative errno value otherwise + * and rte_errno is set. + */ +static int +mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct ibv_device *ibv; + struct rte_cryptodev *crypto_dev; + struct ibv_context *ctx; + struct mlx5_crypto_priv *priv; + struct mlx5_hca_attr attr = { 0 }; + struct rte_cryptodev_pmd_init_params init_params = { + .name = "", + .private_data_size = sizeof(struct mlx5_crypto_priv), + .socket_id = pci_dev->device.numa_node, + .max_nb_queue_pairs = + RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS, + }; + RTE_SET_USED(pci_drv); + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + DRV_LOG(ERR, "Non-primary process type is not supported."); + rte_errno = ENOTSUP; + return -rte_errno; + } + ibv = mlx5_os_get_ibv_device(&pci_dev->addr); + if (ibv == NULL) { + DRV_LOG(ERR, "No matching IB device for PCI slot " + PCI_PRI_FMT ".", pci_dev->addr.domain, + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + return -rte_errno; + } + DRV_LOG(INFO, "PCI information matches for device \"%s\".", ibv->name); + ctx = mlx5_glue->dv_open_device(ibv); + if (ctx == NULL) { + DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); + rte_errno = ENODEV; + return -rte_errno; + } + if (mlx5_devx_cmd_query_hca_attr(ctx, &attr) != 0 || + attr.crypto == 0 || attr.aes_xts == 0) { + DRV_LOG(ERR, "Not enough capabilities to support crypto " + "operations, maybe old FW/OFED version?"); + claim_zero(mlx5_glue->close_device(ctx)); + rte_errno = ENOTSUP; + return -ENOTSUP; + } + crypto_dev = rte_cryptodev_pmd_create(ibv->name, &pci_dev->device, + &init_params); + if (crypto_dev == NULL) { + DRV_LOG(ERR, "Failed to create device \"%s\".", ibv->name); + claim_zero(mlx5_glue->close_device(ctx)); + return -ENODEV; + } + DRV_LOG(INFO, + "Crypto device %s was created successfully.", ibv->name); + crypto_dev->dev_ops = &mlx5_crypto_ops; + crypto_dev->dequeue_burst = NULL; + crypto_dev->enqueue_burst = NULL; + crypto_dev->feature_flags = MLX5_CRYPTO_FEATURE_FLAGS; + crypto_dev->driver_id = mlx5_crypto_driver_id; + priv = crypto_dev->data->dev_private; + priv->ctx = ctx; + priv->pci_dev = pci_dev; + priv->crypto_dev = crypto_dev; + if (mlx5_crypto_hw_global_prepare(priv) != 0) { + rte_cryptodev_pmd_destroy(priv->crypto_dev); + claim_zero(mlx5_glue->close_device(priv->ctx)); + return -1; + } + pthread_mutex_lock(&priv_list_lock); + TAILQ_INSERT_TAIL(&mlx5_crypto_priv_list, priv, next); + pthread_mutex_unlock(&priv_list_lock); + return 0; +} + +static int +mlx5_crypto_pci_remove(struct rte_pci_device *pdev) +{ + struct mlx5_crypto_priv *priv = NULL; + + pthread_mutex_lock(&priv_list_lock); + TAILQ_FOREACH(priv, &mlx5_crypto_priv_list, next) + if (rte_pci_addr_cmp(&priv->pci_dev->addr, &pdev->addr) != 0) + break; + if (priv) + TAILQ_REMOVE(&mlx5_crypto_priv_list, priv, next); + pthread_mutex_unlock(&priv_list_lock); + if (priv) { + mlx5_crypto_hw_global_release(priv); + rte_cryptodev_pmd_destroy(priv->crypto_dev); + claim_zero(mlx5_glue->close_device(priv->ctx)); + } + return 0; +} + +static const struct rte_pci_id mlx5_crypto_pci_id_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6) + }, + { + .vendor_id = 0 + } + }; + +static struct mlx5_pci_driver mlx5_crypto_driver = { + .driver_class = MLX5_CLASS_CRYPTO, + .pci_driver = { + .driver = { + .name = RTE_STR(MLX5_CRYPTO_DRIVER_NAME), + }, + .id_table = mlx5_crypto_pci_id_map, + .probe = mlx5_crypto_pci_probe, + .remove = mlx5_crypto_pci_remove, + .drv_flags = 0, + }, +}; + +RTE_INIT(rte_mlx5_crypto_init) +{ + mlx5_common_init(); + if (mlx5_glue != NULL) + mlx5_pci_driver_register(&mlx5_crypto_driver); +} + +RTE_PMD_REGISTER_CRYPTO_DRIVER(mlx5_cryptodev_driver, mlx5_drv, + mlx5_crypto_driver_id); + +RTE_LOG_REGISTER(mlx5_crypto_logtype, MLX5_CRYPTO_LOG_NAME, NOTICE) +RTE_PMD_EXPORT_NAME(MLX5_CRYPTO_DRIVER_NAME, __COUNTER__); +RTE_PMD_REGISTER_PCI_TABLE(MLX5_CRYPTO_DRIVER_NAME, mlx5_crypto_pci_id_map); +RTE_PMD_REGISTER_KMOD_DEP(MLX5_CRYPTO_DRIVER_NAME, "* ib_uverbs & mlx5_core & mlx5_ib"); diff --git a/drivers/crypto/mlx5/mlx5_crypto_utils.h b/drivers/crypto/mlx5/mlx5_crypto_utils.h new file mode 100644 index 0000000000..b6c60ca782 --- /dev/null +++ b/drivers/crypto/mlx5/mlx5_crypto_utils.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#ifndef RTE_PMD_MLX5_CRYPTO_UTILS_H_ +#define RTE_PMD_MLX5_CRYPTO_UTILS_H_ + +#include + +extern int mlx5_crypto_logtype; + +#define MLX5_CRYPTO_LOG_PREFIX "mlx5_crypto" +/* Generic printf()-like logging macro with automatic line feed. */ +#define DRV_LOG(level, ...) \ + PMD_DRV_LOG_(level, mlx5_crypto_logtype, MLX5_CRYPTO_LOG_PREFIX, \ + __VA_ARGS__ PMD_DRV_LOG_STRIP PMD_DRV_LOG_OPAREN, \ + PMD_DRV_LOG_CPAREN) + +#endif /* RTE_PMD_MLX5_CRYPTO_UTILS_H_ */ diff --git a/drivers/crypto/mlx5/version.map b/drivers/crypto/mlx5/version.map new file mode 100644 index 0000000000..4a76d1d52d --- /dev/null +++ b/drivers/crypto/mlx5/version.map @@ -0,0 +1,3 @@ +DPDK_21 { + local: *; +}; From patchwork Thu Jul 1 13:25:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95133 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 674DEA0A0C; Thu, 1 Jul 2021 15:27:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 514F6412DD; Thu, 1 Jul 2021 15:27:13 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2053.outbound.protection.outlook.com [40.107.223.53]) by mails.dpdk.org (Postfix) with ESMTP id 636F2412DD for ; Thu, 1 Jul 2021 15:27:11 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J6rwxCntpZWFzN56p/CTUACoar4XcOnVMnRPcQLulwDRKwuY5pl7orDN15P8kVGwFYvqe1krXA6ypOB/nN7rlrRbmUVjljZiM5hAvJMfTTd1H5prc6ScIw+ztwjsqL5XUFwhbbFntUXB9B+zf1/6DovuOhLeA+R67EYtllna6ORcKFhf6PiU+1ndhbu0z7+qSw9ztsVi3M/p3FougXt3Svt7m1rnkCwIpk0EYB+V71VCVDHe/w6bdoopPciv3FjDhzgwp51hufyJqxAP6sjAU4RwjBa/xUUScMTY8YFpa2yMwBGoaRDAOQuoAIdryzvnHjmLDgG55ozuEABwwEwSMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RrCYRd50edBwH5r+h0x17a1wjmEb4eEioA/v3XZe9xw=; b=XPsqvDV0q5vFMBDmsxNiGpba1fFG57imhL1GcOgwAgMdbyt8FMGAR+39JL5dc6ryuCvtaQnYPkBDh2idcRoOl7w7YlXEgpb1aebaKHSjmxNIF9aycchQQtRoHDLPa0Heb6vDq9cV4yoIzkXVrlx7vgICY6ni0dGzIpquynrQ1uxM6JcR1TdholvZLHSngYJr8HXBoG6lCxd+cpR9KwOeGvMjHuhQBLoSU0htgmoEUreQDMV7G6T2cxY95qi36jh+Xv5oy2qpfMtNGZ9jkF57a9tbafodXtnBfz+kpf7fkMt9xnwX5Cp3QAC1nZphGm7ZQf/01L+cqfbpNAeXBfk7CQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RrCYRd50edBwH5r+h0x17a1wjmEb4eEioA/v3XZe9xw=; b=hyASL315hfiPBJnXuQPdu0Ki3++NEQ76WV89OuyduV66Uuawz57nmxR6wT2y5N157gjNc1GMChcWYfbLkcb2COUK0FEDIWHVbFOJECcXjUru2PrIqCVESz7avaH43NQfn6EPgxi7iVtDsT5XQiL/aFn5otIq4BLKSW/GtSXUFBNEFih2R4p0R75XwkJl80s/zpHRhAgwqj+gqdSFPHK1GboaCS/3VJV2ngLvbJ2t1xGKiWPwy+LgQeqhpgqNqVvH2kXS1a36e/BcUd6fs5aVBpI5GJF8vyASAvpD47RFsmBwm4ed0x97408CT0QXmMVtH7hFtqEwjfRuTSF9hNB6BA== Received: from BN8PR12CA0004.namprd12.prod.outlook.com (2603:10b6:408:60::17) by BN8PR12MB3170.namprd12.prod.outlook.com (2603:10b6:408:98::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.23; Thu, 1 Jul 2021 13:26:56 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::97) by BN8PR12CA0004.outlook.office365.com (2603:10b6:408:60::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:26:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:26:56 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:26:53 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:25:56 +0300 Message-ID: <20210701132609.53727-3-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b5ffb3a2-a45e-46fb-3399-08d93c93e5ef X-MS-TrafficTypeDiagnostic: BN8PR12MB3170: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:949; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uOrKIdha8CbDIfIxC5v3HoBOdxdBKDN3Hf8bAvqLE5Z+ZXCSmI7MyInFusrcv/oqVmEBH/q9TWBzMttbo8tWS+eJjuvaBOO87AYWKtnB5L0ArLJm6Ii5m1wSirp+RBX5ZTwroHvpNZaj7Ry98cw+LbZTmfVT4VoEgo+R7jtBmwnmBdvLb5raLznhIcyCrsJU12emovKLWR0BwFCwc9OddhJXWQA3hm8Xltt8aZBGpIH3IO3+heYXYjXdv7eylgN69Gzt2t0l/HANPdyba3/ikti9i8aLItNlix+9z84nmEetKCpwxNh0gHVbYC/TIHd2+lBZnCltINipuFG6uV8PgBFR+O8aib0Lvv1epQ/M47roY0RHEsYI+xnuiHKJUdTtqaiMqa2+2OOadvLHQDwD9XB6sdeztPJCFD8qn48Z5dsOGlbtZZW61N22Pck3VzCRprW3ThjxjuK9VBVjK6ddp56Kisv0URR47LD4LKR7r/OWq8vQagHuv5am4Em6IuZveBSb/UmK8ThzrZSEWnxXjjvOlLjQfnPDMhe2258qJN4ScHwI0p2bstqU2bu58BaQ1gaGXmEvs+dUkIBljrH7/6A9KfyZ1esq7VkIXpMbaiR/uo5IMYMyXUqRnhHkYt2p//GeG+QwaUhJcPIuE64NBK7+a5cUCXxR7sCtV5WFC3KWkD64/vU15IP5kFBACJ0cva4oYA4/JTdJw05tbrH/DA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(39860400002)(136003)(396003)(346002)(46966006)(36840700001)(4326008)(83380400001)(54906003)(336012)(426003)(36860700001)(2616005)(8676002)(2906002)(316002)(8936002)(26005)(6916009)(36756003)(107886003)(82740400003)(70206006)(5660300002)(86362001)(1076003)(55016002)(7636003)(47076005)(82310400003)(356005)(186003)(6286002)(478600001)(7696005)(16526019)(70586007); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:26:56.3364 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b5ffb3a2-a45e-46fb-3399-08d93c93e5ef X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB3170 Subject: [dpdk-dev] [PATCH v5 02/15] crypto/mlx5: add DEK object management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" A DEK(Data encryption Key) is an mlx5 HW object which represents the cipher algorithm key. The DEKs are used during data encryption/decryption operations. In symmetric algorithms like AES-STS, we use the same DEK for both encryption and decryption. Use the mlx5 hash-list tool to manage the DEK objects in the PMD. Provide the compare, create and destroy functions to manage DEKs in hash-list and introduce an internal API to setup and unset the DEK management and to prepare and destroy specific DEK object. The DEK hash-list will be created in dev_configure routine and destroyed in dev_close routine. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- drivers/crypto/mlx5/meson.build | 1 + drivers/crypto/mlx5/mlx5_crypto.c | 42 +++++--- drivers/crypto/mlx5/mlx5_crypto.h | 51 ++++++++++ drivers/crypto/mlx5/mlx5_crypto_dek.c | 136 ++++++++++++++++++++++++++ 4 files changed, 214 insertions(+), 16 deletions(-) create mode 100644 drivers/crypto/mlx5/mlx5_crypto.h create mode 100644 drivers/crypto/mlx5/mlx5_crypto_dek.c diff --git a/drivers/crypto/mlx5/meson.build b/drivers/crypto/mlx5/meson.build index 6fd70bc477..d55cdbfe6f 100644 --- a/drivers/crypto/mlx5/meson.build +++ b/drivers/crypto/mlx5/meson.build @@ -11,6 +11,7 @@ fmt_name = 'mlx5_crypto' deps += ['common_mlx5', 'eal', 'cryptodev'] sources = files( 'mlx5_crypto.c', + 'mlx5_crypto_dek.c', ) cflags_options = [ '-std=c11', diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index f89d508a7e..dfc3da3d3f 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -3,12 +3,9 @@ */ #include -#include #include +#include #include -#include -#include -#include #include #include @@ -17,6 +14,7 @@ #include #include "mlx5_crypto_utils.h" +#include "mlx5_crypto.h" #define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto #define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5 @@ -24,16 +22,6 @@ #define MLX5_CRYPTO_FEATURE_FLAGS \ RTE_CRYPTODEV_FF_HW_ACCELERATED -struct mlx5_crypto_priv { - TAILQ_ENTRY(mlx5_crypto_priv) next; - struct ibv_context *ctx; /* Device context. */ - struct rte_pci_device *pci_dev; - struct rte_cryptodev *crypto_dev; - void *uar; /* User Access Region. */ - uint32_t pdn; /* Protection Domain number. */ - struct ibv_pd *pd; -}; - TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list = TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list); static pthread_mutex_t priv_list_lock = PTHREAD_MUTEX_INITIALIZER; @@ -51,11 +39,33 @@ static const struct rte_driver mlx5_drv = { static struct cryptodev_driver mlx5_cryptodev_driver; +static int +mlx5_crypto_dev_configure(struct rte_cryptodev *dev, + struct rte_cryptodev_config *config __rte_unused) +{ + struct mlx5_crypto_priv *priv = dev->data->dev_private; + + if (mlx5_crypto_dek_setup(priv) != 0) { + DRV_LOG(ERR, "Dek hash list creation has failed."); + return -ENOMEM; + } + return 0; +} + +static int +mlx5_crypto_dev_close(struct rte_cryptodev *dev) +{ + struct mlx5_crypto_priv *priv = dev->data->dev_private; + + mlx5_crypto_dek_unset(priv); + return 0; +} + static struct rte_cryptodev_ops mlx5_crypto_ops = { - .dev_configure = NULL, + .dev_configure = mlx5_crypto_dev_configure, .dev_start = NULL, .dev_stop = NULL, - .dev_close = NULL, + .dev_close = mlx5_crypto_dev_close, .dev_infos_get = NULL, .stats_get = NULL, .stats_reset = NULL, diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h new file mode 100644 index 0000000000..5a54cb0dca --- /dev/null +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#ifndef MLX5_CRYPTO_H_ +#define MLX5_CRYPTO_H_ + +#include + +#include +#include + +#include + +#define MLX5_CRYPTO_DEK_HTABLE_SZ (1 << 11) +#define MLX5_CRYPTO_KEY_LENGTH 80 + +struct mlx5_crypto_priv { + TAILQ_ENTRY(mlx5_crypto_priv) next; + struct ibv_context *ctx; /* Device context. */ + struct rte_pci_device *pci_dev; + struct rte_cryptodev *crypto_dev; + void *uar; /* User Access Region. */ + uint32_t pdn; /* Protection Domain number. */ + struct ibv_pd *pd; + struct mlx5_hlist *dek_hlist; /* Dek hash list. */ +}; + +struct mlx5_crypto_dek { + struct mlx5_hlist_entry entry; /* Pointer to DEK hash list entry. */ + struct mlx5_devx_obj *obj; /* Pointer to DEK DevX object. */ + uint8_t data[MLX5_CRYPTO_KEY_LENGTH]; /* DEK key data. */ + bool size_is_48; /* Whether the key\data size is 48 bytes or not. */ +} __rte_cache_aligned; + +int +mlx5_crypto_dek_destroy(struct mlx5_crypto_priv *priv, + struct mlx5_crypto_dek *dek); + +struct mlx5_crypto_dek * +mlx5_crypto_dek_prepare(struct mlx5_crypto_priv *priv, + struct rte_crypto_cipher_xform *cipher); + +int +mlx5_crypto_dek_setup(struct mlx5_crypto_priv *priv); + +void +mlx5_crypto_dek_unset(struct mlx5_crypto_priv *priv); + +#endif /* MLX5_CRYPTO_H_ */ + diff --git a/drivers/crypto/mlx5/mlx5_crypto_dek.c b/drivers/crypto/mlx5/mlx5_crypto_dek.c new file mode 100644 index 0000000000..e09c20c844 --- /dev/null +++ b/drivers/crypto/mlx5/mlx5_crypto_dek.c @@ -0,0 +1,136 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#include +#include +#include +#include + +#include +#include + +#include "mlx5_crypto_utils.h" +#include "mlx5_crypto.h" + +struct mlx5_crypto_dek_ctx { + struct rte_crypto_cipher_xform *cipher; + struct mlx5_crypto_priv *priv; +}; + +int +mlx5_crypto_dek_destroy(struct mlx5_crypto_priv *priv, + struct mlx5_crypto_dek *dek) +{ + return mlx5_hlist_unregister(priv->dek_hlist, &dek->entry); +} + +struct mlx5_crypto_dek * +mlx5_crypto_dek_prepare(struct mlx5_crypto_priv *priv, + struct rte_crypto_cipher_xform *cipher) +{ + struct mlx5_hlist *dek_hlist = priv->dek_hlist; + struct mlx5_crypto_dek_ctx dek_ctx = { + .cipher = cipher, + .priv = priv, + }; + struct rte_crypto_cipher_xform *cipher_ctx = cipher; + uint64_t key64 = __rte_raw_cksum(cipher_ctx->key.data, + cipher_ctx->key.length, 0); + struct mlx5_hlist_entry *entry = mlx5_hlist_register(dek_hlist, + key64, &dek_ctx); + + return entry == NULL ? NULL : + container_of(entry, struct mlx5_crypto_dek, entry); +} + +static int +mlx5_crypto_dek_match_cb(struct mlx5_hlist *list __rte_unused, + struct mlx5_hlist_entry *entry, + uint64_t key __rte_unused, void *cb_ctx) +{ + struct mlx5_crypto_dek_ctx *ctx = cb_ctx; + struct rte_crypto_cipher_xform *cipher_ctx = ctx->cipher; + struct mlx5_crypto_dek *dek = + container_of(entry, typeof(*dek), entry); + uint32_t key_len = dek->size_is_48 ? 48 : 80; + + if (key_len != cipher_ctx->key.length) + return -1; + return memcmp(cipher_ctx->key.data, dek->data, key_len); +} + +static struct mlx5_hlist_entry * +mlx5_crypto_dek_create_cb(struct mlx5_hlist *list __rte_unused, + uint64_t key __rte_unused, void *cb_ctx) +{ + struct mlx5_crypto_dek_ctx *ctx = cb_ctx; + struct rte_crypto_cipher_xform *cipher_ctx = ctx->cipher; + struct mlx5_crypto_dek *dek = rte_zmalloc(__func__, sizeof(*dek), + RTE_CACHE_LINE_SIZE); + struct mlx5_devx_dek_attr dek_attr = { + .pd = ctx->priv->pdn, + .key_purpose = MLX5_CRYPTO_KEY_PURPOSE_AES_XTS, + .has_keytag = 1, + }; + + if (dek == NULL) { + DRV_LOG(ERR, "Failed to allocate dek memory."); + return NULL; + } + switch (cipher_ctx->key.length) { + case 48: + dek->size_is_48 = true; + dek_attr.key_size = MLX5_CRYPTO_KEY_SIZE_128b; + break; + case 80: + dek->size_is_48 = false; + dek_attr.key_size = MLX5_CRYPTO_KEY_SIZE_256b; + break; + default: + DRV_LOG(ERR, "Key size not supported."); + return NULL; + } + memcpy(&dek_attr.key, cipher_ctx->key.data, cipher_ctx->key.length); + dek->obj = mlx5_devx_cmd_create_dek_obj(ctx->priv->ctx, &dek_attr); + if (dek->obj == NULL) { + rte_free(dek); + return NULL; + } + memcpy(&dek->data, cipher_ctx->key.data, cipher_ctx->key.length); + return &dek->entry; +} + +static void +mlx5_crypto_dek_remove_cb(struct mlx5_hlist *list __rte_unused, + struct mlx5_hlist_entry *entry) +{ + struct mlx5_crypto_dek *dek = + container_of(entry, typeof(*dek), entry); + + claim_zero(mlx5_devx_cmd_destroy(dek->obj)); + rte_free(dek); +} + + +int +mlx5_crypto_dek_setup(struct mlx5_crypto_priv *priv) +{ + priv->dek_hlist = mlx5_hlist_create("dek_hlist", + MLX5_CRYPTO_DEK_HTABLE_SZ, + 0, MLX5_HLIST_WRITE_MOST | + MLX5_HLIST_DIRECT_KEY, + mlx5_crypto_dek_create_cb, + mlx5_crypto_dek_match_cb, + mlx5_crypto_dek_remove_cb); + if (priv->dek_hlist == NULL) + return -1; + return 0; +} + +void +mlx5_crypto_dek_unset(struct mlx5_crypto_priv *priv) +{ + mlx5_hlist_destroy(priv->dek_hlist); + priv->dek_hlist = NULL; +} From patchwork Thu Jul 1 13:25:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95134 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A6693A0A0C; Thu, 1 Jul 2021 15:27:18 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 77E6E412F4; Thu, 1 Jul 2021 15:27:14 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2088.outbound.protection.outlook.com [40.107.244.88]) by mails.dpdk.org (Postfix) with ESMTP id 8A172412DE for ; Thu, 1 Jul 2021 15:27:11 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fmFkVeRHSBkj2jHj+cikKpy84u/xZPg/BHGiPFn7pU8DQywxn/hhNFbWeJVk5AONgWktwKwlUs44cH3DLuNIstdHdx2z62iOWMPafKeX2j9DDyiBAu94baaCG3OiW8ONS2ARBTo9CuC3VRS08j3zyqvvHKjOJNHE69kF1Ydox7rrnyDs3IP79k/GpU7+MQEoksmuuO/I3ENz7Ho45DEEUqD25dhCxBh8e6j3t9xZBZfg0QGNiSD9Hk3xGyMFcDvGy+U9tAKDnHEA+Yl2om/AZp3U3l/H11MUrGh8qpMyJ3ZScWagSAOQMPHj9Cg2FmPHJBL4Pa30XibJL93Zv3ik2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W1AS0c3sS+vyRznlEk1pedm38BFzChobf7E9yrh3oHI=; b=WrDUN5AVLKD/ILwKDcYpwa6Wy1tJDAzv5/gqpSacK7ch6dpsiHxx+PP2zN5bIq3zPWXxuS+DyOV/KRl9IFmEaw8IcujNBvT6IYZCifc7MnBUGD62TC7ujV3cNWvlBzv54+ak9FJgulSEkSdlS/T8C4YJ7ld/2BloP0HUveidhks7EoH27KfW4rUNFDGK89UbFQwtg8adgz1m7a6gQHZVqKysWdZLNEXAZ7dp1fbMUlKncwuvn3PFUBXv/1JvINW44ADqhPg9PDe4ZO3qJXQ2iK8b4w8urb8vJ12+sdZrJXpt1B/pkt+XUsHbsHCpT4vBhphVAv1/s/4u6K2ZQEyHZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W1AS0c3sS+vyRznlEk1pedm38BFzChobf7E9yrh3oHI=; b=uTbymbFf349nWLV7v/apOG1Q7mtBQOsf9lXT+0SEuDAGAzRzmMpwmUHr3a7rVAckzdnmb2acI/8QX+WJeNqw6UF8VP8wzRYKUcbF9EcoNI+zdMxt/mfHOQNV1pqhuseD1d393q+RdtVhTlOsmJAuuOjCJjswzG9yVzeYEGf3/CrHs6kjo94jY3/F/hxxt17KghanJqNpB3gV0nYW4MGu4DiEmzZ7o7ekTOLNhqvG7VA7FfQXWyHGWg6T7fcQj61iknfGbyvdF7TW9d5xG7rK8bGLoVIya3ZxlWhkonZudv/Iy+1s8x6zNhySbSKpGqpHeYsMzFiiNlTXxlpePvESqA== Received: from BN8PR12CA0028.namprd12.prod.outlook.com (2603:10b6:408:60::41) by MN2PR12MB4567.namprd12.prod.outlook.com (2603:10b6:208:263::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.20; Thu, 1 Jul 2021 13:27:10 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::8a) by BN8PR12CA0028.outlook.office365.com (2603:10b6:408:60::41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:09 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:26:54 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:25:57 +0300 Message-ID: <20210701132609.53727-4-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a37b9ebf-74dc-46df-32b5-08d93c93edfb X-MS-TrafficTypeDiagnostic: MN2PR12MB4567: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: apv8LG/l+AMViGyOyMPEkULo8C6mUQP20AQzbjOwbG6JLr49O39OxZ3taM4SsFYjMF+A9dQWlh+Iqii2eojK1n08v050JF9G9gs4t6wlSsCTbNFGfcZ88il2u4ADt+Rp9Jt3Aud1S/h/n9iBJ1i5Mqlt+M5ROTwiln1nxQRlSlbUEeKXAhBvAWsrCqP3KSU5RqyQGrta2hI3b7ZZhUkmb0ExQvrdwZdiL3ANcvddX3djN98WOVYMSfG0CYlBuj42gzXzH9B49trYs+wCmDaoT8tu99LG3sxCAo9E9WnlGm3Tv5WCrdqpR9qwYtfM11MPOJcnqEqTttdcfCxDfeH3/g3HMAdTDwok9M/rNSObMimOZyB2FsArbLbJZtDvdzF96zpBx6j99RQM5qUgz+CZbML2iN6bmW/Owtf+dQNDzk6AIFnt3csxmWWa6OB4mhIhCTrKM3WTyTx08/cYwzEY9SBRIi7HVZzHCyzU6WvPv+4PWYpmCVbTs1B27K6fc72WBts4TL6tyjsj8UHdV/HkmNo+lwW6g3V1vwD6j478NzWZ6mWLuvp2pKAA/jeISt/VVxnqACi3myiJsL+uz9eFt2MHVQYEuovDj5/CN5PEGJTote7iJ3tUfmjFFsLG+EK1XsyRrqDxfdJBPsLQVHL0+pbO/O2kfJvgeHrB4TIsKsCsw7OrD11BY6/3DEwCC5RgyNlsKvjv/HNqVTF+GymBpA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(376002)(136003)(346002)(39860400002)(46966006)(36840700001)(54906003)(6916009)(36906005)(316002)(26005)(16526019)(70586007)(5660300002)(4326008)(186003)(2906002)(7696005)(336012)(426003)(2616005)(107886003)(478600001)(1076003)(6286002)(36756003)(86362001)(8676002)(83380400001)(7636003)(356005)(82310400003)(70206006)(36860700001)(82740400003)(47076005)(8936002)(55016002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:09.8358 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a37b9ebf-74dc-46df-32b5-08d93c93edfb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4567 Subject: [dpdk-dev] [PATCH v5 03/15] crypto/mlx5: add session operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Sessions are used in symmetric transformations in order to prepare objects and data for packet processing stage. A mlx5 session includes iv_offset, pointer to mlx5_crypto_dek struct, bsf_size, bsf_p_type, block size index, encryption_order and encryption standard. Implement the next session operations: mlx5_crypto_sym_session_get_size- returns the size of the mlx5 session struct. mlx5_crypto_sym_session_configure- prepares the DEK hash-list and saves all the session data. mlx5_crypto_sym_session_clear - destroys the DEK hash-list. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- doc/guides/cryptodevs/features/mlx5.ini | 5 + doc/guides/cryptodevs/mlx5.rst | 10 ++ drivers/crypto/mlx5/mlx5_crypto.c | 172 +++++++++++++++++++++++- 3 files changed, 182 insertions(+), 5 deletions(-) diff --git a/doc/guides/cryptodevs/features/mlx5.ini b/doc/guides/cryptodevs/features/mlx5.ini index ceadd967b6..bd757b5211 100644 --- a/doc/guides/cryptodevs/features/mlx5.ini +++ b/doc/guides/cryptodevs/features/mlx5.ini @@ -4,12 +4,17 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Symmetric crypto = Y HW Accelerated = Y +Cipher multiple data units = Y +Cipher wrapped key = Y ; ; Supported crypto algorithms of a mlx5 crypto driver. ; [Cipher] +AES XTS (128) = Y +AES XTS (256) = Y ; ; Supported authentication algorithms of a mlx5 crypto driver. diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst index 05a0a449e2..dd1d1a615d 100644 --- a/doc/guides/cryptodevs/mlx5.rst +++ b/doc/guides/cryptodevs/mlx5.rst @@ -53,6 +53,16 @@ Supported NICs * Mellanox\ |reg| ConnectX\ |reg|-6 200G MCX654106A-HCAT (2x200G) + +Limitations +----------- + +- AES-XTS keys provided in xform must include keytag and should be wrappend. +- The supported data-unit lengths are 512B and 1KB. In case the `dataunit_len` + is not provided in the cipher xform, the OP length is limited to the above + values and 1MB. + + Prerequisites ------------- diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index dfc3da3d3f..dd9685dcad 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -3,6 +3,7 @@ */ #include +#include #include #include #include @@ -20,7 +21,9 @@ #define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5 #define MLX5_CRYPTO_FEATURE_FLAGS \ - RTE_CRYPTODEV_FF_HW_ACCELERATED + (RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | \ + RTE_CRYPTODEV_FF_CIPHER_WRAPPED_KEY | \ + RTE_CRYPTODEV_FF_CIPHER_MULTIPLE_DATA_UNITS) TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list = TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list); @@ -30,6 +33,32 @@ int mlx5_crypto_logtype; uint8_t mlx5_crypto_driver_id; +const struct rte_cryptodev_capabilities mlx5_crypto_caps[] = { + { /* AES XTS */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_XTS, + .block_size = 16, + .key_size = { + .min = 32, + .max = 64, + .increment = 32 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .dataunit_set = + RTE_CRYPTO_CIPHER_DATA_UNIT_LEN_512_BYTES | + RTE_CRYPTO_CIPHER_DATA_UNIT_LEN_4096_BYTES, + }, } + }, } + }, +}; + static const char mlx5_crypto_drv_name[] = RTE_STR(MLX5_CRYPTO_DRIVER_NAME); static const struct rte_driver mlx5_drv = { @@ -39,6 +68,49 @@ static const struct rte_driver mlx5_drv = { static struct cryptodev_driver mlx5_cryptodev_driver; +struct mlx5_crypto_session { + uint32_t bs_bpt_eo_es; + /* + * bsf_size, bsf_p_type, encryption_order and encryption standard, + * saved in big endian format. + */ + uint32_t bsp_res; + /* + * crypto_block_size_pointer and reserved 24 bits saved in big endian + * format. + */ + uint32_t iv_offset:16; + /* Starting point for Initialisation Vector. */ + struct mlx5_crypto_dek *dek; /* Pointer to dek struct. */ + uint32_t dek_id; /* DEK ID */ +} __rte_packed; + +static void +mlx5_crypto_dev_infos_get(struct rte_cryptodev *dev, + struct rte_cryptodev_info *dev_info) +{ + RTE_SET_USED(dev); + if (dev_info != NULL) { + dev_info->driver_id = mlx5_crypto_driver_id; + dev_info->feature_flags = MLX5_CRYPTO_FEATURE_FLAGS; + dev_info->capabilities = mlx5_crypto_caps; + dev_info->max_nb_queue_pairs = 0; + dev_info->min_mbuf_headroom_req = 0; + dev_info->min_mbuf_tailroom_req = 0; + dev_info->sym.max_nb_sessions = 0; + /* + * If 0, the device does not have any limitation in number of + * sessions that can be used. + */ + } +} + +static unsigned int +mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) +{ + return sizeof(struct mlx5_crypto_session); +} + static int mlx5_crypto_dev_configure(struct rte_cryptodev *dev, struct rte_cryptodev_config *config __rte_unused) @@ -61,19 +133,109 @@ mlx5_crypto_dev_close(struct rte_cryptodev *dev) return 0; } +static int +mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, + struct rte_cryptodev_sym_session *session, + struct rte_mempool *mp) +{ + struct mlx5_crypto_priv *priv = dev->data->dev_private; + struct mlx5_crypto_session *sess_private_data; + struct rte_crypto_cipher_xform *cipher; + uint8_t encryption_order; + int ret; + + if (unlikely(xform->next != NULL)) { + DRV_LOG(ERR, "Xform next is not supported."); + return -ENOTSUP; + } + if (unlikely((xform->type != RTE_CRYPTO_SYM_XFORM_CIPHER) || + (xform->cipher.algo != RTE_CRYPTO_CIPHER_AES_XTS))) { + DRV_LOG(ERR, "Only AES-XTS algorithm is supported."); + return -ENOTSUP; + } + ret = rte_mempool_get(mp, (void *)&sess_private_data); + if (ret != 0) { + DRV_LOG(ERR, + "Failed to get session %p private data from mempool.", + sess_private_data); + return -ENOMEM; + } + cipher = &xform->cipher; + sess_private_data->dek = mlx5_crypto_dek_prepare(priv, cipher); + if (sess_private_data->dek == NULL) { + rte_mempool_put(mp, sess_private_data); + DRV_LOG(ERR, "Failed to prepare dek."); + return -ENOMEM; + } + if (cipher->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) + encryption_order = MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_MEMORY; + else + encryption_order = MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_WIRE; + sess_private_data->bs_bpt_eo_es = rte_cpu_to_be_32 + (MLX5_BSF_SIZE_64B << MLX5_BSF_SIZE_OFFSET | + MLX5_BSF_P_TYPE_CRYPTO << MLX5_BSF_P_TYPE_OFFSET | + encryption_order << MLX5_ENCRYPTION_ORDER_OFFSET | + MLX5_ENCRYPTION_STANDARD_AES_XTS); + switch (xform->cipher.dataunit_len) { + case 0: + sess_private_data->bsp_res = 0; + break; + case 512: + sess_private_data->bsp_res = rte_cpu_to_be_32 + ((uint32_t)MLX5_BLOCK_SIZE_512B << + MLX5_BLOCK_SIZE_OFFSET); + break; + case 4096: + sess_private_data->bsp_res = rte_cpu_to_be_32 + ((uint32_t)MLX5_BLOCK_SIZE_4096B << + MLX5_BLOCK_SIZE_OFFSET); + break; + default: + DRV_LOG(ERR, "Cipher data unit length is not supported."); + return -ENOTSUP; + } + sess_private_data->iv_offset = cipher->iv.offset; + sess_private_data->dek_id = + rte_cpu_to_be_32(sess_private_data->dek->obj->id & + 0xffffff); + set_sym_session_private_data(session, dev->driver_id, + sess_private_data); + DRV_LOG(DEBUG, "Session %p was configured.", sess_private_data); + return 0; +} + +static void +mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev, + struct rte_cryptodev_sym_session *sess) +{ + struct mlx5_crypto_priv *priv = dev->data->dev_private; + struct mlx5_crypto_session *spriv = get_sym_session_private_data(sess, + dev->driver_id); + + if (unlikely(spriv == NULL)) { + DRV_LOG(ERR, "Failed to get session %p private data.", spriv); + return; + } + mlx5_crypto_dek_destroy(priv, spriv->dek); + set_sym_session_private_data(sess, dev->driver_id, NULL); + rte_mempool_put(rte_mempool_from_obj(spriv), spriv); + DRV_LOG(DEBUG, "Session %p was cleared.", spriv); +} + static struct rte_cryptodev_ops mlx5_crypto_ops = { .dev_configure = mlx5_crypto_dev_configure, .dev_start = NULL, .dev_stop = NULL, .dev_close = mlx5_crypto_dev_close, - .dev_infos_get = NULL, + .dev_infos_get = mlx5_crypto_dev_infos_get, .stats_get = NULL, .stats_reset = NULL, .queue_pair_setup = NULL, .queue_pair_release = NULL, - .sym_session_get_size = NULL, - .sym_session_configure = NULL, - .sym_session_clear = NULL, + .sym_session_get_size = mlx5_crypto_sym_session_get_size, + .sym_session_configure = mlx5_crypto_sym_session_configure, + .sym_session_clear = mlx5_crypto_sym_session_clear, .sym_get_raw_dp_ctx_size = NULL, .sym_configure_raw_dp_ctx = NULL, }; From patchwork Thu Jul 1 13:25:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95135 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 07F7CA0A0C; Thu, 1 Jul 2021 15:27:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F18BC412FF; Thu, 1 Jul 2021 15:27:15 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2041.outbound.protection.outlook.com [40.107.94.41]) by mails.dpdk.org (Postfix) with ESMTP id CB3E5412D6 for ; Thu, 1 Jul 2021 15:27:12 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TKaIGLfxm339EwyPi7ertZzYc2rj8t8DLmaB8GzFquqySkK4T/ieIugUC/WtR84uv7+sLDEiwNaqfpvxaGFUHeFD/nty4gaSW47ymbxfUz0PQHH0YEAwniUriB4Rv0JGIu+3m2qhi1aMT0hhnKxpO4AvWeJOW24uIesLGUjIbSvKsoN9Ou5jzaL67TzuAYXwN8p1Z7TJXAvIHY1QBqLgezdS7qZ4foxUGCakbi0u4PduZ0/DF4w//7UrWxsnP1rX9YDXLiDI9n5cmlYTDRHpTN1eFSx330vQqb66cCJyFcr+SAYOZaFakcEm+feH8mS6emZhPHs3p3Owp5HJvb6/NA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8ps5eZZBnQIb3nEKbsrA+g39DNMuWkTfEmTQijJGhZc=; b=m5S7ztma2UATklvOgidurBghf3dHfaIdgHaQPSr/KPFvFGvEX272HORN1hg2O3+Hge38DAXBdJIAp7+ZFYlJasfrLLfad5s+60mVjsl6DjVPYG29MZkKc4+f61DHPWHvkl1YSOQ6M/evYo6t+ma8OCMzCNJQfklrRIiP0Iq63egvZnRtG8SpDJ91/fTAXM3BC5xp60NG9yOSp+LgzNBByVBzus3/A1tsPat0oQ2peVpRf+FPYxHaXGixfKsFHpXJ98YFSPiszEdpqBE0aUxMUby8kbXdJW4O/FpfwZrpaDYadzzNzY5e/x63j2M3w8ydiEmv6mfcRJ2Ss5WmayjYrw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8ps5eZZBnQIb3nEKbsrA+g39DNMuWkTfEmTQijJGhZc=; b=BzByeTNwV9BO34R7ho1kGuNN1zSGC0i1BzKZVqz/QgEv3pYzruCpf4qPFE/R8SnztAsnoLUma0eo9sbm+yAAO13O37VeAugyMTjCL8kARTaLFPW/yPRctRCi6lxrI4k0diZwy/SIvmi8WiYTaWIDzvdValmXXNLY4L6RHiw7HDZKN1mv9Jsgw0V3vDl8up1cKMj/cAVDHAf3x0jXF+oJfzzR/HWPox4Q8ZR4tHuLWLJrMgyg4ejFp3ebWMSTxeLdDj7uwQRVs50f9aCmX3HD0KBGdSf3hFLMjp3QAVlx2wkKJpKFf6rpfPipy++HvojsBuq39hNVvVYOvIIlf8R7jg== Received: from BN8PR12CA0002.namprd12.prod.outlook.com (2603:10b6:408:60::15) by DM8PR12MB5416.namprd12.prod.outlook.com (2603:10b6:8:28::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.19; Thu, 1 Jul 2021 13:27:11 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::97) by BN8PR12CA0002.outlook.office365.com (2603:10b6:408:60::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:11 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:26:56 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:25:58 +0300 Message-ID: <20210701132609.53727-5-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7d7d73e4-8ba7-44ad-6419-08d93c93eee2 X-MS-TrafficTypeDiagnostic: DM8PR12MB5416: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2150; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Sxe41mYpj8SnTGuR65Z1JYkabWzio7yIfVPxI8yy4vx4g2fo7NkxaMDXxfoCvJHU3UM5FG2uNTD8BVgvYnzzx4VtdUim8dRR3QCGwLO6ZIDNYVkwHsf6DmtJ2TMC6/ii4j2ZcFrdvxvtorvPcwTIsflydZJZL65SySWJQIB3XHxUdDkXvWEHs3zCU+qKIfZcO1myfRPs2C2MwR6Dpe3WLCHTKdu+jRyzHyUTUEOfuSsyOa7Fz8TbR4VzzhLhmBTLYh+D7Ret7oFYrb/gUV5psVb0wgcMx4dYZdn4XN3nKGBoNNpG6bxWb/DJ5wO+7QmKVIuIxFGc8q9uZyJh9dhktD63fVykKatprbXz7stkMBvVV1l6da/J0SuFXTFjk2DU0oAXdVyEKCPKha/0+HSE1P1XMAdT5X+YmIbKRRyBTwyFB85RoBOGHckZdE4u0U9HzFFSKfFUO+b5kE3r9hK37dDjDHTNgJWKDR/AC9df43d3QK2YrEtn5+erkRm7FvczE+/DeOg9KqXCb5q84J/KHRtJ1Y1KQoOIeyFc30fi3EhimkyMdnELPBtK7MqmQX1aktkQTO1YU+ccVgs5FKyLpgizIZSwVPeukUeR8hYUMxrrYeAvjUopw0HvDoYr7Eb6SyT2ZEpLL2DUKvOrEdt8CjiVTDg5gqbwWe3uzbp0gNlgLdkckLgrr5DFULxm/h1LyAsyG8C5HIgfwrg1Oa5sgA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39850400004)(136003)(396003)(346002)(376002)(36840700001)(46966006)(2616005)(83380400001)(478600001)(36906005)(54906003)(47076005)(1076003)(86362001)(7636003)(2906002)(316002)(356005)(55016002)(36756003)(336012)(26005)(70206006)(186003)(16526019)(8676002)(4326008)(426003)(6286002)(5660300002)(7696005)(70586007)(107886003)(6916009)(8936002)(82310400003)(82740400003)(36860700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:11.3470 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7d7d73e4-8ba7-44ad-6419-08d93c93eee2 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5416 Subject: [dpdk-dev] [PATCH v5 04/15] crypto/mlx5: add basic operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The basic dev control operations are configure, close and get info. Extended the existing support of configure and close: -mlx5_crypto_dev_configure- function used to configure device. -mlx5_crypto_dev_close- function used to close a configured device. Added config struct to user private data with the fields socket id, number of queue pairs and feature flags to be disabled. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- drivers/crypto/mlx5/mlx5_crypto.c | 26 +++++++++++++++++++------- drivers/crypto/mlx5/mlx5_crypto.h | 1 + 2 files changed, 20 insertions(+), 7 deletions(-) diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index dd9685dcad..0ce46ffd2f 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -105,22 +105,27 @@ mlx5_crypto_dev_infos_get(struct rte_cryptodev *dev, } } -static unsigned int -mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) -{ - return sizeof(struct mlx5_crypto_session); -} - static int mlx5_crypto_dev_configure(struct rte_cryptodev *dev, - struct rte_cryptodev_config *config __rte_unused) + struct rte_cryptodev_config *config) { struct mlx5_crypto_priv *priv = dev->data->dev_private; + if (config == NULL) { + DRV_LOG(ERR, "Invalid crypto dev configure parameters."); + return -EINVAL; + } + if ((config->ff_disable & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) != 0) { + DRV_LOG(ERR, + "Disabled symmetric crypto feature is not supported."); + return -ENOTSUP; + } if (mlx5_crypto_dek_setup(priv) != 0) { DRV_LOG(ERR, "Dek hash list creation has failed."); return -ENOMEM; } + priv->dev_config = *config; + DRV_LOG(DEBUG, "Device %u was configured.", dev->driver_id); return 0; } @@ -130,9 +135,16 @@ mlx5_crypto_dev_close(struct rte_cryptodev *dev) struct mlx5_crypto_priv *priv = dev->data->dev_private; mlx5_crypto_dek_unset(priv); + DRV_LOG(DEBUG, "Device %u was closed.", dev->driver_id); return 0; } +static unsigned int +mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) +{ + return sizeof(struct mlx5_crypto_session); +} + static int mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index 5a54cb0dca..4c07356028 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -24,6 +24,7 @@ struct mlx5_crypto_priv { uint32_t pdn; /* Protection Domain number. */ struct ibv_pd *pd; struct mlx5_hlist *dek_hlist; /* Dek hash list. */ + struct rte_cryptodev_config dev_config; }; struct mlx5_crypto_dek { From patchwork Thu Jul 1 13:25:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95136 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 58752A0A0C; Thu, 1 Jul 2021 15:27:33 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 323DA41303; Thu, 1 Jul 2021 15:27:17 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2084.outbound.protection.outlook.com [40.107.94.84]) by mails.dpdk.org (Postfix) with ESMTP id 8CB63412E9 for ; Thu, 1 Jul 2021 15:27:13 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JNxy95euCGuE3D5UNpDNvslxh/qjgz//W+nX3H7p6Vju/d5kitn5OF0OvO7CKhJrBC72HZWBMNSOaGH9Jh/0j2/6OQNDhatpBvJI5UTtmX9WmasbRRvjocnzpyU7OECc1MUz63IF+A5yQAwI7i8mvWFpgVH2gYkwCgBWWp2qhVIR/gt1kfbDhXTymLkKZO2DlZSW2fd7ve+DxElNLcQOpl4MDWxwpKgfdo9TiZrII5REJBVQ7oSwN/CNoxBzhbeU9NBSW8f8rD+u4a9XkcKpHySmSsAHQEibSSP8gW+CouGqNmxZfen88H3MhLduK7+T4f33/bpMdw3rAhyl/GuZAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+xCbH4eDchMlFs7nT2/QmDwZNqdAOohlZFwWkU3AmMA=; b=GLB5UGa7Ur+V2/m0aLJUR1PVWKslqAjURVrqnMN9RCAp9UbYwbMGnLmqLh/JL20fl4084b8/Sw+rM690NBy0HMb35bzMKmiqDrfOteS4p36OE7lPT1R38A62f9I5+rVhH+HAP+PBJF7NP95gNOvIvni+K9LGPRdHSHdZqie8xA7ULos7RojgS4q2wCd+gLC5haSeauxuZdsy7oIFcqF9YtN0SEAhU10Ks0m6VjN/i6Ntt8QXEbi1CZSjbRoB2uhSo1/O0mwhb+oehA8xQP02fX8nsimHHmS7jgP1/tTyo5mcviX7jEVjZn4jDH85HwD5hvEWJwvQdgMWOuXWCU6CVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+xCbH4eDchMlFs7nT2/QmDwZNqdAOohlZFwWkU3AmMA=; b=UnEuOE5rp7tHeKFZ6iuY1qB6L42XPq/+kEWQ11r0MSjmZ4yvYDr9MXu3fh59yl1v4Kp+5NmL0pcdjBcK2YCOTWDH8Nq2Snelbjuj91oDfUmig2tkcQ1lSJk6ga1OvfSRCGm2Jy8MwtpTTnDXUFOEE1bkNZb624Bd1gDtOY69pMEMQOKROSfTmXeQgFqDsl0VxgDr70QkOZC7Nfr5XSG+74RSz+6j+nF2LzUwNmc/7ZCYmvMZX8JItaZH4/H69eMpwnDPWTQdH7Tt3PH4CXLBmJ9xu80Fs5vNeQ+BEDQEWShxn8r1E7RKrPQ62UNi5d9SV0cbWtsLpSo8JD5pUBKI4A== Received: from BN8PR12CA0024.namprd12.prod.outlook.com (2603:10b6:408:60::37) by BN7PR12MB2594.namprd12.prod.outlook.com (2603:10b6:408:28::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.19; Thu, 1 Jul 2021 13:27:12 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::aa) by BN8PR12CA0024.outlook.office365.com (2603:10b6:408:60::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.21 via Frontend Transport; Thu, 1 Jul 2021 13:27:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:11 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:26:57 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:25:59 +0300 Message-ID: <20210701132609.53727-6-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5eab817b-55ee-4bf2-a459-08d93c93ef2e X-MS-TrafficTypeDiagnostic: BN7PR12MB2594: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2512; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EhkbhXysEBdlMyh8U2m+h4cnI80SNCO374gFXcT+ox9P8uXUI09Px9Mx8oXvZUGITExZopP4pglUIm+s4p8ar9RzjsCbAuwxrlOX4PqtMEKzlxPU7hX4WUD3O/AnMM1JvnRJlFFtWeTkjoshS+CuiEBkml8lB6AnsdXF4ucqSNoG8mvzK/N8FmbfaOiwj6wwkR2MPYYakT1w57yY+DWoobyeOlEHRy7/Z7Rh87T4xhCT6+4ww16LQR2aTahiIk8sal/xKAwH+ej94qE+hqRlPjGphbFTg548AiSFVED+MKNkPePxdvtWXEAR6byKKxj/AXMaedsx/RBFHjcl9OBJVAX3TO1/SREoLnpvI37Bna2MALkMGkaqwSfgMlWoQKJzv1YzBiYvI5w/kLNV3JgQE0UbkbLpu82cAtta6hs+/qrlBHEBidUu9Ib3vaUlQZwIf5Wm4lBLJP5jr6FikCqy35fJ+TaH8k8FhFjLK/egS8wy8J6nGmaG5X0csKOWZfILZ/+iLjS4ejpuvDRr76Ks9oPOq2hu9z9CJnpz032bhiyzbDguiz8Ga0IRF+Ii10jijz6IeXfnlTnAjdk2MVJ1+P/yRNV5GgAhYpXceXTyq3cOwziuHvd2ctVcD6pKa1IBASxCPxdsRUH3dFVrgJgMuA9qqili9po5eZS5ynm/IiWNOKflddGzqzpCWkC3fsobmuxi1E9cw99s2YU1s1cIlw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(39860400002)(346002)(136003)(376002)(36840700001)(46966006)(6286002)(83380400001)(7696005)(5660300002)(70206006)(70586007)(16526019)(86362001)(8676002)(26005)(186003)(54906003)(107886003)(36860700001)(36906005)(4326008)(36756003)(8936002)(1076003)(316002)(478600001)(47076005)(82740400003)(6916009)(2616005)(356005)(7636003)(55016002)(426003)(2906002)(336012)(82310400003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:11.8487 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5eab817b-55ee-4bf2-a459-08d93c93ef2e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR12MB2594 Subject: [dpdk-dev] [PATCH v5 05/15] crypto/mlx5: add queue pairs operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The HW queue pairs are a pair of send queue and receive queue of independent work queues packed together in one object for the purpose of transferring data between nodes of a network. Completion Queue is a FIFO queue of completed work requests. In crypto driver we use one QP in loopback in order to encrypt and decrypt data locally without sending it to the wire. In the configured QP we only use the SQ to perform the encryption and decryption operations. Added implementation for the QP setup function which creates the CQ, creates the QP and changes its state to RTS (ready to send). Added implementation for the release QP function to release all the QP resources. Added the ops structure that contains any operation which is supported by the cryptodev. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- drivers/crypto/mlx5/mlx5_crypto.c | 129 +++++++++++++++++++++++++++++- drivers/crypto/mlx5/mlx5_crypto.h | 11 +++ 2 files changed, 137 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index 0ce46ffd2f..f1d690c750 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -19,6 +20,7 @@ #define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto #define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5 +#define MLX5_CRYPTO_MAX_QPS 1024 #define MLX5_CRYPTO_FEATURE_FLAGS \ (RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | \ @@ -94,7 +96,7 @@ mlx5_crypto_dev_infos_get(struct rte_cryptodev *dev, dev_info->driver_id = mlx5_crypto_driver_id; dev_info->feature_flags = MLX5_CRYPTO_FEATURE_FLAGS; dev_info->capabilities = mlx5_crypto_caps; - dev_info->max_nb_queue_pairs = 0; + dev_info->max_nb_queue_pairs = MLX5_CRYPTO_MAX_QPS; dev_info->min_mbuf_headroom_req = 0; dev_info->min_mbuf_tailroom_req = 0; dev_info->sym.max_nb_sessions = 0; @@ -235,6 +237,127 @@ mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev, DRV_LOG(DEBUG, "Session %p was cleared.", spriv); } +static int +mlx5_crypto_queue_pair_release(struct rte_cryptodev *dev, uint16_t qp_id) +{ + struct mlx5_crypto_qp *qp = dev->data->queue_pairs[qp_id]; + + if (qp->qp_obj != NULL) + claim_zero(mlx5_devx_cmd_destroy(qp->qp_obj)); + if (qp->umem_obj != NULL) + claim_zero(mlx5_glue->devx_umem_dereg(qp->umem_obj)); + if (qp->umem_buf != NULL) + rte_free(qp->umem_buf); + mlx5_devx_cq_destroy(&qp->cq_obj); + rte_free(qp); + dev->data->queue_pairs[qp_id] = NULL; + return 0; +} + +static int +mlx5_crypto_qp2rts(struct mlx5_crypto_qp *qp) +{ + /* + * In Order to configure self loopback, when calling these functions the + * remote QP id that is used is the id of the same QP. + */ + if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_RST2INIT_QP, + qp->qp_obj->id)) { + DRV_LOG(ERR, "Failed to modify QP to INIT state(%u).", + rte_errno); + return -1; + } + if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_INIT2RTR_QP, + qp->qp_obj->id)) { + DRV_LOG(ERR, "Failed to modify QP to RTR state(%u).", + rte_errno); + return -1; + } + if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_RTR2RTS_QP, + qp->qp_obj->id)) { + DRV_LOG(ERR, "Failed to modify QP to RTS state(%u).", + rte_errno); + return -1; + } + return 0; +} + +static int +mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, + const struct rte_cryptodev_qp_conf *qp_conf, + int socket_id) +{ + struct mlx5_crypto_priv *priv = dev->data->dev_private; + struct mlx5_devx_qp_attr attr = {0}; + struct mlx5_crypto_qp *qp; + uint16_t log_nb_desc = rte_log2_u32(qp_conf->nb_descriptors); + uint32_t umem_size = RTE_BIT32(log_nb_desc) * + MLX5_CRYPTO_WQE_SET_SIZE + + sizeof(*qp->db_rec) * 2; + uint32_t alloc_size = sizeof(*qp); + struct mlx5_devx_cq_attr cq_attr = { + .uar_page_id = mlx5_os_get_devx_uar_page_id(priv->uar), + }; + + if (dev->data->queue_pairs[qp_id] != NULL) + mlx5_crypto_queue_pair_release(dev, qp_id); + alloc_size = RTE_ALIGN(alloc_size, RTE_CACHE_LINE_SIZE); + alloc_size += sizeof(struct rte_crypto_op *) * RTE_BIT32(log_nb_desc); + qp = rte_zmalloc_socket(__func__, alloc_size, RTE_CACHE_LINE_SIZE, + socket_id); + if (qp == NULL) { + DRV_LOG(ERR, "Failed to allocate QP memory."); + rte_errno = ENOMEM; + return -rte_errno; + } + if (mlx5_devx_cq_create(priv->ctx, &qp->cq_obj, log_nb_desc, + &cq_attr, socket_id) != 0) { + DRV_LOG(ERR, "Failed to create CQ."); + goto error; + } + qp->umem_buf = rte_zmalloc_socket(__func__, umem_size, 4096, socket_id); + if (qp->umem_buf == NULL) { + DRV_LOG(ERR, "Failed to allocate QP umem."); + rte_errno = ENOMEM; + goto error; + } + qp->umem_obj = mlx5_glue->devx_umem_reg(priv->ctx, + (void *)(uintptr_t)qp->umem_buf, + umem_size, + IBV_ACCESS_LOCAL_WRITE); + if (qp->umem_obj == NULL) { + DRV_LOG(ERR, "Failed to register QP umem."); + goto error; + } + attr.pd = priv->pdn; + attr.uar_index = mlx5_os_get_devx_uar_page_id(priv->uar); + attr.cqn = qp->cq_obj.cq->id; + attr.log_page_size = rte_log2_u32(sysconf(_SC_PAGESIZE)); + attr.rq_size = 0; + attr.sq_size = RTE_BIT32(log_nb_desc); + attr.dbr_umem_valid = 1; + attr.wq_umem_id = qp->umem_obj->umem_id; + attr.wq_umem_offset = 0; + attr.dbr_umem_id = qp->umem_obj->umem_id; + attr.dbr_address = RTE_BIT64(log_nb_desc) * + MLX5_CRYPTO_WQE_SET_SIZE; + qp->qp_obj = mlx5_devx_cmd_create_qp(priv->ctx, &attr); + if (qp->qp_obj == NULL) { + DRV_LOG(ERR, "Failed to create QP(%u).", rte_errno); + goto error; + } + qp->db_rec = RTE_PTR_ADD(qp->umem_buf, (uintptr_t)attr.dbr_address); + if (mlx5_crypto_qp2rts(qp)) + goto error; + qp->ops = (struct rte_crypto_op **)RTE_ALIGN((uintptr_t)(qp + 1), + RTE_CACHE_LINE_SIZE); + dev->data->queue_pairs[qp_id] = qp; + return 0; +error: + mlx5_crypto_queue_pair_release(dev, qp_id); + return -1; +} + static struct rte_cryptodev_ops mlx5_crypto_ops = { .dev_configure = mlx5_crypto_dev_configure, .dev_start = NULL, @@ -243,8 +366,8 @@ static struct rte_cryptodev_ops mlx5_crypto_ops = { .dev_infos_get = mlx5_crypto_dev_infos_get, .stats_get = NULL, .stats_reset = NULL, - .queue_pair_setup = NULL, - .queue_pair_release = NULL, + .queue_pair_setup = mlx5_crypto_queue_pair_setup, + .queue_pair_release = mlx5_crypto_queue_pair_release, .sym_session_get_size = mlx5_crypto_sym_session_get_size, .sym_session_configure = mlx5_crypto_sym_session_configure, .sym_session_clear = mlx5_crypto_sym_session_clear, diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index 4c07356028..a3bc65b0ed 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -11,9 +11,11 @@ #include #include +#include #define MLX5_CRYPTO_DEK_HTABLE_SZ (1 << 11) #define MLX5_CRYPTO_KEY_LENGTH 80 +#define MLX5_CRYPTO_WQE_SET_SIZE 1024 struct mlx5_crypto_priv { TAILQ_ENTRY(mlx5_crypto_priv) next; @@ -27,6 +29,15 @@ struct mlx5_crypto_priv { struct rte_cryptodev_config dev_config; }; +struct mlx5_crypto_qp { + struct mlx5_devx_cq cq_obj; + struct mlx5_devx_obj *qp_obj; + struct mlx5dv_devx_umem *umem_obj; + void *umem_buf; + volatile uint32_t *db_rec; + struct rte_crypto_op **ops; +}; + struct mlx5_crypto_dek { struct mlx5_hlist_entry entry; /* Pointer to DEK hash list entry. */ struct mlx5_devx_obj *obj; /* Pointer to DEK DevX object. */ From patchwork Thu Jul 1 13:26:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95137 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C8C63A0A0C; Thu, 1 Jul 2021 15:27:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 590964130B; Thu, 1 Jul 2021 15:27:18 +0200 (CEST) Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam08on2060.outbound.protection.outlook.com [40.107.101.60]) by mails.dpdk.org (Postfix) with ESMTP id 15B46412E9 for ; Thu, 1 Jul 2021 15:27:14 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YuimOEyDCSFX6qk89spvJDnfrqUG0FXLvQ9Xo+A3pkwSF/xU61gEU0kAytIuY+K5CJTs55gUoEJ4hnGUDNa5LAYsUAu+rSApWkOAgOJcTYkl+flz9TsONdb9MCSLr5YC8M1+3aHfCnA0ryTh7amV45b2mhwOXBBS2ksf4+kNl3U91+EvEf+tkzeifyYh7YKssDnAowEjq9l0NE/W5rqSJcg6Kgdrp7/Yx0NlAgdO3qWUrJmj81A5rlGgzgHH3Fshz6BHq7iwwnyPgMGhVTweR4r7OMKH9LO/c2pgoid0WqAK5DRP9eO6fxUHJhpj5gw2bqV6/iv8PgkE+H0UZunVYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/bxEzybKhgl75DMl9q1ZnfNf0STxKODZ6Ym/NRVP+Gc=; b=lBLbISoChKZjHVZScNBx7X8fs7EWSXDDrJFiKPF1FKMm1b9/3qvAEzLRVyBHadHYnMJShaPm+GlsthcXU8ir5piM2PXc3FxcF7iJgdzl616wc58aIEUwBoOQsfCE8Iw8PwdPFuUp+hPuJg0pssr5wCT1XqUOBHL9RxWmWVldhwlzW0a2QHiFdVh1ABcmaLrA3ZvDmn3d0eY1E5+L2wYr8NtYTL1nwnQlR3P5nOm1IUX8uYX/+/YNWdcKSzgj0+cRRbI8/qyM+R3zuLxU3BLRZPWtuy5kjfIUMBLIrBfcskTox2GX/Q2PjKaWH3Svj1S0CD+tq5zw7sF7W1KsCMyVdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/bxEzybKhgl75DMl9q1ZnfNf0STxKODZ6Ym/NRVP+Gc=; b=qlPRPVUuEWZLoENhZNwCiC6vE8QGUHdPdPqEpp5ulhgJHEsAyWgv8+OnDeGbSqa00vSAWLMRtU0AIS9B2d7zu+oVAKUVLWm5VWuPGw2C0wPjqlqOEPsmUcmcigSB9y5CvzN5gb+HLdNs8U4MIu+Jp16nlbfUPxhMG+/30hhn0Ag8vyiiT2oRifVY7xf9nuPFS9ePZ0sx3PiMAsB8tYHUDm+xWtepQtyK1U3YsgVMtm8HydL17PfEFzlRF2JaaL1zCtQeyWitS/QHLhmQDqX+ILxGc5t+K78TQd0IKbWoXHIYqRe59DpWgLRHl2pbHAe9QVgeru/R5tRxmXrL3RnHTg== Received: from BN8PR12CA0002.namprd12.prod.outlook.com (2603:10b6:408:60::15) by BL0PR12MB2530.namprd12.prod.outlook.com (2603:10b6:207:43::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22; Thu, 1 Jul 2021 13:27:12 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::5a) by BN8PR12CA0002.outlook.office365.com (2603:10b6:408:60::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:12 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:26:59 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:26:00 +0300 Message-ID: <20210701132609.53727-7-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 87b19061-f7f6-49d8-e49a-08d93c93ef7a X-MS-TrafficTypeDiagnostic: BL0PR12MB2530: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1303; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0G3Do8R/lvAP0ZWUKTAM9LWyO2uVs+pO7EtCVCeSJ39w8XzjS+HDoTVZ13C/m1vz3juL+4khHfn5hct3s5OMXQFQhPB7Y+xHBCOVqzNPg1S7/hmkajhpYGhjF/t/lRXzVtHavzmZRfMguZF4wvwgEBte5TkrXoj7p/G0fISktYGZyMJ2wdXH3EVbZ6C0bPhH2jTFZTtDndrVvgyv+EDCz1dEzQ2PqVvnwdjrkYVg3BzwkYBbsGsbXrTnxyG+mgSoB4pTs17xzGnrag0vaOhiD6nXTU1tsPLRUKlrtHEnNt4lW8cWR702V+B0QbJtIxJ60EAyTnphN4jrpyEhd2OZS1LSofOUT+4Y4hQ9u5Dv9z1WVnbfLFUSExDuoxJ17u2NSglM5HlPLcn1Y2yS3LeqntxOcrfdG/UqVBOJ4YyL7ef9eh69WE0ycyU+4pgUZF2EAglOMq371YQt49Kuri+gpGsBEIiXPvWGMoIsyrPvw1T6lqJ/BUZLdnTRl1npdORQggnM17fw7d6rtDpPEqjJBJhad6Th8SmUxhliFqbLAOoBMTBCvQBUvBZW4rVTUSxAwV10pSldvr3FdEol9zEoWfVviHGxE7SCcqxZuFtXfJXm3d+Z0UVAKhG1JRdrkWcFmIjTVJd8s5X4ncXz6uJ/MQTV29NEHdE8gW2KGduhRiiZdahZ4Uz5N3Op618drU+7LE92VIO4QWSd0p7SfxtjOw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(136003)(376002)(396003)(346002)(46966006)(36840700001)(82740400003)(2906002)(4326008)(316002)(356005)(107886003)(7696005)(36906005)(70206006)(336012)(426003)(8936002)(6286002)(70586007)(36756003)(8676002)(47076005)(83380400001)(7636003)(55016002)(26005)(82310400003)(5660300002)(186003)(6916009)(16526019)(478600001)(6666004)(1076003)(54906003)(2616005)(36860700001)(86362001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:12.2644 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 87b19061-f7f6-49d8-e49a-08d93c93ef7a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB2530 Subject: [dpdk-dev] [PATCH v5 06/15] crypto/mlx5: add dev stop and start operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the dev_start function that is used to start a configured device. Add the dev_stop function that is used to stop a configured device. Both functions set the dev parameter as used and return 0. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- drivers/crypto/mlx5/mlx5_crypto.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index f1d690c750..1e24cf8c25 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -131,6 +131,19 @@ mlx5_crypto_dev_configure(struct rte_cryptodev *dev, return 0; } +static void +mlx5_crypto_dev_stop(struct rte_cryptodev *dev) +{ + RTE_SET_USED(dev); +} + +static int +mlx5_crypto_dev_start(struct rte_cryptodev *dev) +{ + RTE_SET_USED(dev); + return 0; +} + static int mlx5_crypto_dev_close(struct rte_cryptodev *dev) { @@ -360,8 +373,8 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, static struct rte_cryptodev_ops mlx5_crypto_ops = { .dev_configure = mlx5_crypto_dev_configure, - .dev_start = NULL, - .dev_stop = NULL, + .dev_start = mlx5_crypto_dev_start, + .dev_stop = mlx5_crypto_dev_stop, .dev_close = mlx5_crypto_dev_close, .dev_infos_get = mlx5_crypto_dev_infos_get, .stats_get = NULL, From patchwork Thu Jul 1 13:26:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95138 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 416C2A0A0C; Thu, 1 Jul 2021 15:27:47 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7B1734130E; Thu, 1 Jul 2021 15:27:19 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2076.outbound.protection.outlook.com [40.107.236.76]) by mails.dpdk.org (Postfix) with ESMTP id 545DD412EC for ; Thu, 1 Jul 2021 15:27:14 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PEV3P7XvZzrTyj95tg6Q6Vi9k+0FB6wDmuYh3bqhNafedZCUOWIs287beEMdaB7BFECpp67ZEjxLb/YURZRpbG5d8YOi97KNtz63F0KFdMnKN4SxaZlLsJEQY2L34FBsowByAxLH+bQTgpXw2vs0QbqdPhVCic6Q/siVjJReRiysNel945Th4+eW0tL6ykciApgTQAfpGIXGq1xFAk+IA8ZuTXH6HnDcroT1w5+EN+YIWx2I4ljqfe4oirJ4dOsFUlTErIo7+tD6Nl4GzsdkFttbS7T/PG1TIEVWo0oH4p3/XVHMa4LbfOXkatXyJr9mzwv0hNQDMoAQ484mAm075Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UUJL1N4f/F/fp4mCsrK1ywFVhUrxyQOiAOlurUueBYo=; b=FTARky1fM3W3CLubZB2bFJDqfo6sphxyNVocoe+3Dliw+wD6T0a6RCtVsrq9kF4iCVXwAfvPVRECgsqDiuIQ1QeOjVNFLKD2xX+wA4O20uAxUeiz6R+jy3cviPfQoOqkvlCnRodN+ngZoJueLwJSei+UgOyuMzGOE8qnal5/RmekDUwQEHU12xrQ9eA9InnT5OpW1yxcLZ21cDMKLb5neF6DplaPaInmmkLWXv3fg5h3dCuPaUykUYod1ayAjcHAXEMxIEDcOgoAOnq/kIdmGc/++etwn7+AVlXmz4P4y669dpbHA+f6aF1V06E/V9zNbLcwBDG6cMyZS+DDGIburQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UUJL1N4f/F/fp4mCsrK1ywFVhUrxyQOiAOlurUueBYo=; b=eaCEY+7JBqvId4TIKe3AzOrtIpzENsGXKjrJ06lDImOtYMyHup+bv4UHWvdkl8vxc64Z3XfqVGwS/9WzpWIizeClr1C/NgkNtlcycZuI5SzvblOH0o6fEuKiGGk+7vddqwsTe0Wy37LFAHXHwcTO4ziTkSlk0N7odK1G+mUt06ZfWnn7i83FdM8/+/oKQlPqtKhFQmPgZRJzIDfJJ+q9svvVzk1/eSJha3Ya9uMdhbDo7eHO6sv5KXz9kMTsPZtUY5akBkLu+VpW+xkCNMZpPk0uan+bQHvqbVUkYiaMYc/w4MQCCJB46DOmhvtHZEDoWSeyqMftZ4/LNpCTSzRxCg== Received: from BN8PR12CA0016.namprd12.prod.outlook.com (2603:10b6:408:60::29) by MN2PR12MB4518.namprd12.prod.outlook.com (2603:10b6:208:266::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.20; Thu, 1 Jul 2021 13:27:13 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::2c) by BN8PR12CA0016.outlook.office365.com (2603:10b6:408:60::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:12 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:27:00 +0000 From: Shiri Kuzin To: CC: , , , "Michael Baum" Date: Thu, 1 Jul 2021 16:26:01 +0300 Message-ID: <20210701132609.53727-8-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 622a8fa1-fb2d-4aa1-eb48-08d93c93efd7 X-MS-TrafficTypeDiagnostic: MN2PR12MB4518: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:785; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fIMG+XQRZ16iWcuzEe9amoGKOTde91NjtpCxJG8B4vW1Dsl3+xU7b2qv+EOj3TxLrioMXnP5sxkBLJDrgMEPWe50GJBboub3It/Dgz1XLY3NCn6MPO5XTQBEi4FZ3ZcE6TXIW6qb76IQFEzivXdrK9b0h7f40YBM91QpJFukbvq8hRTFL36jjuWTMZUN5lsXNUPpsHwhVrWRo5TXmmCFCZpH0WmozhL6vxb94OSTNlRqyNOiI0jB/h49O/+bEldeasdbKUTXbR2iXgTvcWlYbyAXhbqdxD9cm6T98biFExdvaKb4wbfD2qabLBwpt4FSotieWNqrqFEWEupMibq53BWrW40WghTUpULGG8lYQbul3pYaYZPvU9v9CaBk4ecqIy4NpCbwhpU08m/m7szRQ0WDG82EUeWXGOtoVICxHvFGDzwaHPa3EaEfSBZWM0Xw+xtglSjY8yVIuBR5giIjciUCBhgy4Gaz0zVFdP7yTHB/z/Ocv1T9snKZYkkyLDyXU6sN1/r5Ani74AsjnKYKcJiCLHENhgr7f+GG36PQe1CnvUZLWdMXwrqxvLvtAcAdmlnD8kQoigj9XNBTnOMl3rpSSkkOY+fUuYRDTI6jlrMjEkcz37T6mu1YD2EGCf8Zm0q9N/IZzHNrAEtQYRI3TyFHx+BRo3PXfqe328Wuw/ev8r8c++qojqr09v6jwWDX2HOU1+9LS9u7MK6RrrNnyQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(396003)(376002)(346002)(136003)(36840700001)(46966006)(16526019)(186003)(83380400001)(36756003)(86362001)(5660300002)(8676002)(1076003)(7636003)(54906003)(426003)(26005)(6916009)(336012)(82740400003)(55016002)(2616005)(478600001)(70586007)(316002)(36860700001)(8936002)(356005)(6286002)(7696005)(107886003)(82310400003)(2906002)(4326008)(70206006)(47076005)(36906005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:12.9740 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 622a8fa1-fb2d-4aa1-eb48-08d93c93efd7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4518 Subject: [dpdk-dev] [PATCH v5 07/15] crypto/mlx5: add memory region management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Mellanox user space drivers don't deal with physical addresses as part of a memory protection mechanism. The device translates the given virtual address to a physical address using the given memory key as an address space identifier. That's why any mbuf virtual address is moved directly to the HW descriptor(WQE). The mapping between the virtual address to the physical address is saved in MR configured by the kernel to the HW. Each MR has a key that should also be moved to the WQE by the SW. When the SW sees an unmapped address, it extends the address range and creates a MR using a system call. Add memory region cache management: - 2 level cache per queue-pair - no locks. - 1 shared cache between all the queues using a lock. Using this way, the MR key search per data-path address is optimized. Signed-off-by: Shiri Kuzin Signed-off-by: Michael Baum Acked-by: Matan Azrad --- doc/guides/cryptodevs/mlx5.rst | 6 +++ drivers/crypto/mlx5/mlx5_crypto.c | 63 +++++++++++++++++++++++++++++++ drivers/crypto/mlx5/mlx5_crypto.h | 3 ++ 3 files changed, 72 insertions(+) diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst index dd1d1a615d..ece881220e 100644 --- a/doc/guides/cryptodevs/mlx5.rst +++ b/doc/guides/cryptodevs/mlx5.rst @@ -26,6 +26,12 @@ the MKEY is configured to perform crypto operations. The encryption does not require text to be aligned to the AES block size (128b). +For security reasons and to increase robustness, this driver only deals with virtual +memory addresses. The way resources allocations are handled by the kernel, +combined with hardware specifications that allow handling virtual memory +addresses directly, ensure that DPDK applications cannot access random +physical memory (or memory that does not belong to the current process). + The PMD uses libibverbs and libmlx5 to access the device firmware or to access the hardware components directly. There are different levels of objects and bypassing abilities. diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index 1e24cf8c25..367a4759b3 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -261,6 +261,7 @@ mlx5_crypto_queue_pair_release(struct rte_cryptodev *dev, uint16_t qp_id) claim_zero(mlx5_glue->devx_umem_dereg(qp->umem_obj)); if (qp->umem_buf != NULL) rte_free(qp->umem_buf); + mlx5_mr_btree_free(&qp->mr_ctrl.cache_bh); mlx5_devx_cq_destroy(&qp->cq_obj); rte_free(qp); dev->data->queue_pairs[qp_id] = NULL; @@ -342,6 +343,14 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, DRV_LOG(ERR, "Failed to register QP umem."); goto error; } + if (mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, + priv->dev_config.socket_id) != 0) { + DRV_LOG(ERR, "Cannot allocate MR Btree for qp %u.", + (uint32_t)qp_id); + rte_errno = ENOMEM; + goto error; + } + qp->mr_ctrl.dev_gen_ptr = &priv->mr_scache.dev_gen; attr.pd = priv->pdn; attr.uar_index = mlx5_os_get_devx_uar_page_id(priv->uar); attr.cqn = qp->cq_obj.cq->id; @@ -448,6 +457,40 @@ mlx5_crypto_hw_global_prepare(struct mlx5_crypto_priv *priv) return 0; } +/** + * Callback for memory event. + * + * @param event_type + * Memory event type. + * @param addr + * Address of memory. + * @param len + * Size of memory. + */ +static void +mlx5_crypto_mr_mem_event_cb(enum rte_mem_event event_type, const void *addr, + size_t len, void *arg __rte_unused) +{ + struct mlx5_crypto_priv *priv; + + /* Must be called from the primary process. */ + MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); + switch (event_type) { + case RTE_MEM_EVENT_FREE: + pthread_mutex_lock(&priv_list_lock); + /* Iterate all the existing mlx5 devices. */ + TAILQ_FOREACH(priv, &mlx5_crypto_priv_list, next) + mlx5_free_mr_by_addr(&priv->mr_scache, + priv->ctx->device->name, + addr, len); + pthread_mutex_unlock(&priv_list_lock); + break; + case RTE_MEM_EVENT_ALLOC: + default: + break; + } +} + /** * DPDK callback to register a PCI device. * @@ -530,6 +573,22 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, claim_zero(mlx5_glue->close_device(priv->ctx)); return -1; } + if (mlx5_mr_btree_init(&priv->mr_scache.cache, + MLX5_MR_BTREE_CACHE_N * 2, rte_socket_id()) != 0) { + DRV_LOG(ERR, "Failed to allocate shared cache MR memory."); + mlx5_crypto_hw_global_release(priv); + rte_cryptodev_pmd_destroy(priv->crypto_dev); + claim_zero(mlx5_glue->close_device(priv->ctx)); + rte_errno = ENOMEM; + return -rte_errno; + } + priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr; + priv->mr_scache.dereg_mr_cb = mlx5_common_verbs_dereg_mr; + /* Register callback function for global shared MR cache management. */ + if (TAILQ_EMPTY(&mlx5_crypto_priv_list)) + rte_mem_event_callback_register("MLX5_MEM_EVENT_CB", + mlx5_crypto_mr_mem_event_cb, + NULL); pthread_mutex_lock(&priv_list_lock); TAILQ_INSERT_TAIL(&mlx5_crypto_priv_list, priv, next); pthread_mutex_unlock(&priv_list_lock); @@ -549,6 +608,10 @@ mlx5_crypto_pci_remove(struct rte_pci_device *pdev) TAILQ_REMOVE(&mlx5_crypto_priv_list, priv, next); pthread_mutex_unlock(&priv_list_lock); if (priv) { + if (TAILQ_EMPTY(&mlx5_crypto_priv_list)) + rte_mem_event_callback_unregister("MLX5_MEM_EVENT_CB", + NULL); + mlx5_mr_release_cache(&priv->mr_scache); mlx5_crypto_hw_global_release(priv); rte_cryptodev_pmd_destroy(priv->crypto_dev); claim_zero(mlx5_glue->close_device(priv->ctx)); diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index a3bc65b0ed..d1ea8024d4 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -12,6 +12,7 @@ #include #include +#include #define MLX5_CRYPTO_DEK_HTABLE_SZ (1 << 11) #define MLX5_CRYPTO_KEY_LENGTH 80 @@ -27,6 +28,7 @@ struct mlx5_crypto_priv { struct ibv_pd *pd; struct mlx5_hlist *dek_hlist; /* Dek hash list. */ struct rte_cryptodev_config dev_config; + struct mlx5_mr_share_cache mr_scache; /* Global shared MR cache. */ }; struct mlx5_crypto_qp { @@ -36,6 +38,7 @@ struct mlx5_crypto_qp { void *umem_buf; volatile uint32_t *db_rec; struct rte_crypto_op **ops; + struct mlx5_mr_ctrl mr_ctrl; }; struct mlx5_crypto_dek { From patchwork Thu Jul 1 13:26:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95139 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3F37DA0A0C; Thu, 1 Jul 2021 15:27:56 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ED76041319; Thu, 1 Jul 2021 15:27:20 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2056.outbound.protection.outlook.com [40.107.94.56]) by mails.dpdk.org (Postfix) with ESMTP id 030CE412EC for ; Thu, 1 Jul 2021 15:27:14 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OvP4bi9P9ritLS+kQcSknf4yBG+lHtapFSB4CVk4rmK2bzL1WwDtwXsAa+iVD8lf2yjCM+DVNvw9zQaQDrp02E1CuKm3f4Ad48TBbS5Lc0ZHUlVqUA4YrK0e6aekG3YOdGYfo33y1BO+pAQrqsARotCWhhgDCiqV/N9573BB7SbaxYvX9f/h7vcbOjuru9a/9cHsKxsQxKhNe773J6Oejhp1WmBfy7He2y/CMiAIh1EnfU+OPBD0ylU28PNKPriLI2Wl3A1ETSIAA51eXFPcdC3Mhx+FRV1KGdrAXD8fljNUQhvPHG/FbUxAy8HQW4ZitRJRKeJilio/1mlQoWVH+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PQaW3/L0VD96uSlG3eadVZCRSzocUDVfrrtppt9zduQ=; b=bXeb1tHDfPUfSYujSSfE3b3nLndtmchfpGPDrYOo0kSCSJeDezYhQ2Lw700dAKzAVjBveZJbFzCHeWla2gK6E27xVL6mb7z6hCTBwqOcg4QpymuB5tvHEy7dIbxivmUChsrGyI6hWHLo+IrsUGlVzwQiNm0ooB1Iq7Nsn8ELus4ZbBty9xe3xYerXz1EUuNfqTSkddIXWIZXfVSMDUt3g+v5hXggJIlkIyrbg/BPtUX+v4IpocVTs4d1q8wvHkWpRUaVPOSKHenSeapBt2/rqV1zBS5NkOSzZd5oM8qb3Yu7dQrfA5ndInlkClceuoorEjjRci4OMx21snJF+dznug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PQaW3/L0VD96uSlG3eadVZCRSzocUDVfrrtppt9zduQ=; b=G/dzDZaQGtiGle7EeB6c/a4rGUq85O+gzP881AAkZn4aoD4PAkQfNZITmaIA/OpZoVRyRAlfN8uEKkB868pX9gw84KpDkAOqkr2+OH/HPbJ3YOBJzGcgCOJ5l2vzzKWRb4XVHFfqK001a6Y4uPP4f2Sk29w2r/3m0MiohnYlUmb2Tl5aSsPDJoqbWJTa1GbFe6ihvCq18Nl4QHkU6O/Iq280RlXvbDZDKU7S+sURmTyyyWjx6i6jmbQB2s9Yx9vBI/Gqu9BIRhNgbJpxoqaPTxWWnkiLkjTQr5Is7I+6nPiZGzqI7enMNb+g1x6o/VJ4LbFZSfHOin6c6+YxcDsX7A== Received: from BN8PR12CA0002.namprd12.prod.outlook.com (2603:10b6:408:60::15) by DM8PR12MB5416.namprd12.prod.outlook.com (2603:10b6:8:28::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.19; Thu, 1 Jul 2021 13:27:13 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::97) by BN8PR12CA0002.outlook.office365.com (2603:10b6:408:60::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:13 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:27:02 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:26:02 +0300 Message-ID: <20210701132609.53727-9-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 45262b9d-0623-4c5f-b707-08d93c93f01a X-MS-TrafficTypeDiagnostic: DM8PR12MB5416: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:117; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d0MhNmyixKC9VWXEcndhJNh2qU+y3Db9ohtdOlJhKXZXNyzjn1rhDD05Dzq1erL8xnVildPKcqaOHCYEc3UJHXsF7QEqY52KKWkEW+jP+X47g5ss8cwf44r/xXgpVSRiIg161VLAgn0wjbmtkD8P64H5XAaafhXfxip/5wyH/4dcG/ktMu53oHy9LW4TklssmWx/kfB46zUdR/1/PYITG+cZGQlOuMJlZdnd7Q+JyGxT7X+iyqFUP3xWgcyZoBUSl1NR/yqitClO0OMIuOtnpHcTJ8FG5LqI8SejjpWHDWFEvVENpk/jOB9MjPwFDKtJjz7Mmvpgy2Cny4pLtloUIIx4NCms/1LiQM31cOJ1ARyvUQlZfiIWfcQopFmxvw/Atamj7lWq4fBc0Ed6vYLnb4awQ+zXEzF7rxS93ho99EMjNpG/X56LHh5mNeCNscRkPfaQCLar4fqUhwEFw/PIIQVfhmFUzYSzn9KpYxIAxLJzBcERnHVDk9Zl/OnIEqEllOcG87BHatMcLz8uhqiXDcrJP+DjNqF4VIQEcZ7DautPyh15aQLJKMYWCLjY0YyAId4UQaofEvny2U4K8wkuAy1DUGJCFH/og+j30FkIktWysd8cBFRYGYjnsl9YUPnpjetv2rJ1SzL0IvIGIQimwrn8OJNKKWfHoU0dzn3H459UtaK4jcOI4TcGZVsr4xAxhQBs7YQVntPp/EEE/e3L4A== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39850400004)(136003)(396003)(346002)(376002)(36840700001)(46966006)(2616005)(83380400001)(478600001)(36906005)(54906003)(47076005)(1076003)(86362001)(7636003)(2906002)(316002)(356005)(55016002)(36756003)(336012)(26005)(70206006)(186003)(16526019)(8676002)(4326008)(426003)(6286002)(5660300002)(7696005)(70586007)(107886003)(6916009)(8936002)(82310400003)(82740400003)(36860700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:13.4048 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 45262b9d-0623-4c5f-b707-08d93c93f01a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5416 Subject: [dpdk-dev] [PATCH v5 08/15] crypto/mlx5: create login object using DevX X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To work with crypto engines that are marked with wrapped_import_method, a login session is required. A crypto login object needs to be created using DevX. The crypto login object contains: - The credential pointer. - The import_KEK pointer to be used for all secured information communicated in crypto commands (key fields), including the provided credential in this command. - The credential secret, wrapped by the import_KEK indicated in this command. Size includes 8 bytes IV for wrapping. Added devargs for the required login values: - wcs_file - path to the file containing the credential. - import_kek_id - the import KEK pointer. - credential_id - the credential pointer. Create the login DevX object in pci_probe function and destroy it in pci_remove. Destroying the crypto login object means logout. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- doc/guides/cryptodevs/mlx5.rst | 60 +++++++++++++++++ drivers/crypto/mlx5/mlx5_crypto.c | 103 ++++++++++++++++++++++++++++++ drivers/crypto/mlx5/mlx5_crypto.h | 7 ++ 3 files changed, 170 insertions(+) diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst index ece881220e..6cae5affbd 100644 --- a/doc/guides/cryptodevs/mlx5.rst +++ b/doc/guides/cryptodevs/mlx5.rst @@ -44,6 +44,51 @@ To get the best performances: Enabling librte_crypto_mlx5 causes DPDK applications to be linked against libibverbs. +In order to move the device to crypto operational mode, credential and KEK +(Key Encrypting Key) should be set as the first step. +The credential will be used by the software in order to perform crypto login, and the KEK is +the AES Key Wrap Algorithm (rfc3394) key that will be used for sensitive data +wrapping. +The credential and the AES-XTS keys should be provided to the hardware, as ciphertext +encrypted by the KEK. + +When crypto engines are defined to work in wrapped import method, they come out +of the factory in Commissioning mode, and thus, cannot be used for crypto operations +yet. A dedicated tool is used for changing the mode from Commissioning to +Operational, while setting the first import_KEK and credential in plaintext. +The mlxreg dedicated tool should be used as follows: + +- Set CRYPTO_OPERATIONAL register to set the device in crypto operational mode. + + The input to this tool is: + The first credential in plaintext, 40B. + The first import_KEK in plaintext: kek size 0 for 16B or 1 for 32B, kek data. + + Example: + mlxreg -d /dev/mst/mt4123_pciconf0 --reg_name CRYPTO_OPERATIONAL --get + + The "wrapped_crypto_operational" value will be "0x00000000". + The command to set the register should be executed only once, and all the + values mentioned above should be specified in the same command. + + Example: + mlxreg -d /dev/mst/mt4123_pciconf0 --reg_name CRYPTO_OPERATIONAL + --set "credential[0]=0x10000000, credential[1]=0x10000000, kek[0]=0x00000000" + + All values not specified will remain 0. + "wrapped_crypto_going_to_commissioning" and "wrapped_crypto_operational" + should not be specified. + + All the device ports should set it in order to move to operational mode. + +- Query CRYPTO_OPERATIONAL register to make sure the device is in Operational + mode. + + Example: + mlxreg -d /dev/mst/mt4123_pciconf0 --reg_name CRYPTO_OPERATIONAL --get + The "wrapped_crypto_operational" value will be "0x00000001" if the mode was + successfully changed to operational mode. + Driver options -------------- @@ -53,6 +98,21 @@ Driver options Select the class of the driver that should probe the device. `crypto` for the mlx5 crypto driver. +- ``wcs_file`` parameter [string] - mandatory + + File path including only the wrapped credential in string format of hexadecimal + numbers, represent 48 bytes (8 bytes IV added by the AES key wrap algorithm). + +- ``import_kek_id`` parameter [int] + + The identifier of the KEK, default value is 0 represents the operational + register import_kek.. + +- ``credential_id`` parameter [int] + + The identifier of the credential, default value is 0 represents the operational + register credential. + Supported NICs -------------- diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index 367a4759b3..95bbb83fc3 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -457,6 +457,101 @@ mlx5_crypto_hw_global_prepare(struct mlx5_crypto_priv *priv) return 0; } + +static int +mlx5_crypto_args_check_handler(const char *key, const char *val, void *opaque) +{ + struct mlx5_crypto_devarg_params *devarg_prms = opaque; + struct mlx5_devx_crypto_login_attr *attr = &devarg_prms->login_attr; + unsigned long tmp; + FILE *file; + int ret; + int i; + + if (strcmp(key, "class") == 0) + return 0; + if (strcmp(key, "wcs_file") == 0) { + file = fopen(val, "rb"); + if (file == NULL) { + rte_errno = ENOTSUP; + return -rte_errno; + } + for (i = 0 ; i < MLX5_CRYPTO_CREDENTIAL_SIZE ; i++) { + ret = fscanf(file, "%02hhX", &attr->credential[i]); + if (ret <= 0) { + fclose(file); + DRV_LOG(ERR, + "Failed to read credential from file."); + rte_errno = EINVAL; + return -rte_errno; + } + } + fclose(file); + devarg_prms->login_devarg = true; + return 0; + } + errno = 0; + tmp = strtoul(val, NULL, 0); + if (errno) { + DRV_LOG(WARNING, "%s: \"%s\" is an invalid integer.", key, val); + return -errno; + } + if (strcmp(key, "import_kek_id") == 0) + attr->session_import_kek_ptr = (uint32_t)tmp; + else if (strcmp(key, "credential_id") == 0) + attr->credential_pointer = (uint32_t)tmp; + else + DRV_LOG(WARNING, "Invalid key %s.", key); + return 0; +} + +static struct mlx5_devx_obj * +mlx5_crypto_config_login(struct rte_devargs *devargs, + struct ibv_context *ctx) +{ + /* + * Set credential pointer and session import KEK pointer to a default + * value of 0. + */ + struct mlx5_crypto_devarg_params login = { + .login_devarg = false, + .login_attr = { + .credential_pointer = 0, + .session_import_kek_ptr = 0, + } + }; + struct rte_kvargs *kvlist; + + if (devargs == NULL) { + DRV_LOG(ERR, + "No login devargs in order to enable crypto operations in the device."); + rte_errno = EINVAL; + return NULL; + } + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) { + DRV_LOG(ERR, "Failed to parse devargs."); + rte_errno = EINVAL; + return NULL; + } + if (rte_kvargs_process(kvlist, NULL, mlx5_crypto_args_check_handler, + &login) != 0) { + DRV_LOG(ERR, "Devargs handler function Failed."); + rte_kvargs_free(kvlist); + rte_errno = EINVAL; + return NULL; + } + rte_kvargs_free(kvlist); + if (login.login_devarg == false) { + DRV_LOG(ERR, + "No login credential devarg in order to enable crypto operations " + "in the device."); + rte_errno = EINVAL; + return NULL; + } + return mlx5_devx_cmd_create_crypto_login_obj(ctx, &login.login_attr); +} + /** * Callback for memory event. * @@ -512,6 +607,7 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, struct ibv_device *ibv; struct rte_cryptodev *crypto_dev; struct ibv_context *ctx; + struct mlx5_devx_obj *login; struct mlx5_crypto_priv *priv; struct mlx5_hca_attr attr = { 0 }; struct rte_cryptodev_pmd_init_params init_params = { @@ -550,6 +646,11 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, rte_errno = ENOTSUP; return -ENOTSUP; } + login = mlx5_crypto_config_login(pci_dev->device.devargs, ctx); + if (login == NULL) { + DRV_LOG(ERR, "Failed to configure login."); + return -rte_errno; + } crypto_dev = rte_cryptodev_pmd_create(ibv->name, &pci_dev->device, &init_params); if (crypto_dev == NULL) { @@ -566,6 +667,7 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, crypto_dev->driver_id = mlx5_crypto_driver_id; priv = crypto_dev->data->dev_private; priv->ctx = ctx; + priv->login_obj = login; priv->pci_dev = pci_dev; priv->crypto_dev = crypto_dev; if (mlx5_crypto_hw_global_prepare(priv) != 0) { @@ -614,6 +716,7 @@ mlx5_crypto_pci_remove(struct rte_pci_device *pdev) mlx5_mr_release_cache(&priv->mr_scache); mlx5_crypto_hw_global_release(priv); rte_cryptodev_pmd_destroy(priv->crypto_dev); + claim_zero(mlx5_devx_cmd_destroy(priv->login_obj)); claim_zero(mlx5_glue->close_device(priv->ctx)); } return 0; diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index d1ea8024d4..a45a210e58 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -29,6 +29,7 @@ struct mlx5_crypto_priv { struct mlx5_hlist *dek_hlist; /* Dek hash list. */ struct rte_cryptodev_config dev_config; struct mlx5_mr_share_cache mr_scache; /* Global shared MR cache. */ + struct mlx5_devx_obj *login_obj; }; struct mlx5_crypto_qp { @@ -48,6 +49,12 @@ struct mlx5_crypto_dek { bool size_is_48; /* Whether the key\data size is 48 bytes or not. */ } __rte_cache_aligned; + +struct mlx5_crypto_devarg_params { + bool login_devarg; + struct mlx5_devx_crypto_login_attr login_attr; +}; + int mlx5_crypto_dek_destroy(struct mlx5_crypto_priv *priv, struct mlx5_crypto_dek *dek); From patchwork Thu Jul 1 13:26:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95140 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 15C5CA0A0C; Thu, 1 Jul 2021 15:28:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 07173412F2; Thu, 1 Jul 2021 15:27:22 +0200 (CEST) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam07on2082.outbound.protection.outlook.com [40.107.95.82]) by mails.dpdk.org (Postfix) with ESMTP id B82F541303 for ; Thu, 1 Jul 2021 15:27:16 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JvoKOFIX7aKwacRo22APS9Bzza/XO/ZeMam9zUTedHrDTVGZfqJwxXKaD59rH0J8+nQLs5+3vNvzm/17XC7479k1y7kFMpPvwdNLOVQi1iOZGTAp3byVGJyqhTBD43i39+mVQZ5NOn8A2aIxtvNHaPizbOvBWPNAx915fkK5HuPN1K1v5nTfF/o99W7mO4nAzFT27ZCnojj+hn8E1TUnhi45QdiSFizDyEMKTlQU4HiqQOpMq0U4PzVl0llSDG4jtN4B619Nb1J3O8iN/3AGPTQ3tjMrsUvv5WzxxdN/aOa+RJxWpkSCOQyExyGOLiqyKR/84JcKHCzkoNC6K0kP2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NT2Q7TfN1fHSvrWThv1vyZUupBF7ngC7leD7qT4soYo=; b=isd6xQK6ZgnTPZaswsqBuLLPrpWDY3gS1XLUR+60r5GH3EMJ833DFmiwUXk82w5OPZrBeVkl68mmwbmgpn7uXXMrHugBypJOjUMMJrMlVg4L9uwo+WcYuZYksUA7ESptbxXmJZTeBuZROZJLGVv5nYcHxe8142Jdf7Gflnjv+Qi0G/WLEVH56tUuEZ5d2FVxnFDcE6zS6Cp+BlriOg1kjrCnikPPDbfT0QLz2LoT/Wk0opgZnIj/wORBLRAI/3cyfQrIy9EqfhJq7yt66Rp5SMpdPV9JfwntF3WgNvueWzK++T1lKJ/Z7GYLF3HtYKa3xyOIczgvZfpsSNSRz1SEWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NT2Q7TfN1fHSvrWThv1vyZUupBF7ngC7leD7qT4soYo=; b=mfFnNhoWkJYtjcL7w9AHlD1Yp1ZdmcCbdqd9YxAiBXT5WyL3rBFGJ+6okEbnwDNbG7J6h1haEHypAfV0M0CpiRDRSgff5VaTfxxeohB7H8ibBczMkyuQSEdTlhRkS6e+Cryi9hbzgvjwnfd6WgnzVs/i66NWhMnuko81KXe+/a4yJ0uhZXknzgp+073LUzkcFXg4ynB05j/tH5veAqoj2ofi8eTMYXM1Wao+BuByBoGbDbYJUV80NlNjQBNXmN8rIpCb3ApiO7I9avDvGK9Us3z/4mZPJZwGrG7gKDANFCrlIgEMCFR6GJ+nd0U8HABv5ynFxmIJODKDZZeNChEPpg== Received: from BN8PR12CA0016.namprd12.prod.outlook.com (2603:10b6:408:60::29) by BYAPR12MB3142.namprd12.prod.outlook.com (2603:10b6:a03:de::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.23; Thu, 1 Jul 2021 13:27:14 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::c9) by BN8PR12CA0016.outlook.office365.com (2603:10b6:408:60::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:14 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:27:04 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:26:03 +0300 Message-ID: <20210701132609.53727-10-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 452280bb-7416-4df7-138f-08d93c93f086 X-MS-TrafficTypeDiagnostic: BYAPR12MB3142: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:139; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KdLSnssvFUdNb4K8Y5ZTdkwRLFGfQc1TCjkwturQcKzF3ATbtVGkboHwVY/sVoga8QnyKhXXdPr1rvi5WNO5E8PZ+ykaeyrM0o2N8uDJelr8UVS+6Kt3OnsQWcxcjLAT8UCZrYYDjhF3LC6mXvJ1RfnluPyniuejMa9Jzb03rlmyHu8LMsuOP2QpBEGyad92DuzUC8gn76RoEn8IEZFa6OlvO9mEuv692PHCYj6yYf8RJNUz783byMwO8GH/tIUr4KUWnsuuo5OXWQHsmzpZOj9tFV2qIOS7D1CjotjBnCxCX8do69FVrN34Mls2qjTka8SG8KhwiocSqxJE+iS3KozdCDmVWUwijQAoSrxAt+XFbSSgzYINpAd6ScwItkilXQ8A3KnZU+SThTXTbC9X1eF6NICyB9mfniI/zc0NZESW33OFXK8s0FDwN2RL9qx8q3TN920u/PSAMSnQE3RrN9xLwnqkqaWs8ScinJdVuYNc18AQCX0zSTnG5bAzgrawd48tKN1XSUlTVWgRqjSLrPTrd0xKBflRSOPzXcgZhNOeKaXFl13BjyniK19eE4mMh+Uosl36o1IenJHUNcLTFCh9xR+JG6qm8X4RRFpPbKWW+UaMnvxrNCHOqqjIHoWByp0tL4Y9pZsgZjW0uwpPbDei9GJAb/z1QbsZFE6UngQDKnJPASM/nKs5b04hjng0W/1cj9toicLH2upysH+/1w== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(136003)(376002)(346002)(39860400002)(396003)(36840700001)(46966006)(8936002)(7636003)(6916009)(316002)(4326008)(8676002)(54906003)(5660300002)(83380400001)(55016002)(7696005)(26005)(186003)(107886003)(70206006)(1076003)(6286002)(16526019)(82310400003)(36906005)(36756003)(356005)(2906002)(426003)(86362001)(82740400003)(70586007)(478600001)(336012)(36860700001)(47076005)(2616005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:14.0994 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 452280bb-7416-4df7-138f-08d93c93f086 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3142 Subject: [dpdk-dev] [PATCH v5 09/15] crypto/mlx5: add keytag devarg X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Suanming Mou A keytag is a piece of data encrypted together with a DEK. When a DEK is referenced by an MKEY.bsf through its index, the keytag is also supplied in the BSF as plaintext. The HW will decrypt the DEK (and the attached keytag) and will fail the operation if the keytags don't match. This commit adds the configuration of the keytag with devargs. Signed-off-by: Suanming Mou Signed-off-by: Matan Azrad --- doc/guides/cryptodevs/mlx5.rst | 7 +++++ drivers/crypto/mlx5/mlx5_crypto.c | 50 +++++++++++++++++-------------- drivers/crypto/mlx5/mlx5_crypto.h | 3 +- 3 files changed, 37 insertions(+), 23 deletions(-) diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst index 6cae5affbd..c3632484a5 100644 --- a/doc/guides/cryptodevs/mlx5.rst +++ b/doc/guides/cryptodevs/mlx5.rst @@ -52,6 +52,9 @@ wrapping. The credential and the AES-XTS keys should be provided to the hardware, as ciphertext encrypted by the KEK. +A keytag (64 bits) should be appended to the AES-XTS keys (before wrapping), +and will be validated when the hardware attempts to access it. + When crypto engines are defined to work in wrapped import method, they come out of the factory in Commissioning mode, and thus, cannot be used for crypto operations yet. A dedicated tool is used for changing the mode from Commissioning to @@ -113,6 +116,10 @@ Driver options The identifier of the credential, default value is 0 represents the operational register credential. +- ``keytag`` parameter [int] + + The plaintext of the keytag appanded to the AES-XTS keys, default value is 0. + Supported NICs -------------- diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index 95bbb83fc3..6fd6bd1511 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -500,56 +500,52 @@ mlx5_crypto_args_check_handler(const char *key, const char *val, void *opaque) attr->session_import_kek_ptr = (uint32_t)tmp; else if (strcmp(key, "credential_id") == 0) attr->credential_pointer = (uint32_t)tmp; + else if (strcmp(key, "keytag") == 0) + devarg_prms->keytag = tmp; else DRV_LOG(WARNING, "Invalid key %s.", key); return 0; } -static struct mlx5_devx_obj * -mlx5_crypto_config_login(struct rte_devargs *devargs, - struct ibv_context *ctx) +static int +mlx5_crypto_parse_devargs(struct rte_devargs *devargs, + struct mlx5_crypto_devarg_params *devarg_prms) { - /* - * Set credential pointer and session import KEK pointer to a default - * value of 0. - */ - struct mlx5_crypto_devarg_params login = { - .login_devarg = false, - .login_attr = { - .credential_pointer = 0, - .session_import_kek_ptr = 0, - } - }; + struct mlx5_devx_crypto_login_attr *attr = &devarg_prms->login_attr; struct rte_kvargs *kvlist; + /* Default values. */ + attr->credential_pointer = 0; + attr->session_import_kek_ptr = 0; + devarg_prms->keytag = 0; if (devargs == NULL) { DRV_LOG(ERR, "No login devargs in order to enable crypto operations in the device."); rte_errno = EINVAL; - return NULL; + return -1; } kvlist = rte_kvargs_parse(devargs->args, NULL); if (kvlist == NULL) { DRV_LOG(ERR, "Failed to parse devargs."); rte_errno = EINVAL; - return NULL; + return -1; } if (rte_kvargs_process(kvlist, NULL, mlx5_crypto_args_check_handler, - &login) != 0) { + devarg_prms) != 0) { DRV_LOG(ERR, "Devargs handler function Failed."); rte_kvargs_free(kvlist); rte_errno = EINVAL; - return NULL; + return -1; } rte_kvargs_free(kvlist); - if (login.login_devarg == false) { + if (devarg_prms->login_devarg == false) { DRV_LOG(ERR, "No login credential devarg in order to enable crypto operations " "in the device."); rte_errno = EINVAL; - return NULL; + return -1; } - return mlx5_devx_cmd_create_crypto_login_obj(ctx, &login.login_attr); + return 0; } /** @@ -609,6 +605,7 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, struct ibv_context *ctx; struct mlx5_devx_obj *login; struct mlx5_crypto_priv *priv; + struct mlx5_crypto_devarg_params devarg_prms = { 0 }; struct mlx5_hca_attr attr = { 0 }; struct rte_cryptodev_pmd_init_params init_params = { .name = "", @@ -617,6 +614,8 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, .max_nb_queue_pairs = RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS, }; + int ret; + RTE_SET_USED(pci_drv); if (rte_eal_process_type() != RTE_PROC_PRIMARY) { DRV_LOG(ERR, "Non-primary process type is not supported."); @@ -646,7 +645,13 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, rte_errno = ENOTSUP; return -ENOTSUP; } - login = mlx5_crypto_config_login(pci_dev->device.devargs, ctx); + ret = mlx5_crypto_parse_devargs(pci_dev->device.devargs, &devarg_prms); + if (ret) { + DRV_LOG(ERR, "Failed to parse devargs."); + return -rte_errno; + } + login = mlx5_devx_cmd_create_crypto_login_obj(ctx, + &devarg_prms.login_attr); if (login == NULL) { DRV_LOG(ERR, "Failed to configure login."); return -rte_errno; @@ -686,6 +691,7 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, } priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr; priv->mr_scache.dereg_mr_cb = mlx5_common_verbs_dereg_mr; + priv->keytag = rte_cpu_to_be_64(devarg_prms.keytag); /* Register callback function for global shared MR cache management. */ if (TAILQ_EMPTY(&mlx5_crypto_priv_list)) rte_mem_event_callback_register("MLX5_MEM_EVENT_CB", diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index a45a210e58..ad70052967 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -30,6 +30,7 @@ struct mlx5_crypto_priv { struct rte_cryptodev_config dev_config; struct mlx5_mr_share_cache mr_scache; /* Global shared MR cache. */ struct mlx5_devx_obj *login_obj; + uint64_t keytag; }; struct mlx5_crypto_qp { @@ -49,10 +50,10 @@ struct mlx5_crypto_dek { bool size_is_48; /* Whether the key\data size is 48 bytes or not. */ } __rte_cache_aligned; - struct mlx5_crypto_devarg_params { bool login_devarg; struct mlx5_devx_crypto_login_attr login_attr; + uint64_t keytag; }; int From patchwork Thu Jul 1 13:26:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95141 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 18DDCA0A0C; Thu, 1 Jul 2021 15:28:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8A91C41314; Thu, 1 Jul 2021 15:27:23 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2083.outbound.protection.outlook.com [40.107.236.83]) by mails.dpdk.org (Postfix) with ESMTP id C919541308 for ; Thu, 1 Jul 2021 15:27:17 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F5FdQFGpcOzM219MDP+YwvraaSGLGjSne963WWZWLkAPM1DvuuW05V5aZGRUDJrPRaqotlujJXer9auQnNqMiR/sdUtkMdDFa3+sG0YnSjcS22tyboRuSongVZwse1BR3SxShpR49dVnJtIgt0yhpOflupDPvTy8qATuMvUSMrIo/0Bd+DdY7MfAc19i7ugQGu2SwPb5UARop762KJoAz3AS86BhIDlsRvwbo78GpaPmS0URMP69XoNpnQLODoHJ30DiKC1xjtV8h4fITYoaxp1bBHUGjXXMJqOCk6gCO1TeSyOSerBL3VZWHfix4kqR1kwW1Py8yC7CxNVnf+GRCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9FzmoclwsRka3U4dP7hRJcgNte1dHvdZP+Y+qsECWKo=; b=KA5CVtTB5W6Etvd+oNtfX9VBYqTu8S8e4xBQaDWIG3ox4yljQLJ41hgTnZ7d4d/CRbXJCfXSDzSOzJjiY7PWRtXIXlh652kV+jhHc0fxliAmMDzdzCZAqoqHyj4l4zjEVnUYYFXedm1ysxaCGBYbg2L4VQQw5E6omURoFTdS1SNOY1W2MDz2uRTkMUoq8wB3sr4rvBJZxhb497TaFmrJhTLsjzEvGFukxoQFie1q3IpAFVY4NF79axhrJO/SlJrF4EkRdSt68+YkQ5u0mJKnWTyPLqpxUoaVFcEZdmXOZUmNi36A+En794/sQKLRlC0DuL5ybW1Th3sjYaH/PU68mg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9FzmoclwsRka3U4dP7hRJcgNte1dHvdZP+Y+qsECWKo=; b=JtFg+Nixe5G+lhlcNBYCJ0WeRfg14mlJmoJsaZRJcaDxgGmzp3PsoHMWniqUO8nkWLR4iZNVDTwOj0nbGKtxl7FiHJoO6RGrQ/gbIxyR0OnxWgWd1GKgXKGUzaW3Ly4ZZUbCYUZ0o1ZwsUqWgEfirPOzP4ShO4vZwM+SXQjfgjICIzk2qzv46JjmUu9VWYM4HbkV0zPBZcan9aN9ABvoDowANDQ8gh5gVYAngtN6hZbh/tz4xUyy1cQakusmf4BnSa8jTj8VaAM+CTScEEy4FPR3g3nwKeZOsVAIY5LWecS37uhhGIH31Cy7e0d3G6ML567NfNGNqj/vRQT6ZVNWhw== Received: from BN8PR12CA0002.namprd12.prod.outlook.com (2603:10b6:408:60::15) by CY4PR12MB1126.namprd12.prod.outlook.com (2603:10b6:903:3d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.20; Thu, 1 Jul 2021 13:27:16 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::e3) by BN8PR12CA0002.outlook.office365.com (2603:10b6:408:60::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:15 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:27:05 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:26:04 +0300 Message-ID: <20210701132609.53727-11-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f4855cba-cdae-4c7e-2b7f-08d93c93f1a4 X-MS-TrafficTypeDiagnostic: CY4PR12MB1126: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:193; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iGEsWYXmq7W2zHZ5K6y3hvozWKEWWc72L3EpZlQUa1fOkqGnEswiwuXpsqRxyFo0G4QfNG3iqrOeQJqOr5mK3DrH58L4kKQPCN/9dBPpmy/pQ44S0F47B2ECjdYxXKSDcM95YGOCFO91ZKkcTkkm7a7fj3SdQZvXu8fVnxwXqyNCn216R9y8Yu1ZcmJQ78daQbE4Qy6xkPhEghS10ZqdZGAL3u51ypdl50cxgP+JBH+aFY+5uux20k3FyVGMHy/YzvCgSDLfFdNEwfbgnGvMm0iiKXMr9WBhgo2W5RQBleC5LHMIquTFMNEiaj/O3FW651J3gIPBvaSE3c5YDng/EH7I+Q6JBRNha9rm4/vhkKLLyREORtZ0IoRtpTacmQlYHbMRaoFBokva+s3byOV/pohe+6bnDVYpSOjO0LP/mw1HzvKsBpRuaz6FtgopN5cwGJcZ1Q8TBxkGd5LLXQ33cKRZiBXdBKx6txHJy7rQnRLsWuor2kjdtygI12pnmh+ofNqh/yfYrITt+TZCALbyx0jXFcFsFQHO+l0D+98RnAMmsU0r1Kiyyx1ciw0KSk8RtK1lBVH9CrWVfW7uhk4BQ+Ty0SGdZmZqGqApkoFnFTSkODu0TFVtSy+lPQ8LKLKkQMK2rHhdNp1lFwiv5frIal3NX6KqtMTpK8zzwysUFuPm6tI6ARQFdb/yuMYuKVDmiEN4eguG52h0UBK2Y1N8lg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39850400004)(346002)(136003)(376002)(396003)(36840700001)(46966006)(7696005)(5660300002)(2906002)(2616005)(8936002)(70206006)(47076005)(478600001)(1076003)(36756003)(36860700001)(82310400003)(8676002)(86362001)(6916009)(70586007)(36906005)(55016002)(54906003)(16526019)(426003)(336012)(7636003)(4326008)(107886003)(83380400001)(6286002)(26005)(316002)(356005)(82740400003)(186003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:15.9724 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f4855cba-cdae-4c7e-2b7f-08d93c93f1a4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1126 Subject: [dpdk-dev] [PATCH v5 10/15] crypto/mlx5: add maximum segments devarg X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Suanming Mou The mlx5 HW crypto operations are done by attaching crypto property to a memory region. Once done, every access to the memory via the crypto-enabled memory region will result with in-line encryption or decryption of the data. As a result, the design choice is to provide two types of WQEs. One is UMR WQE which sets the crypto property and the other is rdma write WQE which sends DMA command to copy data from local MR to remote MR. The size of the WQEs will be defined by a new devarg called max_segs_num. This devarg also defines the maximum segments in mbuf chain that will be supported for crypto operations. Signed-off-by: Suanming Mou Signed-off-by: Matan Azrad --- doc/guides/cryptodevs/mlx5.rst | 4 ++++ drivers/crypto/mlx5/mlx5_crypto.c | 33 +++++++++++++++++++++++++++---- drivers/crypto/mlx5/mlx5_crypto.h | 6 ++++++ 3 files changed, 39 insertions(+), 4 deletions(-) diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst index c3632484a5..dd4705b744 100644 --- a/doc/guides/cryptodevs/mlx5.rst +++ b/doc/guides/cryptodevs/mlx5.rst @@ -120,6 +120,10 @@ Driver options The plaintext of the keytag appanded to the AES-XTS keys, default value is 0. +- ``max_segs_num`` parameter [int] + + Maximum number of mbuf chain segments(src or dest), default value is 8. + Supported NICs -------------- diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index 6fd6bd1511..a6949775a9 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -21,6 +21,7 @@ #define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto #define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5 #define MLX5_CRYPTO_MAX_QPS 1024 +#define MLX5_CRYPTO_MAX_SEGS 56 #define MLX5_CRYPTO_FEATURE_FLAGS \ (RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | \ @@ -496,14 +497,24 @@ mlx5_crypto_args_check_handler(const char *key, const char *val, void *opaque) DRV_LOG(WARNING, "%s: \"%s\" is an invalid integer.", key, val); return -errno; } - if (strcmp(key, "import_kek_id") == 0) + if (strcmp(key, "max_segs_num") == 0) { + if (!tmp || tmp > MLX5_CRYPTO_MAX_SEGS) { + DRV_LOG(WARNING, "Invalid max_segs_num: %d, should" + " be less than %d.", + (uint32_t)tmp, MLX5_CRYPTO_MAX_SEGS); + rte_errno = EINVAL; + return -rte_errno; + } + devarg_prms->max_segs_num = (uint32_t)tmp; + } else if (strcmp(key, "import_kek_id") == 0) { attr->session_import_kek_ptr = (uint32_t)tmp; - else if (strcmp(key, "credential_id") == 0) + } else if (strcmp(key, "credential_id") == 0) { attr->credential_pointer = (uint32_t)tmp; - else if (strcmp(key, "keytag") == 0) + } else if (strcmp(key, "keytag") == 0) { devarg_prms->keytag = tmp; - else + } else { DRV_LOG(WARNING, "Invalid key %s.", key); + } return 0; } @@ -518,6 +529,7 @@ mlx5_crypto_parse_devargs(struct rte_devargs *devargs, attr->credential_pointer = 0; attr->session_import_kek_ptr = 0; devarg_prms->keytag = 0; + devarg_prms->max_segs_num = 8; if (devargs == NULL) { DRV_LOG(ERR, "No login devargs in order to enable crypto operations in the device."); @@ -614,6 +626,7 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, .max_nb_queue_pairs = RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS, }; + uint16_t rdmw_wqe_size; int ret; RTE_SET_USED(pci_drv); @@ -692,6 +705,18 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr; priv->mr_scache.dereg_mr_cb = mlx5_common_verbs_dereg_mr; priv->keytag = rte_cpu_to_be_64(devarg_prms.keytag); + priv->max_segs_num = devarg_prms.max_segs_num; + priv->umr_wqe_size = sizeof(struct mlx5_wqe_umr_bsf_seg) + + sizeof(struct mlx5_umr_wqe) + + RTE_ALIGN(priv->max_segs_num, 4) * + sizeof(struct mlx5_wqe_dseg); + rdmw_wqe_size = sizeof(struct mlx5_rdma_write_wqe) + + sizeof(struct mlx5_wqe_dseg) * + (priv->max_segs_num <= 2 ? 2 : 2 + + RTE_ALIGN(priv->max_segs_num - 2, 4)); + priv->wqe_set_size = priv->umr_wqe_size + rdmw_wqe_size; + priv->umr_wqe_stride = priv->umr_wqe_size / MLX5_SEND_WQE_BB; + priv->max_rdmar_ds = rdmw_wqe_size / sizeof(struct mlx5_wqe_dseg); /* Register callback function for global shared MR cache management. */ if (TAILQ_EMPTY(&mlx5_crypto_priv_list)) rte_mem_event_callback_register("MLX5_MEM_EVENT_CB", diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index ad70052967..48ca1cb9a2 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -25,12 +25,17 @@ struct mlx5_crypto_priv { struct rte_cryptodev *crypto_dev; void *uar; /* User Access Region. */ uint32_t pdn; /* Protection Domain number. */ + uint32_t max_segs_num; /* Maximum supported data segs. */ struct ibv_pd *pd; struct mlx5_hlist *dek_hlist; /* Dek hash list. */ struct rte_cryptodev_config dev_config; struct mlx5_mr_share_cache mr_scache; /* Global shared MR cache. */ struct mlx5_devx_obj *login_obj; uint64_t keytag; + uint16_t wqe_set_size; + uint16_t umr_wqe_size; + uint16_t umr_wqe_stride; + uint16_t max_rdmar_ds; }; struct mlx5_crypto_qp { @@ -54,6 +59,7 @@ struct mlx5_crypto_devarg_params { bool login_devarg; struct mlx5_devx_crypto_login_attr login_attr; uint64_t keytag; + uint32_t max_segs_num; }; int From patchwork Thu Jul 1 13:26:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95142 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 40F24A0A0C; Thu, 1 Jul 2021 15:28:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9F7984132A; Thu, 1 Jul 2021 15:27:24 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2050.outbound.protection.outlook.com [40.107.93.50]) by mails.dpdk.org (Postfix) with ESMTP id 35FD741308 for ; Thu, 1 Jul 2021 15:27:18 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WorzpeJrUmltpjfF3IC97EoJbWurl5h6tzO+F72HjVIVL1cRcN3sRpyQHoD4bSi0MmlPUBJ+i9nuH98hduNk9TB5IKeRLW1o7FFsRvnNZvla/1vskHOm4O3Rd5ZZ83VdjrJAxRwVVnnRkFRvEo9lwbHW02Om+r4WBaJl+7tqbzPw0W3bI0mtECiJUQB3Z7ff0NSFwViwhXJ35UBiJF97GhnsGkkxPstH4H8NN3HEY29qiJjADULv9vYZ7U5LaBzAPmEouDACxSnYhaxkM7j1YxwEaN7R7qw6oCf15SJHza+9Mew+/2LJn09SpudZpMSbYfPvrv/E28rzNscigGuy3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NRsd49RG/HSXOYyTWHchJVUOWdERwTqaZMF5t5dQmAc=; b=FbWooAskwOhqtGj2rofERigf3JDju8TpOHMoYeDfJzGgxCQZnZyHDiA/QAW1VX5FX6yyI0Kw1EqoyS1P7CXKf6XRzkyFoZd92jWt9SLLlBPgr5EHPp6VZzSpwINvRxgekC+zj4D3oJVLr4hGZezYlbChWYORvFOujk7yEFDmRqZ6v1Asd6jsmzAqGzinTxB0rm80Y1SbgGHy6nhw+3cRJW0RaIerr3EjvYOWF+ioJuM7rBySFmiCP/IvPazXbSB+xXuijYmf9Ub3fH4tWsD1OXG+inVGZIegFJJoel6NVM9GrK0m93dPRRuXVdfXNrz6NCev7Zc5tcGp1+sjOMHHJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NRsd49RG/HSXOYyTWHchJVUOWdERwTqaZMF5t5dQmAc=; b=GxV1IXvPgXbNb/jv8teEMuxLAKVKvitwuTZv6i7h8imKDZowulHl9lHD2hBI66iISmLWbLaM8h/Ql6Pp87aVrmVyLuvEXIpuOGKPCERldRNq5t5A5Y60wXVmDaIG04qMvn911Ft3ybiFcYShj5lmmt/GWV28npuldruiMJy6GrR1ksbgwYB5Ek2Nq6NfRTykjdFYcbO7u49CemzKB72QySyBWI8i3MyYYaUxfsXGs3IHsJrGNdz8qK9oVCQzFHYn9WyoGOx3cZwFDNoBddokG2M0L+Z3KOwQsU+wmep9xEOiGaYD/P5wRIcyJr5feUbcsu7pZs8WAFQl+jN+wdFEwg== Received: from BN8PR12CA0009.namprd12.prod.outlook.com (2603:10b6:408:60::22) by DM5PR12MB1595.namprd12.prod.outlook.com (2603:10b6:4:3::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.20; Thu, 1 Jul 2021 13:27:16 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::5d) by BN8PR12CA0009.outlook.office365.com (2603:10b6:408:60::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:16 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:27:07 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:26:05 +0300 Message-ID: <20210701132609.53727-12-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 06d762de-55d1-4f91-c18c-08d93c93f1fe X-MS-TrafficTypeDiagnostic: DM5PR12MB1595: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:295; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: DZ6zkf1S69CcpTUrWv1+bdaEMSoI9C5rF1hS4oQGu7XJ+eodzkt4+z4qXJ9QSYtmXJPuzPHF7irj5JnoUNq/6Y4QBfygEKoOYJsQtEET0/SuOWZEmh6J0vqueS43p1PgJ2EjWLBCceNCSRkGwWGzlBCcCVYgoSNOJndj8qMW9UVRixvTR32RpcmX+V15gA2uGxBOR6nJUGTsmdpxBHVRNvro9ZAF1vUKhIC3bGwcq+4OrgtwzcmeIHT+6SFtPLCipfR9l6ErLnVw8mf9ekmf3UcD/NH9g1BPMY2pYXnpyuxU/Z0fbAlKyZgvkzUvzfzTFH5DWdi3CcdkdCyYYIfxjiB+wHAO19C4o/bs1l9anpSi8UCXM5GSg7/UKQcbDTKWXutbOMLbB2xxsQnKFPvWB5IH7Z8AD0vtfwUbRzwSU7iJxyYK2GBvfd8aQrNY5rj63u15/GC6k7lEOdyIQqx3LoG6kt+QrGkaX2oXk7JIZgXVncxUKXEU89mdsqqwavwsPI45MVUFqkzC3zefOsgQ3wUnJLCjq5Ve4tayHMnYh8ImZGtJNWH98T3BJC3dmG+zqUw2woOpcWMNv0Fg8uktXpnc546R+nv8qooMc337+H6PGwe4fBEBueBW34d+lRGqu4Yy9yHalWYwhFmxaFO/uA/b6rv3H9TifGOsvjQ0/1Z8GCNbpJ+AN82F2JnCwxnMJ4tlBkOqxpfYEFumpe35kA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(136003)(346002)(39850400004)(396003)(36840700001)(46966006)(82740400003)(7636003)(2906002)(47076005)(83380400001)(356005)(70206006)(8936002)(186003)(26005)(16526019)(70586007)(7696005)(107886003)(1076003)(426003)(36906005)(36756003)(54906003)(316002)(336012)(4326008)(2616005)(82310400003)(36860700001)(5660300002)(6916009)(6286002)(55016002)(478600001)(8676002)(86362001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:16.5810 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 06d762de-55d1-4f91-c18c-08d93c93f1fe X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1595 Subject: [dpdk-dev] [PATCH v5 11/15] crypto/mlx5: add WQE set initialization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Suanming Mou Currently, HW handles the WQEs much faster than the software, Using the constant WQE set layout can initialize most of the WQE segments in advanced, and software only needs to configure very limited segments in datapath. This accelerates the software WQE organize in datapath. This commit initializes the fixed WQE set segments. Signed-off-by: Suanming Mou Signed-off-by: Matan Azrad --- drivers/crypto/mlx5/mlx5_crypto.c | 83 +++++++++++++++++++++++++++++-- drivers/crypto/mlx5/mlx5_crypto.h | 10 +++- 2 files changed, 87 insertions(+), 6 deletions(-) diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index a6949775a9..de45e38597 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -297,6 +297,69 @@ mlx5_crypto_qp2rts(struct mlx5_crypto_qp *qp) return 0; } +static void +mlx5_crypto_qp_init(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp) +{ + uint32_t i; + + for (i = 0 ; i < qp->entries_n; i++) { + struct mlx5_wqe_cseg *cseg = RTE_PTR_ADD(qp->umem_buf, i * + priv->wqe_set_size); + struct mlx5_wqe_umr_cseg *ucseg = (struct mlx5_wqe_umr_cseg *) + (cseg + 1); + struct mlx5_wqe_umr_bsf_seg *bsf = + (struct mlx5_wqe_umr_bsf_seg *)(RTE_PTR_ADD(cseg, + priv->umr_wqe_size)) - 1; + struct mlx5_wqe_rseg *rseg; + + /* Init UMR WQE. */ + cseg->sq_ds = rte_cpu_to_be_32((qp->qp_obj->id << 8) | + (priv->umr_wqe_size / MLX5_WSEG_SIZE)); + cseg->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR << + MLX5_COMP_MODE_OFFSET); + cseg->misc = rte_cpu_to_be_32(qp->mkey[i]->id); + ucseg->if_cf_toe_cq_res = RTE_BE32(1u << MLX5_UMRC_IF_OFFSET); + ucseg->mkey_mask = RTE_BE64(1u << 0); /* Mkey length bit. */ + ucseg->ko_to_bs = rte_cpu_to_be_32 + ((RTE_ALIGN(priv->max_segs_num, 4u) << + MLX5_UMRC_KO_OFFSET) | (4 << MLX5_UMRC_TO_BS_OFFSET)); + bsf->keytag = priv->keytag; + /* Init RDMA WRITE WQE. */ + cseg = RTE_PTR_ADD(cseg, priv->umr_wqe_size); + cseg->flags = RTE_BE32((MLX5_COMP_ALWAYS << + MLX5_COMP_MODE_OFFSET) | + MLX5_WQE_CTRL_INITIATOR_SMALL_FENCE); + rseg = (struct mlx5_wqe_rseg *)(cseg + 1); + rseg->rkey = rte_cpu_to_be_32(qp->mkey[i]->id); + } +} + +static int +mlx5_crypto_indirect_mkeys_prepare(struct mlx5_crypto_priv *priv, + struct mlx5_crypto_qp *qp) +{ + struct mlx5_umr_wqe *umr; + uint32_t i; + struct mlx5_devx_mkey_attr attr = { + .pd = priv->pdn, + .umr_en = 1, + .crypto_en = 1, + .set_remote_rw = 1, + .klm_num = RTE_ALIGN(priv->max_segs_num, 4), + }; + + for (umr = (struct mlx5_umr_wqe *)qp->umem_buf, i = 0; + i < qp->entries_n; i++, umr = RTE_PTR_ADD(umr, priv->wqe_set_size)) { + attr.klm_array = (struct mlx5_klm *)&umr->kseg[0]; + qp->mkey[i] = mlx5_devx_cmd_mkey_create(priv->ctx, &attr); + if (!qp->mkey[i]) { + DRV_LOG(ERR, "Failed to allocate indirect mkey."); + return -1; + } + } + return 0; +} + static int mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, const struct rte_cryptodev_qp_conf *qp_conf, @@ -307,7 +370,7 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, struct mlx5_crypto_qp *qp; uint16_t log_nb_desc = rte_log2_u32(qp_conf->nb_descriptors); uint32_t umem_size = RTE_BIT32(log_nb_desc) * - MLX5_CRYPTO_WQE_SET_SIZE + + priv->wqe_set_size + sizeof(*qp->db_rec) * 2; uint32_t alloc_size = sizeof(*qp); struct mlx5_devx_cq_attr cq_attr = { @@ -317,7 +380,9 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, if (dev->data->queue_pairs[qp_id] != NULL) mlx5_crypto_queue_pair_release(dev, qp_id); alloc_size = RTE_ALIGN(alloc_size, RTE_CACHE_LINE_SIZE); - alloc_size += sizeof(struct rte_crypto_op *) * RTE_BIT32(log_nb_desc); + alloc_size += (sizeof(struct rte_crypto_op *) + + sizeof(struct mlx5_devx_obj *)) * + RTE_BIT32(log_nb_desc); qp = rte_zmalloc_socket(__func__, alloc_size, RTE_CACHE_LINE_SIZE, socket_id); if (qp == NULL) { @@ -362,8 +427,7 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, attr.wq_umem_id = qp->umem_obj->umem_id; attr.wq_umem_offset = 0; attr.dbr_umem_id = qp->umem_obj->umem_id; - attr.dbr_address = RTE_BIT64(log_nb_desc) * - MLX5_CRYPTO_WQE_SET_SIZE; + attr.dbr_address = RTE_BIT64(log_nb_desc) * priv->wqe_set_size; qp->qp_obj = mlx5_devx_cmd_create_qp(priv->ctx, &attr); if (qp->qp_obj == NULL) { DRV_LOG(ERR, "Failed to create QP(%u).", rte_errno); @@ -372,8 +436,17 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, qp->db_rec = RTE_PTR_ADD(qp->umem_buf, (uintptr_t)attr.dbr_address); if (mlx5_crypto_qp2rts(qp)) goto error; - qp->ops = (struct rte_crypto_op **)RTE_ALIGN((uintptr_t)(qp + 1), + qp->mkey = (struct mlx5_devx_obj **)RTE_ALIGN((uintptr_t)(qp + 1), RTE_CACHE_LINE_SIZE); + qp->ops = (struct rte_crypto_op **)(qp->mkey + RTE_BIT32(log_nb_desc)); + qp->entries_n = 1 << log_nb_desc; + if (mlx5_crypto_indirect_mkeys_prepare(priv, qp)) { + DRV_LOG(ERR, "Cannot allocate indirect memory regions."); + rte_errno = ENOMEM; + goto error; + } + mlx5_crypto_qp_init(priv, qp); + qp->priv = priv; dev->data->queue_pairs[qp_id] = qp; return 0; error: diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index 48ca1cb9a2..12e5890bf7 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -16,7 +16,6 @@ #define MLX5_CRYPTO_DEK_HTABLE_SZ (1 << 11) #define MLX5_CRYPTO_KEY_LENGTH 80 -#define MLX5_CRYPTO_WQE_SET_SIZE 1024 struct mlx5_crypto_priv { TAILQ_ENTRY(mlx5_crypto_priv) next; @@ -24,6 +23,7 @@ struct mlx5_crypto_priv { struct rte_pci_device *pci_dev; struct rte_cryptodev *crypto_dev; void *uar; /* User Access Region. */ + volatile uint64_t *uar_addr; uint32_t pdn; /* Protection Domain number. */ uint32_t max_segs_num; /* Maximum supported data segs. */ struct ibv_pd *pd; @@ -39,13 +39,21 @@ struct mlx5_crypto_priv { }; struct mlx5_crypto_qp { + struct mlx5_crypto_priv *priv; struct mlx5_devx_cq cq_obj; struct mlx5_devx_obj *qp_obj; + struct rte_cryptodev_stats stats; struct mlx5dv_devx_umem *umem_obj; void *umem_buf; volatile uint32_t *db_rec; struct rte_crypto_op **ops; + struct mlx5_devx_obj **mkey; /* WQE's indirect mekys. */ struct mlx5_mr_ctrl mr_ctrl; + uint8_t *wqe; + uint16_t entries_n; + uint16_t pi; + uint16_t ci; + uint16_t db_pi; }; struct mlx5_crypto_dek { From patchwork Thu Jul 1 13:26:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95143 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 48658A0A0C; Thu, 1 Jul 2021 15:28:23 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1E8B94132F; Thu, 1 Jul 2021 15:27:26 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2056.outbound.protection.outlook.com [40.107.236.56]) by mails.dpdk.org (Postfix) with ESMTP id ACC69412FA for ; Thu, 1 Jul 2021 15:27:18 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bqXY0xOtzU4nGWnnr+We+Y9aQLpHGFK/FTmDeOUsFnUMn6dpDs5MgU0qm+QotgUFfQSpNdf9V0A2mATDGHsLp+lqVZsz0Kp6jEcjpQXPxKljbFF5nshFelNvMy+dj1pPV0oZ6PTrfTmuG6+50MYgC6DI/8fCFLCNGjs9k7QKKZKW70t6hL+flzkzdq9y0MdePqIFkxY0RZhNjQeFGuHwUH3i43/2nU52ouyaZqUSQDPZ8FWObN7bd9O9LVnAgqTsbr+A+3gwlgYLj1X0tm2ZbeE3mvTCGuYwzE3IZC8784bShdjrav67vQe47Svqp201dZQLl2s1ej1N2Ypkp76qYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KycYo8uDXjSCd6RxgRufslfbCZ9jVD4aCCDRq8YftlU=; b=YoFCi2cYn8uM323XdESH9mEOJGQMTSbXy5E1sNRdFSgzAnQhVq9+9fmREnzpN7ekkNdmkH9D6BNzMbqCQb5zZw5zYfYxoymeVXNIJnWELhwcMep3b/2X+sARmnLUhyBI+PYhFWx6VYptoEfRtgB6CjLN7yPu/1sKYVqrm4KObrhnXN9ht8FXiuVbRdbVjBhVT9TRmwlY+FWE8k8OWnAJbq531okLzzPEe9T5XcMFBl9VaM5IqWVvxCmoDpyWCY8PtZL3hH+JrrsZRUS8uIajswiVbs7wphcuNpTDiUcF4OFyW5Pt88eRrUUrsuaW5RjoEeGEQ5Wf/BLIy2x5vGdNbQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KycYo8uDXjSCd6RxgRufslfbCZ9jVD4aCCDRq8YftlU=; b=h+3/gisAlevaU2ynCMGxDScCzBuxO0DbzXVmh7Z7ncmcXT36bd7pl2K1AOVrbfAvLUIVvPV6YkQRS4FqDrBq6teptzJrixAwNCZkj0uN0LrZAkYxHPkwaU7zr9ZO42x/5TyBeLV1WU9sV3UBEufoAxreQSCGP2n6AOz3e/60bu02A0tWmNWTE246a2R4lxw3JCqFd8yqDIMRwMeubuJIDRrBT3LQNhHGSRpTkxLHBWlw5kDGc+Ube6edltggC9UV2vuCph6rSW4ZPr4/sHVLm4FrJ3cd1mkXzPlMgdInKawsdcSw0LYXzcp1pZZA4I3KYc0Ny7Eq7utOJyMzUAe+CQ== Received: from BN8PR12CA0002.namprd12.prod.outlook.com (2603:10b6:408:60::15) by CY4PR12MB1126.namprd12.prod.outlook.com (2603:10b6:903:3d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.20; Thu, 1 Jul 2021 13:27:17 +0000 Received: from BN8NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::d4) by BN8PR12CA0002.outlook.office365.com (2603:10b6:408:60::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT042.mail.protection.outlook.com (10.13.177.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:17 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:27:08 +0000 From: Shiri Kuzin To: CC: , , , "Michael Baum" Date: Thu, 1 Jul 2021 16:26:06 +0300 Message-ID: <20210701132609.53727-13-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ec3dffee-9131-4db4-726f-08d93c93f24d X-MS-TrafficTypeDiagnostic: CY4PR12MB1126: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:316; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Z2T5mlh019p+0SvOsF9yYs+VX986uRWUDIIgpPF0kIBeqkOLeIuwfXVD1nFKJIRw++2zUt+B8X0wESE+SEGN3Q86FhFqb/iXdWowVPjePJvzROMCOcZsNgoH+iX+KdMawMHaByUK4GKqHXSAHWIdaAXNYRdqJuKyD3XSvEa8X4v4/GpZvAr2qosz786LmOF6qLo1XS+Nvf8OXdQpqxEL1Ql4gRLc1WdxdjMjlIkMyAv7W8aPruR5S641VkevgUjCw6HrfWDID//6AV7qiWJSi55daFw/iKFoqDAFFavT4sat/NkI29dicmvUc0Ny8SvE+G6CZ/V+om4ZldO99lGjKayRMckDr43lERAb/setZmU98FNyP8ZeGFgSEeLpFMaxT+2wgHw0zIM4DsTzv5iGYzJoRSQieyb257XKLlT2dZDN3BJ+gNw+XElh99vE6QCACiZefBf3x2NlEO3nQCHFnU2AAFc9/GP9B8GjttnHTIwv1scjJkiNJO8Qj/PSf2UJT07j10b9A9NDxvTb6JRdkj/IhY6HatY9cQExEujGo9pZpM4wKcCNbqhL964i23CxGkuWA0CFpKKtNtepM90+gWVlPKHyYQolVSssr+6FLMqvdVuZgPjykv+Y3VvxQcr6jf9+iQ2S+mhSdEBs/WpXN1OfSgldv3MYFKx8fmiyegJouSl7IZnvqTzz7+sAuIugquMiOV4aLdX+heenwnQfGA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39850400004)(346002)(136003)(376002)(396003)(36840700001)(46966006)(7696005)(5660300002)(2906002)(2616005)(8936002)(70206006)(47076005)(478600001)(1076003)(36756003)(36860700001)(82310400003)(8676002)(30864003)(86362001)(6916009)(70586007)(36906005)(55016002)(54906003)(16526019)(426003)(336012)(7636003)(4326008)(107886003)(83380400001)(6286002)(26005)(316002)(356005)(82740400003)(186003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:17.0817 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ec3dffee-9131-4db4-726f-08d93c93f24d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1126 Subject: [dpdk-dev] [PATCH v5 12/15] crypto/mlx5: add enqueue and dequeue operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Suanming Mou The crypto operations are done with the WQE set which contains one UMR WQE and one rdma write WQE. Most segments of the WQE set are initialized properly during queue setup, only limited segments are initialized according to the crypto detail in the datapath process. This commit adds the enquue and dequeue operations and updates the WQE set segments accordingly. Signed-off-by: Suanming Mou Signed-off-by: Matan Azrad Signed-off-by: Michael Baum --- doc/guides/cryptodevs/features/mlx5.ini | 5 + drivers/crypto/mlx5/mlx5_crypto.c | 286 +++++++++++++++++++++++- drivers/crypto/mlx5/mlx5_crypto.h | 3 + 3 files changed, 290 insertions(+), 4 deletions(-) diff --git a/doc/guides/cryptodevs/features/mlx5.ini b/doc/guides/cryptodevs/features/mlx5.ini index bd757b5211..a89526add0 100644 --- a/doc/guides/cryptodevs/features/mlx5.ini +++ b/doc/guides/cryptodevs/features/mlx5.ini @@ -6,6 +6,11 @@ [Features] Symmetric crypto = Y HW Accelerated = Y +In Place SGL = Y +OOP SGL In SGL Out = Y +OOP SGL In LB Out = Y +OOP LB In SGL Out = Y +OOP LB In LB Out = Y Cipher multiple data units = Y Cipher wrapped key = Y diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index de45e38597..47dbb9c385 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -25,6 +25,10 @@ #define MLX5_CRYPTO_FEATURE_FLAGS \ (RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | \ + RTE_CRYPTODEV_FF_IN_PLACE_SGL | RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT | \ + RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT | \ + RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT | \ + RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT | \ RTE_CRYPTODEV_FF_CIPHER_WRAPPED_KEY | \ RTE_CRYPTODEV_FF_CIPHER_MULTIPLE_DATA_UNITS) @@ -297,6 +301,279 @@ mlx5_crypto_qp2rts(struct mlx5_crypto_qp *qp) return 0; } +static __rte_noinline uint32_t +mlx5_crypto_get_block_size(struct rte_crypto_op *op) +{ + uint32_t bl = op->sym->cipher.data.length; + + switch (bl) { + case (1 << 20): + return RTE_BE32(MLX5_BLOCK_SIZE_1MB << MLX5_BLOCK_SIZE_OFFSET); + case (1 << 12): + return RTE_BE32(MLX5_BLOCK_SIZE_4096B << + MLX5_BLOCK_SIZE_OFFSET); + case (1 << 9): + return RTE_BE32(MLX5_BLOCK_SIZE_512B << MLX5_BLOCK_SIZE_OFFSET); + default: + DRV_LOG(ERR, "Unknown block size: %u.", bl); + return UINT32_MAX; + } +} + +/** + * Query LKey from a packet buffer for QP. If not found, add the mempool. + * + * @param priv + * Pointer to the priv object. + * @param addr + * Search key. + * @param mr_ctrl + * Pointer to per-queue MR control structure. + * @param ol_flags + * Mbuf offload features. + * + * @return + * Searched LKey on success, UINT32_MAX on no match. + */ +static __rte_always_inline uint32_t +mlx5_crypto_addr2mr(struct mlx5_crypto_priv *priv, uintptr_t addr, + struct mlx5_mr_ctrl *mr_ctrl, uint64_t ol_flags) +{ + uint32_t lkey; + + /* Check generation bit to see if there's any change on existing MRs. */ + if (unlikely(*mr_ctrl->dev_gen_ptr != mr_ctrl->cur_gen)) + mlx5_mr_flush_local_cache(mr_ctrl); + /* Linear search on MR cache array. */ + lkey = mlx5_mr_lookup_lkey(mr_ctrl->cache, &mr_ctrl->mru, + MLX5_MR_CACHE_N, addr); + if (likely(lkey != UINT32_MAX)) + return lkey; + /* Take slower bottom-half on miss. */ + return mlx5_mr_addr2mr_bh(priv->pd, 0, &priv->mr_scache, mr_ctrl, addr, + !!(ol_flags & EXT_ATTACHED_MBUF)); +} + +static __rte_always_inline uint32_t +mlx5_crypto_klm_set(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp, + struct rte_mbuf *mbuf, struct mlx5_wqe_dseg *klm, + uint32_t offset, uint32_t *remain) +{ + uint32_t data_len = (rte_pktmbuf_data_len(mbuf) - offset); + uintptr_t addr = rte_pktmbuf_mtod_offset(mbuf, uintptr_t, offset); + + if (data_len > *remain) + data_len = *remain; + *remain -= data_len; + klm->bcount = rte_cpu_to_be_32(data_len); + klm->pbuf = rte_cpu_to_be_64(addr); + klm->lkey = mlx5_crypto_addr2mr(priv, addr, &qp->mr_ctrl, + mbuf->ol_flags); + return klm->lkey; + +} + +static __rte_always_inline uint32_t +mlx5_crypto_klms_set(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp, + struct rte_crypto_op *op, struct rte_mbuf *mbuf, + struct mlx5_wqe_dseg *klm) +{ + uint32_t remain_len = op->sym->cipher.data.length; + uint32_t nb_segs = mbuf->nb_segs; + uint32_t klm_n = 1u; + + /* First mbuf needs to take the cipher offset. */ + if (unlikely(mlx5_crypto_klm_set(priv, qp, mbuf, klm, + op->sym->cipher.data.offset, &remain_len) == UINT32_MAX)) { + op->status = RTE_CRYPTO_OP_STATUS_ERROR; + return 0; + } + while (remain_len) { + nb_segs--; + mbuf = mbuf->next; + if (unlikely(mbuf == NULL || nb_segs == 0)) { + op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; + return 0; + } + if (unlikely(mlx5_crypto_klm_set(priv, qp, mbuf, ++klm, 0, + &remain_len) == UINT32_MAX)) { + op->status = RTE_CRYPTO_OP_STATUS_ERROR; + return 0; + } + klm_n++; + } + return klm_n; +} + +static __rte_always_inline int +mlx5_crypto_wqe_set(struct mlx5_crypto_priv *priv, + struct mlx5_crypto_qp *qp, + struct rte_crypto_op *op, + struct mlx5_umr_wqe *umr) +{ + struct mlx5_crypto_session *sess = get_sym_session_private_data + (op->sym->session, mlx5_crypto_driver_id); + struct mlx5_wqe_cseg *cseg = &umr->ctr; + struct mlx5_wqe_mkey_cseg *mkc = &umr->mkc; + struct mlx5_wqe_dseg *klms = &umr->kseg[0]; + struct mlx5_wqe_umr_bsf_seg *bsf = ((struct mlx5_wqe_umr_bsf_seg *) + RTE_PTR_ADD(umr, priv->umr_wqe_size)) - 1; + uint32_t ds; + bool ipl = op->sym->m_dst == NULL || op->sym->m_dst == op->sym->m_src; + /* Set UMR WQE. */ + uint32_t klm_n = mlx5_crypto_klms_set(priv, qp, op, + ipl ? op->sym->m_src : op->sym->m_dst, klms); + + if (unlikely(klm_n == 0)) + return 0; + bsf->bs_bpt_eo_es = sess->bs_bpt_eo_es; + if (unlikely(!sess->bsp_res)) { + bsf->bsp_res = mlx5_crypto_get_block_size(op); + if (unlikely(bsf->bsp_res == UINT32_MAX)) { + op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; + return 0; + } + } else { + bsf->bsp_res = sess->bsp_res; + } + bsf->raw_data_size = rte_cpu_to_be_32(op->sym->cipher.data.length); + memcpy(bsf->xts_initial_tweak, + rte_crypto_op_ctod_offset(op, uint8_t *, sess->iv_offset), 16); + bsf->res_dp = sess->dek_id; + mkc->len = rte_cpu_to_be_64(op->sym->cipher.data.length); + cseg->opcode = rte_cpu_to_be_32((qp->db_pi << 8) | MLX5_OPCODE_UMR); + qp->db_pi += priv->umr_wqe_stride; + /* Set RDMA_WRITE WQE. */ + cseg = RTE_PTR_ADD(cseg, priv->umr_wqe_size); + klms = RTE_PTR_ADD(cseg, sizeof(struct mlx5_rdma_write_wqe)); + if (!ipl) { + klm_n = mlx5_crypto_klms_set(priv, qp, op, op->sym->m_src, + klms); + if (unlikely(klm_n == 0)) + return 0; + } else { + memcpy(klms, &umr->kseg[0], sizeof(*klms) * klm_n); + } + ds = 2 + klm_n; + cseg->sq_ds = rte_cpu_to_be_32((qp->qp_obj->id << 8) | ds); + cseg->opcode = rte_cpu_to_be_32((qp->db_pi << 8) | + MLX5_OPCODE_RDMA_WRITE); + ds = RTE_ALIGN(ds, 4); + qp->db_pi += ds >> 2; + /* Set NOP WQE if needed. */ + if (priv->max_rdmar_ds > ds) { + cseg += ds; + ds = priv->max_rdmar_ds - ds; + cseg->sq_ds = rte_cpu_to_be_32((qp->qp_obj->id << 8) | ds); + cseg->opcode = rte_cpu_to_be_32((qp->db_pi << 8) | + MLX5_OPCODE_NOP); + qp->db_pi += ds >> 2; /* Here, DS is 4 aligned for sure. */ + } + qp->wqe = (uint8_t *)cseg; + return 1; +} + +static __rte_always_inline void +mlx5_crypto_uar_write(uint64_t val, struct mlx5_crypto_priv *priv) +{ +#ifdef RTE_ARCH_64 + *priv->uar_addr = val; +#else /* !RTE_ARCH_64 */ + rte_spinlock_lock(&priv->uar32_sl); + *(volatile uint32_t *)priv->uar_addr = val; + rte_io_wmb(); + *((volatile uint32_t *)priv->uar_addr + 1) = val >> 32; + rte_spinlock_unlock(&priv->uar32_sl); +#endif +} + +static uint16_t +mlx5_crypto_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + struct mlx5_crypto_qp *qp = queue_pair; + struct mlx5_crypto_priv *priv = qp->priv; + struct mlx5_umr_wqe *umr; + struct rte_crypto_op *op; + uint16_t mask = qp->entries_n - 1; + uint16_t remain = qp->entries_n - (qp->pi - qp->ci); + + if (remain < nb_ops) + nb_ops = remain; + else + remain = nb_ops; + if (unlikely(remain == 0)) + return 0; + do { + op = *ops++; + umr = RTE_PTR_ADD(qp->umem_buf, priv->wqe_set_size * qp->pi); + if (unlikely(mlx5_crypto_wqe_set(priv, qp, op, umr) == 0)) { + if (remain != nb_ops) + break; + return 0; + } + qp->ops[qp->pi] = op; + qp->pi = (qp->pi + 1) & mask; + } while (--remain); + rte_io_wmb(); + qp->db_rec[MLX5_SND_DBR] = rte_cpu_to_be_32(qp->db_pi); + rte_wmb(); + mlx5_crypto_uar_write(*(volatile uint64_t *)qp->wqe, qp->priv); + rte_wmb(); + return nb_ops; +} + +static __rte_noinline void +mlx5_crypto_cqe_err_handle(struct mlx5_crypto_qp *qp, struct rte_crypto_op *op) +{ + const uint32_t idx = qp->ci & (qp->entries_n - 1); + volatile struct mlx5_err_cqe *cqe = (volatile struct mlx5_err_cqe *) + &qp->cq_obj.cqes[idx]; + + op->status = RTE_CRYPTO_OP_STATUS_ERROR; + DRV_LOG(ERR, "CQE ERR:%x.\n", rte_be_to_cpu_32(cqe->syndrome)); +} + +static uint16_t +mlx5_crypto_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + struct mlx5_crypto_qp *qp = queue_pair; + volatile struct mlx5_cqe *restrict cqe; + struct rte_crypto_op *restrict op; + const unsigned int cq_size = qp->entries_n; + const unsigned int mask = cq_size - 1; + uint32_t idx; + uint32_t next_idx = qp->ci & mask; + const uint16_t max = RTE_MIN((uint16_t)(qp->pi - qp->ci), nb_ops); + uint16_t i = 0; + int ret; + + if (unlikely(max == 0)) + return 0; + do { + idx = next_idx; + next_idx = (qp->ci + 1) & mask; + op = qp->ops[idx]; + cqe = &qp->cq_obj.cqes[idx]; + ret = check_cqe(cqe, cq_size, qp->ci); + rte_io_rmb(); + if (unlikely(ret != MLX5_CQE_STATUS_SW_OWN)) { + if (unlikely(ret != MLX5_CQE_STATUS_HW_OWN)) + mlx5_crypto_cqe_err_handle(qp, op); + break; + } + op->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + ops[i++] = op; + qp->ci++; + } while (i < max); + if (likely(i != 0)) { + rte_io_wmb(); + qp->cq_obj.db_rec[0] = rte_cpu_to_be_32(qp->ci); + } + return i; +} + static void mlx5_crypto_qp_init(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp) { @@ -521,8 +798,9 @@ mlx5_crypto_hw_global_prepare(struct mlx5_crypto_priv *priv) if (mlx5_crypto_pd_create(priv) != 0) return -1; priv->uar = mlx5_devx_alloc_uar(priv->ctx, -1); - if (priv->uar == NULL || mlx5_os_get_devx_uar_reg_addr(priv->uar) == - NULL) { + if (priv->uar) + priv->uar_addr = mlx5_os_get_devx_uar_reg_addr(priv->uar); + if (priv->uar == NULL || priv->uar_addr == NULL) { rte_errno = errno; claim_zero(mlx5_glue->dealloc_pd(priv->pd)); DRV_LOG(ERR, "Failed to allocate UAR."); @@ -752,8 +1030,8 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, DRV_LOG(INFO, "Crypto device %s was created successfully.", ibv->name); crypto_dev->dev_ops = &mlx5_crypto_ops; - crypto_dev->dequeue_burst = NULL; - crypto_dev->enqueue_burst = NULL; + crypto_dev->dequeue_burst = mlx5_crypto_dequeue_burst; + crypto_dev->enqueue_burst = mlx5_crypto_enqueue_burst; crypto_dev->feature_flags = MLX5_CRYPTO_FEATURE_FLAGS; crypto_dev->driver_id = mlx5_crypto_driver_id; priv = crypto_dev->data->dev_private; diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index 12e5890bf7..8e90735182 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -36,6 +36,9 @@ struct mlx5_crypto_priv { uint16_t umr_wqe_size; uint16_t umr_wqe_stride; uint16_t max_rdmar_ds; +#ifndef RTE_ARCH_64 + rte_spinlock_t uar32_sl; +#endif /* RTE_ARCH_64 */ }; struct mlx5_crypto_qp { From patchwork Thu Jul 1 13:26:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95145 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F0920A0A0C; Thu, 1 Jul 2021 15:28:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 02B49412FA; Thu, 1 Jul 2021 15:27:31 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2079.outbound.protection.outlook.com [40.107.223.79]) by mails.dpdk.org (Postfix) with ESMTP id 0C5924133D for ; Thu, 1 Jul 2021 15:27:29 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FDAR+B9+BK/kPqKOPyv+yvfAzFaFnjW6YYpjzgvKdZw7c6h5udk7lPSig4So/2aX5uOMumdd8IadEfykSfWs5JVnLvlKGhoV24N4Z/VBfYE0wIo/uvwxqugeVXEwAOY0AfHyGBvKcL5PfAuWi01Gl3+9HTLQZKEUwEc5qwIU88eM5WwLw6zu7Zz7d3lkbkhAKgmxbBveR6ETKWJO9YBCwh9NMxOat+v873N/t4wcN6TXP31tzXP+zxi+h3ghnzrGm4EMP0ayJYJsYrwvg5P2cbRzLJnN2aWIKHAWWXltstYBTF1irpr3p2A+dl3EWuwon9b+6PIqYltRsZgvdrFVbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pvLiO4CnKr25IQSmFqCEuJwqTSV4dD5fF5xJ41Kn2WI=; b=fkIxLT14HCro0tYGjpgNXrxlTu4NbZzOEbdKnb2e6tlfEi3Z7e7Vwj/2RZX9KccJTGtEZp/4V0bPkmXvHl5uEUEY+3K5LbBl7/+tU3gmwxKjyhHtCvt6MCysVXAkotASullaDdbmv8CPLsuxnZoFkohfFEp889P4GAR3EI0jqIz8othjDaSHVPsuNLL1dfwhohkxDNCzC1dTJcG3TtrAtk9SH/DUXGFAnIiul+B48u+HMohFTdYTDXQ7cyBXahVeyBZE7aRhRd3z1SSbYKS2jEUK9TS++evIalCmnEcklgSR/GZmSQ0gZtYk8aMcuxhry3KID/OOkwUzjnCRApFx6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pvLiO4CnKr25IQSmFqCEuJwqTSV4dD5fF5xJ41Kn2WI=; b=ON1a2bIaKQORXCLRIT5o1+9r4vvW4tDveG6NuDkai2IT5xW0/m8imPgwNOMY9VEutfloQEYI/5CDLUGn5im0jSQ43p5zDYnDbND0iDa3ZBEx59TBLEYN/qCVQjajtxVI0cYYDESGcXDDUzMk4jamQJ64svSqUx3OuJzFScnFYd8lJqvKAynfJxaGiMG/ATECBhVRtY5y6Szx/6QkUJXwIspJR4kptFDW94/8+8z6GbN/4qrpgjShtuArPQuRtO8XkItf2yenq1idD6qE7/CqESY4kpu04vMk1WeYrR8ioz9hII/pS7wekkMotgjAEHPUT8pZZ9xCjDK3kMc6RGc9xw== Received: from BN9PR03CA0471.namprd03.prod.outlook.com (2603:10b6:408:139::26) by BL0PR12MB4706.namprd12.prod.outlook.com (2603:10b6:208:82::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.23; Thu, 1 Jul 2021 13:27:20 +0000 Received: from BN8NAM11FT055.eop-nam11.prod.protection.outlook.com (2603:10b6:408:139:cafe::c4) by BN9PR03CA0471.outlook.office365.com (2603:10b6:408:139::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.23 via Frontend Transport; Thu, 1 Jul 2021 13:27:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT055.mail.protection.outlook.com (10.13.177.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:19 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:27:10 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:26:07 +0300 Message-ID: <20210701132609.53727-14-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 464cf2aa-9f91-493e-41c6-08d93c93f3b9 X-MS-TrafficTypeDiagnostic: BL0PR12MB4706: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:158; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HMqYUWnLcWWgMTbFe23B2YxR4H4l3I1VEj5rig8oOLOLVE6eU1R83eV8HwlsPO60x9JjJ1wHRkBXQ0dJwnoKgr4wYYarPXLlVgLn+nODXBSa2alRVmq64tpPXKc5knpHxvs5nbaeUXyWJa7endHt1fDH5dGtK0o/OJNdj/4WbpVnGL44oEokEhZjzFFCXRbVw1T60VeDxSXBEtP/2/ReMSCEseBjWkp5+FTW4yM+LiZBogUELht1S6tYnI6ZMj/u/ijFObaxeDkD1SlYDvOY215tUxCiwN4HcOjyscPgtLG6y0YJ67ebYLhkKaOKRIauZD+ZNOJG81lNl1zyKRsd6atHFz9IPeZv5e4ysvyGNZuJAjYgIF8ULJMOy/P4EQCA/Ddc2gwaHMtxNtXgu4PrjmvcnBvnmrEc8kib8eMq+4P3msZllyBmFEBLq0IbTrDvJP0D1PzYBLGz5Fi/WtikiT8dQVibwnErBh91YElto6h1no//lbr8EzSncRWmkIb8uK76Pb1gJGiusv3kBAzyjH5aV+VHV8v/iV/UaQHALQb6S0z+IfHsIpmG3Wt2DYkjzkc4It36EebrKBxcZRVK4QqhXrpkqf7voim8rxBm3aj7ciARtK0sJZ0TPsv74BXe8BR1uVc/vskWhX4kHvI6H0GzMlzLSgn71+oLdODYxWI5qQvTJnkLSJ13yKSj9BNYHL+8cSiBxa7H3/BMxRtuFg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(376002)(396003)(346002)(136003)(46966006)(36840700001)(83380400001)(55016002)(2906002)(70586007)(16526019)(70206006)(107886003)(2616005)(8936002)(1076003)(4326008)(426003)(356005)(36860700001)(186003)(8676002)(6286002)(6666004)(336012)(7636003)(26005)(36756003)(316002)(86362001)(82310400003)(5660300002)(54906003)(82740400003)(6916009)(7696005)(47076005)(478600001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:19.3001 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 464cf2aa-9f91-493e-41c6-08d93c93f3b9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT055.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4706 Subject: [dpdk-dev] [PATCH v5 13/15] crypto/mlx5: add statistic get and reset operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Suanming Mou This commit adds mlx5 crypto statistic get and reset operations. Signed-off-by: Suanming Mou Signed-off-by: Matan Azrad --- drivers/crypto/mlx5/mlx5_crypto.c | 40 ++++++++++++++++++++++++++++--- 1 file changed, 37 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index 47dbb9c385..44273dd2cd 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -508,13 +508,17 @@ mlx5_crypto_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, op = *ops++; umr = RTE_PTR_ADD(qp->umem_buf, priv->wqe_set_size * qp->pi); if (unlikely(mlx5_crypto_wqe_set(priv, qp, op, umr) == 0)) { - if (remain != nb_ops) + qp->stats.enqueue_err_count++; + if (remain != nb_ops) { + qp->stats.enqueued_count -= remain; break; + } return 0; } qp->ops[qp->pi] = op; qp->pi = (qp->pi + 1) & mask; } while (--remain); + qp->stats.enqueued_count += nb_ops; rte_io_wmb(); qp->db_rec[MLX5_SND_DBR] = rte_cpu_to_be_32(qp->db_pi); rte_wmb(); @@ -531,6 +535,7 @@ mlx5_crypto_cqe_err_handle(struct mlx5_crypto_qp *qp, struct rte_crypto_op *op) &qp->cq_obj.cqes[idx]; op->status = RTE_CRYPTO_OP_STATUS_ERROR; + qp->stats.dequeue_err_count++; DRV_LOG(ERR, "CQE ERR:%x.\n", rte_be_to_cpu_32(cqe->syndrome)); } @@ -570,6 +575,7 @@ mlx5_crypto_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, if (likely(i != 0)) { rte_io_wmb(); qp->cq_obj.db_rec[0] = rte_cpu_to_be_32(qp->ci); + qp->stats.dequeued_count += i; } return i; } @@ -731,14 +737,42 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, return -1; } +static void +mlx5_crypto_stats_get(struct rte_cryptodev *dev, + struct rte_cryptodev_stats *stats) +{ + int qp_id; + + for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) { + struct mlx5_crypto_qp *qp = dev->data->queue_pairs[qp_id]; + + stats->enqueued_count += qp->stats.enqueued_count; + stats->dequeued_count += qp->stats.dequeued_count; + stats->enqueue_err_count += qp->stats.enqueue_err_count; + stats->dequeue_err_count += qp->stats.dequeue_err_count; + } +} + +static void +mlx5_crypto_stats_reset(struct rte_cryptodev *dev) +{ + int qp_id; + + for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) { + struct mlx5_crypto_qp *qp = dev->data->queue_pairs[qp_id]; + + memset(&qp->stats, 0, sizeof(qp->stats)); + } +} + static struct rte_cryptodev_ops mlx5_crypto_ops = { .dev_configure = mlx5_crypto_dev_configure, .dev_start = mlx5_crypto_dev_start, .dev_stop = mlx5_crypto_dev_stop, .dev_close = mlx5_crypto_dev_close, .dev_infos_get = mlx5_crypto_dev_infos_get, - .stats_get = NULL, - .stats_reset = NULL, + .stats_get = mlx5_crypto_stats_get, + .stats_reset = mlx5_crypto_stats_reset, .queue_pair_setup = mlx5_crypto_queue_pair_setup, .queue_pair_release = mlx5_crypto_queue_pair_release, .sym_session_get_size = mlx5_crypto_sym_session_get_size, From patchwork Thu Jul 1 13:26:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95146 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31F48A0A0C; Thu, 1 Jul 2021 15:28:44 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 308C641333; Thu, 1 Jul 2021 15:27:32 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2069.outbound.protection.outlook.com [40.107.92.69]) by mails.dpdk.org (Postfix) with ESMTP id 30BB7412EC for ; Thu, 1 Jul 2021 15:27:30 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VsOD0F7mJ3k3/SiBBOs7ubO4HfNaTEYxBiNFcrH+4YDDSesaA4kuq6xaWDuPxrlxV6UI3eFw/kXhvbtwJ8Oww7x4ZFje74nl7KeLLX4ZgjnrKLudZIfVwmdI9s4ZnZa66Bkj5oiaK+fGduR64g3sEcakbdThPvstova0QvJ94Dk+0tsoC0yG2A8EpLcAh/Gh4/OcsAvC1Kd4Sbj+1uJ1GpkC6oE2XlOcZIUIPVEHh3l9/bvRHIBCWomv6r7HA+RuwRAHE1vaCCuEta2VJl47XC+FVnCIHpoqCQSaXo4IZQGFBRNK4zXb9J18peZxqBixqtKGQJo8c5HtpZZsAQMaAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uV7OfZyQp+XUIzTD5Zpsgu5GCOQl7CYLamsmMEoKI0M=; b=e9wZ/Q86gQYTITNbojBNzJhhItsPv4FHWUe/YP+YrkyDKB0L0AGufC/jtayNV8tV+mk1sJN4dNNlKaE8j1Dps/zio0GgwJZ+dT24Qe+rWWh9RVWyZ8rqRkybQ1Wy0sIkVIBK0xRovJYkMDa3o/bpD4EbTa8XFvUxcZSIZ/RitPNZK8nj4WoXltJuYjR5DCZNuNq8XP06iHtenOIBLDxQzQQNKod9qQyPFOHQNRRcPaM4ar2Z8hu51bt3uxIO3mxsMXWFtHmuKv8Otxo1kfuSa1Zq1A0avOGUhEHYf2O5UZLeh7lQlQo/9srOPef/5enTp6zdQEc6sC9MvvKrl3dBlA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uV7OfZyQp+XUIzTD5Zpsgu5GCOQl7CYLamsmMEoKI0M=; b=BLGsy4eirCP3sVIotmRkOJ3JHF7lCw20/NkDYWa/AN8ZsCXNp7icFmrWeJKQT+eLhT21Yca01ADQwR8neYwHZ+BkQThU00BnkYAOOqwAhk1/OwA6yJfXSX/Tbnguy3/BNd53EtZdDZuwqkcBeiUXlkASgORYM5niLApCuOPECHtYKVEz1uDJsixi1ct8vH5x+7uJsIootYOeSyz/ymxnRvcT5nVbKdHBPpb4rh1zwVGEJ0ObP2MfV+hmiqyBPJv4RKk1TtDGorTNJv7ifwP3Y0YpMqPPJam/VKvWjTq6YcoilsU6CNbeQS64GS2Sh8JSafjaQubRG99cjTyFUiaG3Q== Received: from BN9PR03CA0460.namprd03.prod.outlook.com (2603:10b6:408:139::15) by BYAPR12MB2679.namprd12.prod.outlook.com (2603:10b6:a03:72::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.20; Thu, 1 Jul 2021 13:27:21 +0000 Received: from BN8NAM11FT055.eop-nam11.prod.protection.outlook.com (2603:10b6:408:139:cafe::83) by BN9PR03CA0460.outlook.office365.com (2603:10b6:408:139::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.21 via Frontend Transport; Thu, 1 Jul 2021 13:27:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT055.mail.protection.outlook.com (10.13.177.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:21 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:27:11 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:26:08 +0300 Message-ID: <20210701132609.53727-15-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9e9c2047-d7e9-443e-4e15-08d93c93f4c2 X-MS-TrafficTypeDiagnostic: BYAPR12MB2679: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5236; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GhSXSdyTt7s7jFf98wkqL8XJJgt3ZITRHr2+a3x7HeNZe8scRaCnAFCvApSLGKewrprE5dRulYVy4DqvBiSLB+Kl99mvSg+JDfw6jgursqtXRjOuohU+BDATETMGv2iK3LC9mySzpFNVM5pDxXzsDzOeYlEkK1+AFd7r9JijNQQEXimAnS0RoeqT7i9v4PjxQ26fiI7g35u95O061ycGS8mgC8psYKAVXD76PKAIGbHhrAjsjWlKL3zVtM2p54qxSLY1ZhgnePUvE8uKyFXwaMkbpOWpiQiIXvX4zPHkkm3phAEVew0r+Si5s+sAb8bEfjvWmAbhZiZbEL+Da1Ua7ZEPlGopsPVnKdYZQ9bjDcPeJ0pt4p6GcoAkBUpXQM2B5yoX0o811cNlq9VdpW5odoCUHgOcLcm6/q70lT0UYJkFifZC/dmDl6wfSSMluiU4fuVNg1cop5kJPzYK4JuTJ0KY4SodPPuJHzCNg570e8XiZq4PqWY4e/5F01ztrK7+XfgLQ8JrfsbRyQWRl/86NrDTsg/Gcr6yEjP2RCddTMuwwMw2P41IViVUX+JEtWx+/iBt6qZzFtHk9TU9fe3NRCXctEhEKxdsNZzaLyI2ab31atxYq8YpSok3kIBY7vGGsYFtYChsafUcNAAh/rE3dbWW7TNEUUjQjeoGDheAS62GCNGRJod+rUY6kpsM3V3fsLtQ+MjQlC6qN8XxuHce/A== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(39850400004)(136003)(376002)(346002)(36840700001)(46966006)(316002)(336012)(36756003)(54906003)(4326008)(30864003)(1076003)(5660300002)(2616005)(6286002)(86362001)(426003)(6916009)(8676002)(82310400003)(478600001)(36860700001)(55016002)(47076005)(6666004)(70586007)(16526019)(356005)(7696005)(186003)(82740400003)(2906002)(7636003)(107886003)(8936002)(83380400001)(70206006)(26005)(559001)(579004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:21.0411 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9e9c2047-d7e9-443e-4e15-08d93c93f4c2 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT055.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2679 Subject: [dpdk-dev] [PATCH v5 14/15] test/crypto: add mlx5 crypto driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In order to test the new mlx5 crypto PMD, the driver is added to the crypto test application. Added mlx5 vectors that include 4 testing vectors with length 512 and 4096. Added mlx5 encryption function and decryption function that will both use the mlx5 vectors that will also set the dataunit_len and use a wrapped key. The added tests will test both data integrity and correct stat values. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- app/test/meson.build | 1 + app/test/test_cryptodev.c | 334 ++- app/test/test_cryptodev.h | 3 +- app/test/test_cryptodev_mlx5_test_vectors.h | 2502 +++++++++++++++++++ 4 files changed, 2823 insertions(+), 17 deletions(-) create mode 100644 app/test/test_cryptodev_mlx5_test_vectors.h diff --git a/app/test/meson.build b/app/test/meson.build index 0a5f425578..c36655888e 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -313,6 +313,7 @@ driver_test_names = [ 'cryptodev_aesni_gcm_autotest', 'cryptodev_dpaa_sec_autotest', 'cryptodev_dpaa2_sec_autotest', + 'cryptodev_mlx5_autotest', 'cryptodev_null_autotest', 'cryptodev_octeontx2_autotest', 'cryptodev_openssl_autotest', diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c index 39db52b17a..8dbe324b81 100644 --- a/app/test/test_cryptodev.c +++ b/app/test/test_cryptodev.c @@ -35,6 +35,7 @@ #include "test_cryptodev_hash_test_vectors.h" #include "test_cryptodev_kasumi_test_vectors.h" #include "test_cryptodev_kasumi_hash_test_vectors.h" +#include "test_cryptodev_mlx5_test_vectors.h" #include "test_cryptodev_snow3g_test_vectors.h" #include "test_cryptodev_snow3g_hash_test_vectors.h" #include "test_cryptodev_zuc_test_vectors.h" @@ -1267,6 +1268,39 @@ negative_hmac_sha1_testsuite_setup(void) return 0; } +static int +mlx5_testsuite_setup(void) +{ + struct crypto_testsuite_params *ts_params = &testsuite_params; + uint8_t dev_id = ts_params->valid_devs[0]; + struct rte_cryptodev_info dev_info; + const enum rte_crypto_cipher_algorithm ciphers[] = { + RTE_CRYPTO_CIPHER_AES_XTS + }; + + rte_cryptodev_info_get(dev_id, &dev_info); + + if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) && + !(dev_info.feature_flags & + RTE_CRYPTODEV_FF_CIPHER_WRAPPED_KEY) && + !(dev_info.feature_flags & + RTE_CRYPTODEV_FF_CIPHER_MULTIPLE_DATA_UNITS)) { + RTE_LOG(INFO, USER1, + "Feature flag requirements for MLX5 testsuite not met\n"); + + return TEST_SKIPPED; + } + + if (check_cipher_capabilities_supported(ciphers, + RTE_DIM(ciphers)) != 0) { + RTE_LOG(INFO, USER1, + "Capability requirements for MLX5 testsuite not met\n"); + return TEST_SKIPPED; + } + + return 0; +} + static int dev_configure_and_start(uint64_t ff_disable) { @@ -2366,7 +2400,7 @@ create_wireless_algo_cipher_session(uint8_t dev_id, enum rte_crypto_cipher_operation op, enum rte_crypto_cipher_algorithm algo, const uint8_t *key, const uint8_t key_len, - uint8_t iv_len) + uint8_t iv_len, uint64_t dataunit_len) { uint8_t cipher_key[key_len]; int status; @@ -2378,13 +2412,13 @@ create_wireless_algo_cipher_session(uint8_t dev_id, /* Setup Cipher Parameters */ ut_params->cipher_xform.type = RTE_CRYPTO_SYM_XFORM_CIPHER; ut_params->cipher_xform.next = NULL; - ut_params->cipher_xform.cipher.algo = algo; ut_params->cipher_xform.cipher.op = op; ut_params->cipher_xform.cipher.key.data = cipher_key; ut_params->cipher_xform.cipher.key.length = key_len; ut_params->cipher_xform.cipher.iv.offset = IV_OFFSET; ut_params->cipher_xform.cipher.iv.length = iv_len; + ut_params->cipher_xform.cipher.dataunit_len = dataunit_len; debug_hexdump(stdout, "key:", key, key_len); @@ -3528,7 +3562,7 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata) RTE_CRYPTO_CIPHER_OP_ENCRYPT, RTE_CRYPTO_CIPHER_KASUMI_F8, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -3631,7 +3665,7 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata) RTE_CRYPTO_CIPHER_OP_ENCRYPT, RTE_CRYPTO_CIPHER_KASUMI_F8, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -3718,7 +3752,7 @@ test_kasumi_encryption_oop(const struct kasumi_test_data *tdata) RTE_CRYPTO_CIPHER_OP_ENCRYPT, RTE_CRYPTO_CIPHER_KASUMI_F8, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -3815,7 +3849,7 @@ test_kasumi_encryption_oop_sgl(const struct kasumi_test_data *tdata) RTE_CRYPTO_CIPHER_OP_ENCRYPT, RTE_CRYPTO_CIPHER_KASUMI_F8, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -3896,7 +3930,7 @@ test_kasumi_decryption_oop(const struct kasumi_test_data *tdata) RTE_CRYPTO_CIPHER_OP_DECRYPT, RTE_CRYPTO_CIPHER_KASUMI_F8, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -3986,7 +4020,7 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata) RTE_CRYPTO_CIPHER_OP_DECRYPT, RTE_CRYPTO_CIPHER_KASUMI_F8, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -4078,7 +4112,7 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata) RTE_CRYPTO_CIPHER_OP_ENCRYPT, RTE_CRYPTO_CIPHER_SNOW3G_UEA2, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -4162,7 +4196,7 @@ test_snow3g_encryption_oop(const struct snow3g_test_data *tdata) RTE_CRYPTO_CIPHER_OP_ENCRYPT, RTE_CRYPTO_CIPHER_SNOW3G_UEA2, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -4262,7 +4296,7 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata) RTE_CRYPTO_CIPHER_OP_ENCRYPT, RTE_CRYPTO_CIPHER_SNOW3G_UEA2, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -4376,7 +4410,7 @@ test_snow3g_encryption_offset_oop(const struct snow3g_test_data *tdata) RTE_CRYPTO_CIPHER_OP_ENCRYPT, RTE_CRYPTO_CIPHER_SNOW3G_UEA2, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -4488,7 +4522,7 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata) RTE_CRYPTO_CIPHER_OP_DECRYPT, RTE_CRYPTO_CIPHER_SNOW3G_UEA2, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -4569,7 +4603,7 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata) RTE_CRYPTO_CIPHER_OP_DECRYPT, RTE_CRYPTO_CIPHER_SNOW3G_UEA2, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -5811,7 +5845,7 @@ test_zuc_encryption(const struct wireless_test_data *tdata) RTE_CRYPTO_CIPHER_OP_ENCRYPT, RTE_CRYPTO_CIPHER_ZUC_EEA3, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -5924,7 +5958,7 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata) RTE_CRYPTO_CIPHER_OP_ENCRYPT, RTE_CRYPTO_CIPHER_ZUC_EEA3, tdata->key.data, tdata->key.len, - tdata->cipher_iv.len); + tdata->cipher_iv.len, 0); if (retval < 0) return retval; @@ -6460,6 +6494,193 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata, return 0; } +static int +test_mlx5_encryption(const struct mlx5_test_data *tdata) +{ + struct crypto_testsuite_params *ts_params = &testsuite_params; + struct crypto_unittest_params *ut_params = &unittest_params; + struct rte_cryptodev_sym_capability_idx cap_idx; + struct rte_cryptodev_info dev_info; + struct rte_cryptodev_stats stats; + uint8_t *plaintext, *ciphertext; + uint64_t feat_flags; + unsigned int plaintext_pad_len; + unsigned int plaintext_len; + int retval; + + rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info); + feat_flags = dev_info.feature_flags; + if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) && + (!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) { + printf("Device doesn't support RAW data-path APIs.\n"); + return -ENOTSUP; + } + if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) + return -ENOTSUP; + /* Verify the capabilities */ + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_AES_XTS; + if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0], + &cap_idx) == NULL) + return -ENOTSUP; + /* Create mlx5 session */ + retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0], + RTE_CRYPTO_CIPHER_OP_ENCRYPT, + RTE_CRYPTO_CIPHER_AES_XTS, + tdata->key.data, tdata->key.len, + tdata->cipher_iv.len, 0); + if (retval < 0) + return retval; + ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + if (unlikely(ut_params->ibuf == NULL)) + return -ENOMEM; + /* Clear mbuf payload */ + memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0, + rte_pktmbuf_tailroom(ut_params->ibuf)); + plaintext_len = ceil_byte_length(tdata->plaintext.len); + /* Append data which is padded to a multiple */ + /* of the algorithms block size */ + plaintext_pad_len = RTE_ALIGN_CEIL(plaintext_len, 8); + plaintext = (uint8_t *)rte_pktmbuf_append(ut_params->ibuf, + plaintext_pad_len); + if (unlikely(plaintext == NULL)) + return -ENOMEM; + memcpy(plaintext, tdata->plaintext.data, plaintext_len); + debug_hexdump(stdout, "plaintext:", plaintext, plaintext_len); + retval = create_wireless_algo_cipher_operation(tdata->cipher_iv.data, + tdata->cipher_iv.len, (tdata->cipher.len_bits), + (tdata->cipher.offset_bits)); + if (retval < 0) + return retval; + if (global_api_test_type == CRYPTODEV_RAW_API_TEST) + process_sym_raw_dp_op(ts_params->valid_devs[0], 0, + ut_params->op, 1, 0, 1, tdata->cipher_iv.len); + else + ut_params->op = process_crypto_request(ts_params->valid_devs[0], + ut_params->op); + TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf"); + ut_params->obuf = ut_params->op->sym->m_dst; + if (ut_params->obuf) + ciphertext = rte_pktmbuf_mtod(ut_params->obuf, uint8_t *); + else + ciphertext = plaintext; + debug_hexdump(stdout, "ciphertext:", ciphertext, plaintext_len); + /* Validate obuf */ + TEST_ASSERT_BUFFERS_ARE_EQUAL_BIT( + ciphertext, + tdata->ciphertext.data, + tdata->validCipherLenInBits.len, + "MLX5 Ciphertext data not as expected"); + /* Validate stats */ + TEST_ASSERT_SUCCESS(rte_cryptodev_stats_get(ts_params->valid_devs[0], + &stats), "rte_cryptodev_stats_get failed"); + TEST_ASSERT((stats.enqueued_count == 1), + "rte_cryptodev_stats_get returned unexpected enqueued stat"); + TEST_ASSERT((stats.dequeued_count == 1), + "rte_cryptodev_stats_get returned unexpected dequeued stat"); + TEST_ASSERT((stats.enqueue_err_count == 0), + "rte_cryptodev_stats_get returned error enqueued stat"); + TEST_ASSERT((stats.dequeue_err_count == 0), + "rte_cryptodev_stats_get returned error dequeued stat"); + return 0; +} + +static int +test_mlx5_decryption(const struct mlx5_test_data *tdata) +{ + struct crypto_testsuite_params *ts_params = &testsuite_params; + struct crypto_unittest_params *ut_params = &unittest_params; + struct rte_cryptodev_sym_capability_idx cap_idx; + struct rte_cryptodev_info dev_info; + struct rte_cryptodev_stats stats; + const uint8_t *reference_plaintext; + uint8_t *ciphertext, *plaintext; + uint64_t feat_flags; + unsigned int ciphertext_pad_len; + unsigned int ciphertext_len; + int retval; + + + rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info); + feat_flags = dev_info.feature_flags; + if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) && + (!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) { + printf("Device doesn't support RAW data-path APIs.\n"); + return -ENOTSUP; + } + if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) + return -ENOTSUP; + /* Verify the capabilities */ + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_AES_XTS; + if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0], + &cap_idx) == NULL) + return -ENOTSUP; + /* Create mlx5 session */ + retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0], + RTE_CRYPTO_CIPHER_OP_DECRYPT, + RTE_CRYPTO_CIPHER_AES_XTS, + tdata->key.data, tdata->key.len, + tdata->cipher_iv.len, 0); + if (retval < 0) + return retval; + ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + if (unlikely(ut_params->ibuf == NULL)) + return -ENOMEM; + /* Clear mbuf payload */ + memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0, + rte_pktmbuf_tailroom(ut_params->ibuf)); + ciphertext_len = ceil_byte_length(tdata->ciphertext.len); + /* Append data which is padded to a multiple */ + /* of the algorithms block size */ + ciphertext_pad_len = RTE_ALIGN_CEIL(ciphertext_len, 8); + ciphertext = (uint8_t *)rte_pktmbuf_append(ut_params->ibuf, + ciphertext_pad_len); + if (unlikely(ciphertext == NULL)) + return -ENOMEM; + memcpy(ciphertext, tdata->ciphertext.data, ciphertext_len); + debug_hexdump(stdout, "ciphertext:", ciphertext, ciphertext_len); + /* Create mlx5 operation */ + retval = create_wireless_algo_cipher_operation(tdata->cipher_iv.data, + tdata->cipher_iv.len, (tdata->cipher.len_bits), + (tdata->cipher.offset_bits)); + if (retval < 0) + return retval; + if (global_api_test_type == CRYPTODEV_RAW_API_TEST) + process_sym_raw_dp_op(ts_params->valid_devs[0], 0, + ut_params->op, 1, 0, 1, 0); + else + ut_params->op = process_crypto_request(ts_params->valid_devs[0], + ut_params->op); + TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf"); + ut_params->obuf = ut_params->op->sym->m_dst; + if (ut_params->obuf) + plaintext = rte_pktmbuf_mtod(ut_params->obuf, uint8_t *); + else + plaintext = ciphertext + (tdata->cipher.offset_bits); + debug_hexdump(stdout, "plaintext:", plaintext, ciphertext_len); + reference_plaintext = tdata->plaintext.data + + (tdata->cipher.offset_bits); + /* Validate obuf */ + TEST_ASSERT_BUFFERS_ARE_EQUAL_BIT( + plaintext, + reference_plaintext, + tdata->validCipherLenInBits.len, + "MLX5 Plaintext data not as expected"); + /* Validate stats */ + TEST_ASSERT_SUCCESS(rte_cryptodev_stats_get(ts_params->valid_devs[0], + &stats), "rte_cryptodev_stats_get failed"); + TEST_ASSERT((stats.enqueued_count == 1), + "rte_cryptodev_stats_get returned unexpected enqueued stat"); + TEST_ASSERT((stats.dequeued_count == 1), + "rte_cryptodev_stats_get returned unexpected dequeued stat"); + TEST_ASSERT((stats.enqueue_err_count == 0), + "rte_cryptodev_stats_get returned error enqueued stat"); + TEST_ASSERT((stats.dequeue_err_count == 0), + "rte_cryptodev_stats_get returned error dequeued stat"); + return 0; +} + static int test_kasumi_encryption_test_case_1(void) { @@ -7076,6 +7297,54 @@ test_zuc_auth_cipher_verify_test_case_1_oop_sgl(void) &zuc_auth_cipher_test_case_1, OUT_OF_PLACE, 1); } +static int +test_mlx5_encryption_test_case_1(void) +{ + return test_mlx5_encryption(&mlx5_test_case_cipher_aes_xts_1); +} + +static int +test_mlx5_encryption_test_case_2(void) +{ + return test_mlx5_encryption(&mlx5_test_case_cipher_aes_xts_2); +} + +static int +test_mlx5_encryption_test_case_3(void) +{ + return test_mlx5_encryption(&mlx5_test_case_cipher_aes_xts_3); +} + +static int +test_mlx5_encryption_test_case_4(void) +{ + return test_mlx5_encryption(&mlx5_test_case_cipher_aes_xts_4); +} + +static int +test_mlx5_decryption_test_case_1(void) +{ + return test_mlx5_decryption(&mlx5_test_case_cipher_aes_xts_1); +} + +static int +test_mlx5_decryption_test_case_2(void) +{ + return test_mlx5_decryption(&mlx5_test_case_cipher_aes_xts_2); +} + +static int +test_mlx5_decryption_test_case_3(void) +{ + return test_mlx5_decryption(&mlx5_test_case_cipher_aes_xts_3); +} + +static int +test_mlx5_decryption_test_case_4(void) +{ + return test_mlx5_decryption(&mlx5_test_case_cipher_aes_xts_4); +} + static int test_mixed_check_if_unsupported(const struct mixed_cipher_auth_test_data *tdata) { @@ -14458,6 +14727,31 @@ static struct unit_test_suite cryptodev_mixed_cipher_hash_testsuite = { } }; +static struct unit_test_suite cryptodev_mlx5_testsuite = { + .suite_name = "MLX5 Test Suite", + .setup = mlx5_testsuite_setup, + .unit_test_cases = { + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_decryption_test_case_1), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_decryption_test_case_2), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_decryption_test_case_3), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_decryption_test_case_4), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_encryption_test_case_1), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_encryption_test_case_2), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_encryption_test_case_3), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_encryption_test_case_4), + TEST_CASES_END() + } +}; + + static int run_cryptodev_testsuite(const char *pmd_name) { @@ -14483,6 +14777,7 @@ run_cryptodev_testsuite(const char *pmd_name) &cryptodev_hmac_md5_auth_testsuite, &cryptodev_kasumi_testsuite, &cryptodev_esn_testsuite, + &cryptodev_mlx5_testsuite, &cryptodev_negative_aes_gcm_testsuite, &cryptodev_negative_aes_gmac_testsuite, &cryptodev_mixed_cipher_hash_testsuite, @@ -14590,6 +14885,12 @@ test_cryptodev_sw_kasumi(void /*argv __rte_unused, int argc __rte_unused*/) return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_KASUMI_PMD)); } +static int +test_cryptodev_sw_mlx5(void /*argv __rte_unused, int argc __rte_unused*/) +{ + return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_MLX5_PMD)); +} + static int test_cryptodev_sw_zuc(void /*argv __rte_unused, int argc __rte_unused*/) { @@ -14791,6 +15092,7 @@ REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_gcm_autotest, REGISTER_TEST_COMMAND(cryptodev_null_autotest, test_cryptodev_null); REGISTER_TEST_COMMAND(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g); REGISTER_TEST_COMMAND(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi); +REGISTER_TEST_COMMAND(cryptodev_sw_mlx5_autotest, test_cryptodev_sw_mlx5); REGISTER_TEST_COMMAND(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc); REGISTER_TEST_COMMAND(cryptodev_sw_armv8_autotest, test_cryptodev_armv8); REGISTER_TEST_COMMAND(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl); diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h index f81f8e372f..683fe3ddc8 100644 --- a/app/test/test_cryptodev.h +++ b/app/test/test_cryptodev.h @@ -19,7 +19,7 @@ #define DEFAULT_NUM_XFORMS (2) #define NUM_MBUFS (8191) #define MBUF_CACHE_SIZE (256) -#define MBUF_DATAPAYLOAD_SIZE (2048 + DIGEST_BYTE_LENGTH_SHA512) +#define MBUF_DATAPAYLOAD_SIZE (4096 + DIGEST_BYTE_LENGTH_SHA512) #define MBUF_SIZE (sizeof(struct rte_mbuf) + \ RTE_PKTMBUF_HEADROOM + MBUF_DATAPAYLOAD_SIZE) @@ -71,6 +71,7 @@ #define CRYPTODEV_NAME_CAAM_JR_PMD crypto_caam_jr #define CRYPTODEV_NAME_NITROX_PMD crypto_nitrox_sym #define CRYPTODEV_NAME_BCMFS_PMD crypto_bcmfs +#define CRYPTODEV_NAME_MLX5_PMD mlx5_crypto enum cryptodev_api_test_type { CRYPTODEV_API_TEST = 0, diff --git a/app/test/test_cryptodev_mlx5_test_vectors.h b/app/test/test_cryptodev_mlx5_test_vectors.h new file mode 100644 index 0000000000..2a05aa4626 --- /dev/null +++ b/app/test/test_cryptodev_mlx5_test_vectors.h @@ -0,0 +1,2502 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ +#ifndef TEST_CRYPTODEV_MLX5_TEST_VECTORS_H_ +#define TEST_CRYPTODEV_MLX5_TEST_VECTORS_H_ + +struct mlx5_test_data { + struct { + uint8_t data[64]; + unsigned int len; + } key; + + struct { + uint8_t data[64] __rte_aligned(16); + unsigned int len; + } cipher_iv; + + struct { + uint8_t data[32768]; + unsigned int len; /* length must be in Bits */ + } plaintext; + + struct { + uint8_t data[32768]; + unsigned int len; /* length must be in Bits */ + } ciphertext; + + struct { + unsigned int len; + } validCipherLenInBits; + + struct { + unsigned int len_bits; /* length must be in Bits */ + unsigned int offset_bits; + } cipher; + uint16_t dataunit_len; +}; +static struct mlx5_test_data mlx5_test_case_cipher_aes_xts_1 = { + .key = { + .data = { + 0x94, 0xD8, 0xAE, 0xD7, 0x74, 0xBF, 0x51, 0xF1, + 0x2D, 0x19, 0x0B, 0x9A, 0xB3, 0xDC, 0x1E, 0xA9, + 0x5C, 0xC5, 0x47, 0xC1, 0xBB, 0x94, 0xDB, 0xFA, + 0x2B, 0x1E, 0x54, 0xA5, 0x76, 0xAD, 0x07, 0x4B, + 0x46, 0xE0, 0xAA, 0xF9, 0xF7, 0x5C, 0x64, 0x2D, + 0x8F, 0x04, 0x17, 0x72, 0xE5, 0x8D, 0xFE, 0x9C + }, + .len = 48 + }, + .cipher_iv = { + .data = { + 0x4F, 0xAE, 0xF7, 0x11, 0x7C, 0xDA, 0x59, 0xC6, + 0x6E, 0x4B, 0x92, 0x01, 0x3E, 0x76, 0x8A, 0xD5 + }, + .len = 16 + }, + .plaintext = { + .data = { + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79 + }, + .len = 512 << 3 + }, + .ciphertext = { + .data = { + 0xD3, 0x94, 0x3E, 0xC9, 0xD3, 0x43, 0x13, 0xD3, + 0x07, 0x7E, 0x51, 0x11, 0x97, 0xA5, 0xB1, 0xB2, + 0xB0, 0x55, 0xF8, 0xD6, 0xA2, 0x10, 0x78, 0x76, + 0x37, 0x95, 0x4F, 0x25, 0x99, 0xCC, 0x12, 0xD6, + 0xBB, 0x2A, 0x24, 0xF4, 0x2D, 0x82, 0xFF, 0x56, + 0xEC, 0x55, 0x3B, 0xDD, 0xF9, 0xE1, 0xC8, 0x48, + 0x68, 0x16, 0xD2, 0x2B, 0x95, 0x7C, 0x1F, 0xCB, + 0x32, 0xD6, 0x8F, 0x9A, 0x2E, 0xF7, 0x9B, 0xBE, + 0x72, 0xFC, 0x55, 0x33, 0x27, 0x21, 0x2F, 0x69, + 0xCA, 0x76, 0xA5, 0x9B, 0x21, 0x2E, 0x40, 0x57, + 0x2C, 0x1C, 0x98, 0x41, 0x9A, 0x5E, 0x55, 0x38, + 0xDE, 0xC2, 0x09, 0x57, 0x32, 0xA3, 0x34, 0x40, + 0xC1, 0x8D, 0xCF, 0x70, 0x15, 0xF2, 0x6F, 0x4F, + 0x6A, 0x04, 0xA6, 0x6D, 0xFF, 0x53, 0x25, 0x6A, + 0x0E, 0xD0, 0x87, 0x63, 0xA1, 0x6C, 0xB1, 0x99, + 0x4A, 0x42, 0xF5, 0xF6, 0xEA, 0xA4, 0xEB, 0x6D, + 0x70, 0x9B, 0x0F, 0x85, 0xE2, 0x43, 0x6C, 0x27, + 0x54, 0x57, 0x52, 0x1C, 0xCF, 0x72, 0x11, 0x83, + 0xC3, 0xF5, 0xC6, 0xB6, 0x07, 0xEC, 0x1A, 0xF5, + 0xAC, 0xA1, 0xF2, 0x3A, 0x01, 0x53, 0x0C, 0xA4, + 0x40, 0x19, 0xAF, 0x7B, 0x2D, 0xE7, 0x56, 0x8B, + 0x71, 0x5F, 0x8D, 0x96, 0xF1, 0x5D, 0x33, 0xAC, + 0xE7, 0xAD, 0x96, 0xDB, 0xBB, 0xF7, 0xF5, 0x3F, + 0x0E, 0x02, 0x2E, 0x80, 0xE9, 0xA2, 0x30, 0xD2, + 0x82, 0x65, 0xEC, 0x1A, 0xB2, 0xE8, 0x98, 0x23, + 0x42, 0x79, 0x43, 0x85, 0x3B, 0xF9, 0xFB, 0xBB, + 0xC5, 0x29, 0xAD, 0x95, 0xAE, 0x22, 0x5B, 0x26, + 0xDF, 0x76, 0x65, 0x37, 0x17, 0xBC, 0x58, 0xBB, + 0x1B, 0x0A, 0x71, 0xDC, 0x68, 0xFF, 0x90, 0x36, + 0x37, 0xB7, 0x49, 0x5E, 0x49, 0x4E, 0xE4, 0x1E, + 0x24, 0x39, 0x48, 0xC7, 0x68, 0x19, 0xED, 0x94, + 0xB5, 0xF6, 0x85, 0xFA, 0xE6, 0xB8, 0x2B, 0x9E, + 0x07, 0x9E, 0xFB, 0x1D, 0x61, 0x06, 0x47, 0x2A, + 0x3E, 0x1E, 0xD8, 0x52, 0xFB, 0xB7, 0xE3, 0xB4, + 0x0D, 0xA8, 0x15, 0x1E, 0x98, 0x02, 0xBD, 0x89, + 0x89, 0xE3, 0x38, 0x2C, 0xAB, 0x50, 0x25, 0x30, + 0xB4, 0x5E, 0xA5, 0xCD, 0xA8, 0x9B, 0xA4, 0x2A, + 0xED, 0x19, 0x3B, 0xC3, 0x05, 0x07, 0x57, 0xB5, + 0x52, 0x11, 0x74, 0x95, 0x51, 0x5A, 0xD8, 0xED, + 0xDF, 0x37, 0x91, 0x83, 0x27, 0xD5, 0x08, 0x82, + 0xB3, 0x42, 0x08, 0xC7, 0x81, 0x35, 0x5F, 0x58, + 0x28, 0x69, 0x0D, 0x97, 0x1D, 0x28, 0xE6, 0xB0, + 0x58, 0x93, 0xCE, 0x2A, 0xB4, 0x7D, 0x4B, 0x83, + 0x20, 0x1E, 0x08, 0xEF, 0x68, 0x51, 0xEB, 0xB4, + 0xFA, 0x78, 0xB5, 0xE5, 0x2D, 0x93, 0x07, 0x99, + 0xBB, 0xDD, 0x9A, 0x4E, 0xD6, 0xE7, 0x31, 0x9F, + 0x4D, 0xB4, 0x05, 0x45, 0x89, 0x59, 0x42, 0xFF, + 0x0C, 0xD4, 0xC2, 0xF3, 0xA0, 0xF7, 0xD3, 0xCE, + 0x26, 0x2F, 0x9B, 0xCB, 0x98, 0x24, 0xDE, 0x50, + 0xF6, 0x81, 0x24, 0xA3, 0xB4, 0x99, 0x94, 0xDB, + 0xF5, 0xF4, 0x74, 0xAC, 0xED, 0xF1, 0xD6, 0x2F, + 0x89, 0x48, 0x41, 0x24, 0xBB, 0xA7, 0x64, 0x48, + 0xAE, 0x36, 0xE3, 0xF3, 0xA8, 0x37, 0x4F, 0xE6, + 0x24, 0x92, 0xC4, 0x27, 0x86, 0x68, 0xC4, 0xA2, + 0x2F, 0xB2, 0xA8, 0xFF, 0x27, 0x5F, 0x59, 0xB9, + 0x86, 0x89, 0x5A, 0x09, 0xB8, 0xDA, 0xD6, 0xBA, + 0x26, 0x86, 0x51, 0xC2, 0xDF, 0xCF, 0x07, 0x49, + 0x1C, 0xB7, 0x1A, 0x80, 0xD4, 0x97, 0x13, 0xAA, + 0xE8, 0x38, 0xA7, 0x2C, 0xB7, 0xE2, 0xE0, 0x33, + 0xF9, 0xC3, 0x21, 0xF6, 0x2A, 0xAA, 0x1A, 0xF7, + 0x39, 0x1F, 0x2F, 0x7E, 0x38, 0xC8, 0x82, 0x3F, + 0xCC, 0xD8, 0xE2, 0xD9, 0x02, 0xE6, 0x8E, 0x31, + 0xA7, 0x37, 0xE8, 0x59, 0x3F, 0xE1, 0x60, 0x1C, + 0x74, 0x52, 0xE4, 0x49, 0xF7, 0x21, 0x60, 0x11 + }, + .len = 512 << 3 + }, + .validCipherLenInBits = { + .len = 512 << 3 + }, + .cipher = { + .len_bits = 512, + .offset_bits = 0 + }, + .dataunit_len = 512 +}; + +static struct mlx5_test_data mlx5_test_case_cipher_aes_xts_2 = { + .key = { + .data = { + 0x94, 0xD8, 0xAE, 0xD7, 0x74, 0xBF, 0x51, 0xF1, + 0x2D, 0x19, 0x0B, 0x9A, 0xB3, 0xDC, 0x1E, 0xA9, + 0x5C, 0xC5, 0x47, 0xC1, 0xBB, 0x94, 0xDB, 0xFA, + 0x2B, 0x1E, 0x54, 0xA5, 0x76, 0xAD, 0x07, 0x4B, + 0x46, 0xE0, 0xAA, 0xF9, 0xF7, 0x5C, 0x64, 0x2D, + 0x8F, 0x04, 0x17, 0x72, 0xE5, 0x8D, 0xFE, 0x9C + }, + .len = 48 + }, + .cipher_iv = { + .data = { + 0x4F, 0xAE, 0xF7, 0x11, 0x7C, 0xDA, 0x59, 0xC6, + 0x6E, 0x4B, 0x92, 0x01, 0x3E, 0x76, 0x8A, 0xD5 + }, + .len = 16 + }, + .plaintext = { + .data = { + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79 + }, + .len = 512 << 3 + }, + .ciphertext = { + .data = { + 0xD3, 0x94, 0x3E, 0xC9, 0xD3, 0x43, 0x13, 0xD3, + 0x07, 0x7E, 0x51, 0x11, 0x97, 0xA5, 0xB1, 0xB2, + 0xB0, 0x55, 0xF8, 0xD6, 0xA2, 0x10, 0x78, 0x76, + 0x37, 0x95, 0x4F, 0x25, 0x99, 0xCC, 0x12, 0xD6, + 0xBB, 0x2A, 0x24, 0xF4, 0x2D, 0x82, 0xFF, 0x56, + 0xEC, 0x55, 0x3B, 0xDD, 0xF9, 0xE1, 0xC8, 0x48, + 0x68, 0x16, 0xD2, 0x2B, 0x95, 0x7C, 0x1F, 0xCB, + 0x32, 0xD6, 0x8F, 0x9A, 0x2E, 0xF7, 0x9B, 0xBE, + 0x72, 0xFC, 0x55, 0x33, 0x27, 0x21, 0x2F, 0x69, + 0xCA, 0x76, 0xA5, 0x9B, 0x21, 0x2E, 0x40, 0x57, + 0x2C, 0x1C, 0x98, 0x41, 0x9A, 0x5E, 0x55, 0x38, + 0xDE, 0xC2, 0x09, 0x57, 0x32, 0xA3, 0x34, 0x40, + 0xC1, 0x8D, 0xCF, 0x70, 0x15, 0xF2, 0x6F, 0x4F, + 0x6A, 0x04, 0xA6, 0x6D, 0xFF, 0x53, 0x25, 0x6A, + 0x0E, 0xD0, 0x87, 0x63, 0xA1, 0x6C, 0xB1, 0x99, + 0x4A, 0x42, 0xF5, 0xF6, 0xEA, 0xA4, 0xEB, 0x6D, + 0x70, 0x9B, 0x0F, 0x85, 0xE2, 0x43, 0x6C, 0x27, + 0x54, 0x57, 0x52, 0x1C, 0xCF, 0x72, 0x11, 0x83, + 0xC3, 0xF5, 0xC6, 0xB6, 0x07, 0xEC, 0x1A, 0xF5, + 0xAC, 0xA1, 0xF2, 0x3A, 0x01, 0x53, 0x0C, 0xA4, + 0x40, 0x19, 0xAF, 0x7B, 0x2D, 0xE7, 0x56, 0x8B, + 0x71, 0x5F, 0x8D, 0x96, 0xF1, 0x5D, 0x33, 0xAC, + 0xE7, 0xAD, 0x96, 0xDB, 0xBB, 0xF7, 0xF5, 0x3F, + 0x0E, 0x02, 0x2E, 0x80, 0xE9, 0xA2, 0x30, 0xD2, + 0x82, 0x65, 0xEC, 0x1A, 0xB2, 0xE8, 0x98, 0x23, + 0x42, 0x79, 0x43, 0x85, 0x3B, 0xF9, 0xFB, 0xBB, + 0xC5, 0x29, 0xAD, 0x95, 0xAE, 0x22, 0x5B, 0x26, + 0xDF, 0x76, 0x65, 0x37, 0x17, 0xBC, 0x58, 0xBB, + 0x1B, 0x0A, 0x71, 0xDC, 0x68, 0xFF, 0x90, 0x36, + 0x37, 0xB7, 0x49, 0x5E, 0x49, 0x4E, 0xE4, 0x1E, + 0x24, 0x39, 0x48, 0xC7, 0x68, 0x19, 0xED, 0x94, + 0xB5, 0xF6, 0x85, 0xFA, 0xE6, 0xB8, 0x2B, 0x9E, + 0x07, 0x9E, 0xFB, 0x1D, 0x61, 0x06, 0x47, 0x2A, + 0x3E, 0x1E, 0xD8, 0x52, 0xFB, 0xB7, 0xE3, 0xB4, + 0x0D, 0xA8, 0x15, 0x1E, 0x98, 0x02, 0xBD, 0x89, + 0x89, 0xE3, 0x38, 0x2C, 0xAB, 0x50, 0x25, 0x30, + 0xB4, 0x5E, 0xA5, 0xCD, 0xA8, 0x9B, 0xA4, 0x2A, + 0xED, 0x19, 0x3B, 0xC3, 0x05, 0x07, 0x57, 0xB5, + 0x52, 0x11, 0x74, 0x95, 0x51, 0x5A, 0xD8, 0xED, + 0xDF, 0x37, 0x91, 0x83, 0x27, 0xD5, 0x08, 0x82, + 0xB3, 0x42, 0x08, 0xC7, 0x81, 0x35, 0x5F, 0x58, + 0x28, 0x69, 0x0D, 0x97, 0x1D, 0x28, 0xE6, 0xB0, + 0x58, 0x93, 0xCE, 0x2A, 0xB4, 0x7D, 0x4B, 0x83, + 0x20, 0x1E, 0x08, 0xEF, 0x68, 0x51, 0xEB, 0xB4, + 0xFA, 0x78, 0xB5, 0xE5, 0x2D, 0x93, 0x07, 0x99, + 0xBB, 0xDD, 0x9A, 0x4E, 0xD6, 0xE7, 0x31, 0x9F, + 0x4D, 0xB4, 0x05, 0x45, 0x89, 0x59, 0x42, 0xFF, + 0x0C, 0xD4, 0xC2, 0xF3, 0xA0, 0xF7, 0xD3, 0xCE, + 0x26, 0x2F, 0x9B, 0xCB, 0x98, 0x24, 0xDE, 0x50, + 0xF6, 0x81, 0x24, 0xA3, 0xB4, 0x99, 0x94, 0xDB, + 0xF5, 0xF4, 0x74, 0xAC, 0xED, 0xF1, 0xD6, 0x2F, + 0x89, 0x48, 0x41, 0x24, 0xBB, 0xA7, 0x64, 0x48, + 0xAE, 0x36, 0xE3, 0xF3, 0xA8, 0x37, 0x4F, 0xE6, + 0x24, 0x92, 0xC4, 0x27, 0x86, 0x68, 0xC4, 0xA2, + 0x2F, 0xB2, 0xA8, 0xFF, 0x27, 0x5F, 0x59, 0xB9, + 0x86, 0x89, 0x5A, 0x09, 0xB8, 0xDA, 0xD6, 0xBA, + 0x26, 0x86, 0x51, 0xC2, 0xDF, 0xCF, 0x07, 0x49, + 0x1C, 0xB7, 0x1A, 0x80, 0xD4, 0x97, 0x13, 0xAA, + 0xE8, 0x38, 0xA7, 0x2C, 0xB7, 0xE2, 0xE0, 0x33, + 0xF9, 0xC3, 0x21, 0xF6, 0x2A, 0xAA, 0x1A, 0xF7, + 0x39, 0x1F, 0x2F, 0x7E, 0x38, 0xC8, 0x82, 0x3F, + 0xCC, 0xD8, 0xE2, 0xD9, 0x02, 0xE6, 0x8E, 0x31, + 0xA7, 0x37, 0xE8, 0x59, 0x3F, 0xE1, 0x60, 0x1C, + 0x74, 0x52, 0xE4, 0x49, 0xF7, 0x21, 0x60, 0x11 + }, + .len = 512 << 3 + }, + .validCipherLenInBits = { + .len = 512 << 3 + }, + .cipher = { + .len_bits = 512, + .offset_bits = 0 + }, + .dataunit_len = 0 +}; + +static struct mlx5_test_data mlx5_test_case_cipher_aes_xts_3 = { + .key = { + .data = { + 0x94, 0xD8, 0xAE, 0xD7, 0x74, 0xBF, 0x51, 0xF1, + 0x2D, 0x19, 0x0B, 0x9A, 0xB3, 0xDC, 0x1E, 0xA9, + 0x5C, 0xC5, 0x47, 0xC1, 0xBB, 0x94, 0xDB, 0xFA, + 0x2B, 0x1E, 0x54, 0xA5, 0x76, 0xAD, 0x07, 0x4B, + 0x46, 0xE0, 0xAA, 0xF9, 0xF7, 0x5C, 0x64, 0x2D, + 0x8F, 0x04, 0x17, 0x72, 0xE5, 0x8D, 0xFE, 0x9C + }, + .len = 48 + }, + .cipher_iv = { + .data = { + 0x4F, 0xAE, 0xF7, 0x11, 0x7C, 0xDA, 0x59, 0xC6, + 0x6E, 0x4B, 0x92, 0x01, 0x3E, 0x76, 0x8A, 0xD5 + }, + .len = 16 + }, + .plaintext = { + .data = { + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79 + }, + .len = 4096 << 3 + }, + .ciphertext = { + .data = { + 0xD3, 0x94, 0x3E, 0xC9, 0xD3, 0x43, 0x13, 0xD3, + 0x07, 0x7E, 0x51, 0x11, 0x97, 0xA5, 0xB1, 0xB2, + 0xB0, 0x55, 0xF8, 0xD6, 0xA2, 0x10, 0x78, 0x76, + 0x37, 0x95, 0x4F, 0x25, 0x99, 0xCC, 0x12, 0xD6, + 0xBB, 0x2A, 0x24, 0xF4, 0x2D, 0x82, 0xFF, 0x56, + 0xEC, 0x55, 0x3B, 0xDD, 0xF9, 0xE1, 0xC8, 0x48, + 0x68, 0x16, 0xD2, 0x2B, 0x95, 0x7C, 0x1F, 0xCB, + 0x32, 0xD6, 0x8F, 0x9A, 0x2E, 0xF7, 0x9B, 0xBE, + 0x72, 0xFC, 0x55, 0x33, 0x27, 0x21, 0x2F, 0x69, + 0xCA, 0x76, 0xA5, 0x9B, 0x21, 0x2E, 0x40, 0x57, + 0x2C, 0x1C, 0x98, 0x41, 0x9A, 0x5E, 0x55, 0x38, + 0xDE, 0xC2, 0x09, 0x57, 0x32, 0xA3, 0x34, 0x40, + 0xC1, 0x8D, 0xCF, 0x70, 0x15, 0xF2, 0x6F, 0x4F, + 0x6A, 0x04, 0xA6, 0x6D, 0xFF, 0x53, 0x25, 0x6A, + 0x0E, 0xD0, 0x87, 0x63, 0xA1, 0x6C, 0xB1, 0x99, + 0x4A, 0x42, 0xF5, 0xF6, 0xEA, 0xA4, 0xEB, 0x6D, + 0x70, 0x9B, 0x0F, 0x85, 0xE2, 0x43, 0x6C, 0x27, + 0x54, 0x57, 0x52, 0x1C, 0xCF, 0x72, 0x11, 0x83, + 0xC3, 0xF5, 0xC6, 0xB6, 0x07, 0xEC, 0x1A, 0xF5, + 0xAC, 0xA1, 0xF2, 0x3A, 0x01, 0x53, 0x0C, 0xA4, + 0x40, 0x19, 0xAF, 0x7B, 0x2D, 0xE7, 0x56, 0x8B, + 0x71, 0x5F, 0x8D, 0x96, 0xF1, 0x5D, 0x33, 0xAC, + 0xE7, 0xAD, 0x96, 0xDB, 0xBB, 0xF7, 0xF5, 0x3F, + 0x0E, 0x02, 0x2E, 0x80, 0xE9, 0xA2, 0x30, 0xD2, + 0x82, 0x65, 0xEC, 0x1A, 0xB2, 0xE8, 0x98, 0x23, + 0x42, 0x79, 0x43, 0x85, 0x3B, 0xF9, 0xFB, 0xBB, + 0xC5, 0x29, 0xAD, 0x95, 0xAE, 0x22, 0x5B, 0x26, + 0xDF, 0x76, 0x65, 0x37, 0x17, 0xBC, 0x58, 0xBB, + 0x1B, 0x0A, 0x71, 0xDC, 0x68, 0xFF, 0x90, 0x36, + 0x37, 0xB7, 0x49, 0x5E, 0x49, 0x4E, 0xE4, 0x1E, + 0x24, 0x39, 0x48, 0xC7, 0x68, 0x19, 0xED, 0x94, + 0xB5, 0xF6, 0x85, 0xFA, 0xE6, 0xB8, 0x2B, 0x9E, + 0x07, 0x9E, 0xFB, 0x1D, 0x61, 0x06, 0x47, 0x2A, + 0x3E, 0x1E, 0xD8, 0x52, 0xFB, 0xB7, 0xE3, 0xB4, + 0x0D, 0xA8, 0x15, 0x1E, 0x98, 0x02, 0xBD, 0x89, + 0x89, 0xE3, 0x38, 0x2C, 0xAB, 0x50, 0x25, 0x30, + 0xB4, 0x5E, 0xA5, 0xCD, 0xA8, 0x9B, 0xA4, 0x2A, + 0xED, 0x19, 0x3B, 0xC3, 0x05, 0x07, 0x57, 0xB5, + 0x52, 0x11, 0x74, 0x95, 0x51, 0x5A, 0xD8, 0xED, + 0xDF, 0x37, 0x91, 0x83, 0x27, 0xD5, 0x08, 0x82, + 0xB3, 0x42, 0x08, 0xC7, 0x81, 0x35, 0x5F, 0x58, + 0x28, 0x69, 0x0D, 0x97, 0x1D, 0x28, 0xE6, 0xB0, + 0x58, 0x93, 0xCE, 0x2A, 0xB4, 0x7D, 0x4B, 0x83, + 0x20, 0x1E, 0x08, 0xEF, 0x68, 0x51, 0xEB, 0xB4, + 0xFA, 0x78, 0xB5, 0xE5, 0x2D, 0x93, 0x07, 0x99, + 0xBB, 0xDD, 0x9A, 0x4E, 0xD6, 0xE7, 0x31, 0x9F, + 0x4D, 0xB4, 0x05, 0x45, 0x89, 0x59, 0x42, 0xFF, + 0x0C, 0xD4, 0xC2, 0xF3, 0xA0, 0xF7, 0xD3, 0xCE, + 0x26, 0x2F, 0x9B, 0xCB, 0x98, 0x24, 0xDE, 0x50, + 0xF6, 0x81, 0x24, 0xA3, 0xB4, 0x99, 0x94, 0xDB, + 0xF5, 0xF4, 0x74, 0xAC, 0xED, 0xF1, 0xD6, 0x2F, + 0x89, 0x48, 0x41, 0x24, 0xBB, 0xA7, 0x64, 0x48, + 0xAE, 0x36, 0xE3, 0xF3, 0xA8, 0x37, 0x4F, 0xE6, + 0x24, 0x92, 0xC4, 0x27, 0x86, 0x68, 0xC4, 0xA2, + 0x2F, 0xB2, 0xA8, 0xFF, 0x27, 0x5F, 0x59, 0xB9, + 0x86, 0x89, 0x5A, 0x09, 0xB8, 0xDA, 0xD6, 0xBA, + 0x26, 0x86, 0x51, 0xC2, 0xDF, 0xCF, 0x07, 0x49, + 0x1C, 0xB7, 0x1A, 0x80, 0xD4, 0x97, 0x13, 0xAA, + 0xE8, 0x38, 0xA7, 0x2C, 0xB7, 0xE2, 0xE0, 0x33, + 0xF9, 0xC3, 0x21, 0xF6, 0x2A, 0xAA, 0x1A, 0xF7, + 0x39, 0x1F, 0x2F, 0x7E, 0x38, 0xC8, 0x82, 0x3F, + 0xCC, 0xD8, 0xE2, 0xD9, 0x02, 0xE6, 0x8E, 0x31, + 0xA7, 0x37, 0xE8, 0x59, 0x3F, 0xE1, 0x60, 0x1C, + 0x74, 0x52, 0xE4, 0x49, 0xF7, 0x21, 0x60, 0x11, + 0x8F, 0xD0, 0xA1, 0x4B, 0xE3, 0xF1, 0x18, 0x3D, + 0x90, 0x89, 0x54, 0x27, 0xA0, 0xF9, 0x32, 0x09, + 0x3D, 0x9D, 0x9A, 0x09, 0x53, 0xC6, 0x7E, 0x95, + 0x85, 0x53, 0x98, 0x4C, 0x23, 0xEA, 0x54, 0xBD, + 0x6F, 0x50, 0xBC, 0x4C, 0xCF, 0x37, 0xC5, 0x7B, + 0x4B, 0xCE, 0x84, 0xAF, 0xE2, 0xE2, 0x55, 0x49, + 0xBC, 0xBF, 0x92, 0xCA, 0x1E, 0x5E, 0x10, 0xDF, + 0x60, 0x87, 0x09, 0xA1, 0x4C, 0x1D, 0x7E, 0x1E, + 0x59, 0xE9, 0xCF, 0xDA, 0x45, 0x3F, 0xE2, 0x0F, + 0x53, 0x8D, 0x8B, 0x79, 0xBD, 0xD8, 0xB0, 0xE3, + 0x5B, 0x7C, 0x55, 0x4A, 0x84, 0xF0, 0x1E, 0xF9, + 0xE8, 0xF3, 0x09, 0x4D, 0x0B, 0xD7, 0x77, 0xCC, + 0x3F, 0x70, 0x22, 0x7D, 0x17, 0x27, 0x48, 0x57, + 0xE2, 0x36, 0xA0, 0x84, 0x3B, 0xDE, 0x05, 0x34, + 0xEF, 0x55, 0x12, 0xF4, 0x9A, 0x99, 0x0D, 0x28, + 0x86, 0x28, 0x99, 0x6B, 0x22, 0xEE, 0x63, 0xF0, + 0x68, 0x9C, 0xE1, 0x70, 0xF6, 0x26, 0xD8, 0x3B, + 0xF9, 0x57, 0x18, 0x3D, 0xAD, 0x66, 0xF0, 0xCF, + 0x7B, 0x0C, 0x28, 0x4D, 0xB8, 0xEB, 0x7B, 0x04, + 0x1E, 0x7D, 0x40, 0x5F, 0x5A, 0x1E, 0x7E, 0x08, + 0x7F, 0x4C, 0x1E, 0x18, 0xE5, 0x3E, 0x6C, 0x90, + 0x3C, 0x89, 0x13, 0x2A, 0xC4, 0x2A, 0x94, 0xB5, + 0x3E, 0x18, 0x1C, 0x4C, 0xBA, 0xEA, 0x86, 0xD2, + 0x05, 0xA9, 0x59, 0x9C, 0x80, 0xC2, 0x45, 0xAD, + 0x30, 0x99, 0x18, 0x6A, 0x2F, 0x73, 0x8C, 0xF0, + 0xFE, 0xA4, 0xBD, 0x44, 0x3E, 0xEB, 0x98, 0x75, + 0x48, 0x08, 0x57, 0x45, 0xD8, 0x41, 0xDE, 0x61, + 0x6D, 0x06, 0x93, 0xC4, 0x99, 0x1B, 0x23, 0xB5, + 0x12, 0x22, 0x5C, 0xC7, 0x9E, 0x18, 0xEA, 0x64, + 0xDB, 0xCE, 0x1A, 0xAC, 0x5D, 0x9B, 0x80, 0xE2, + 0xBF, 0x3E, 0xC2, 0xA4, 0x78, 0x4F, 0xF1, 0xE3, + 0x7D, 0x2A, 0x20, 0x94, 0x13, 0xCD, 0xF3, 0x1C, + 0x33, 0x9C, 0xC1, 0x59, 0x85, 0x52, 0xCB, 0xDB, + 0x03, 0xDF, 0x11, 0xE6, 0xAB, 0x95, 0x82, 0x65, + 0x7A, 0x88, 0x73, 0xEE, 0xBA, 0x21, 0x1C, 0x2F, + 0xCD, 0xD7, 0xC6, 0xE5, 0x13, 0xDE, 0x7A, 0x9E, + 0xEE, 0x83, 0x8D, 0xC6, 0x47, 0x63, 0xE0, 0xC7, + 0xC4, 0xBE, 0x19, 0x25, 0xEE, 0xCC, 0x0A, 0x13, + 0x18, 0x9D, 0x34, 0x5B, 0x55, 0x6C, 0xC1, 0x6E, + 0xBF, 0x5A, 0xC5, 0x61, 0x75, 0x77, 0x49, 0x8C, + 0x67, 0x61, 0xE8, 0x72, 0x87, 0xE8, 0xCA, 0xBE, + 0x6E, 0xC3, 0xD7, 0x81, 0x8C, 0x78, 0x79, 0xC8, + 0x72, 0xDA, 0x1A, 0x40, 0x7D, 0x60, 0xE2, 0x5A, + 0x47, 0x38, 0xA1, 0x21, 0x22, 0x6B, 0x54, 0x74, + 0xDD, 0xF3, 0xBC, 0x96, 0x28, 0x7E, 0xC2, 0x8B, + 0x13, 0xEE, 0x53, 0xBC, 0x34, 0x67, 0x07, 0x87, + 0xD5, 0x6B, 0x93, 0x22, 0x21, 0xB9, 0xED, 0x17, + 0xE4, 0xA1, 0x96, 0xB2, 0xC6, 0xFF, 0x79, 0xA0, + 0xA7, 0xF9, 0xDD, 0x92, 0x78, 0xF7, 0xE3, 0x16, + 0x79, 0xEF, 0xEF, 0x31, 0x4D, 0x1D, 0x75, 0xF9, + 0xCF, 0x5A, 0x1B, 0x68, 0x16, 0x7F, 0xAF, 0x5F, + 0x30, 0xB7, 0xEF, 0xF8, 0x94, 0x63, 0x73, 0x3D, + 0xB3, 0x63, 0xE4, 0xE6, 0xD8, 0xAD, 0xF4, 0x80, + 0x5E, 0x82, 0xA3, 0xFB, 0x3D, 0x0F, 0xCF, 0x59, + 0xB8, 0x76, 0xAF, 0x27, 0x83, 0xE3, 0x2D, 0x6F, + 0xE0, 0xF3, 0x11, 0xD5, 0xAE, 0x82, 0x14, 0x1D, + 0x78, 0x95, 0xBF, 0x31, 0x22, 0x1B, 0x80, 0x12, + 0x02, 0xD7, 0x4D, 0x1B, 0x92, 0xE3, 0x15, 0xBA, + 0x67, 0xD6, 0x8F, 0xD4, 0xDA, 0xBF, 0xD4, 0x62, + 0xAD, 0x76, 0xDA, 0x04, 0xA2, 0xEA, 0x98, 0xD3, + 0xC3, 0x6E, 0x5F, 0x26, 0x3C, 0x5E, 0xD9, 0xEA, + 0x09, 0xF0, 0x02, 0xFC, 0xD2, 0x11, 0xF8, 0xA8, + 0x7E, 0xFF, 0x06, 0x28, 0x5B, 0xE5, 0x6F, 0x9A, + 0x00, 0xE7, 0x7F, 0xB9, 0xFB, 0x59, 0xBB, 0xDD, + 0x85, 0xF3, 0x40, 0xCE, 0xA3, 0x5E, 0x2E, 0x2E, + 0x34, 0x91, 0x58, 0x41, 0x00, 0xB5, 0xE1, 0x88, + 0x24, 0x51, 0xC7, 0xB6, 0xF3, 0x21, 0x52, 0x6E, + 0xE7, 0xFC, 0x42, 0xE7, 0x9C, 0xCC, 0x1E, 0x51, + 0x45, 0x39, 0xBE, 0x09, 0xFE, 0x1A, 0xC4, 0xF0, + 0x79, 0xF4, 0x05, 0xC7, 0xA7, 0xF8, 0x0F, 0xB6, + 0x5A, 0x7B, 0xD7, 0xE1, 0x6F, 0xF0, 0x9D, 0x67, + 0xA3, 0xE3, 0x3E, 0x2E, 0xB9, 0x8C, 0x83, 0x9E, + 0xFD, 0x2E, 0xA2, 0x92, 0x99, 0x3C, 0xC0, 0x99, + 0x01, 0xAB, 0x0D, 0xFA, 0x55, 0x96, 0x04, 0x60, + 0x1A, 0xAD, 0x4C, 0xBB, 0x3D, 0xBB, 0x7D, 0x8B, + 0x9F, 0x28, 0x85, 0x7D, 0xB9, 0xE4, 0x05, 0x79, + 0x7B, 0x63, 0xDD, 0x7F, 0x4D, 0xE7, 0x50, 0xD9, + 0x41, 0xFF, 0x53, 0xB1, 0xCE, 0x42, 0x7B, 0xD6, + 0x05, 0x1B, 0x4E, 0xAF, 0xC4, 0x8C, 0x17, 0xC8, + 0x52, 0xBD, 0x03, 0x3B, 0x92, 0x57, 0x4E, 0xA8, + 0x15, 0xC3, 0x26, 0x1C, 0x55, 0xC1, 0xFF, 0xAE, + 0xA9, 0x26, 0x2D, 0xA7, 0x8E, 0x3A, 0x7F, 0xA3, + 0x48, 0xA5, 0xBC, 0x14, 0x84, 0xF2, 0x90, 0xCE, + 0x35, 0x0F, 0x64, 0x6B, 0xD8, 0x1C, 0x12, 0xFE, + 0x5A, 0x4F, 0x0E, 0xCE, 0x81, 0x4E, 0x79, 0x6B, + 0xCF, 0x56, 0xA7, 0xDB, 0x24, 0xBC, 0xB0, 0x84, + 0x4C, 0xB0, 0xDA, 0xBE, 0xE6, 0x8F, 0xD7, 0x8E, + 0x0E, 0xA0, 0xD3, 0x55, 0xC2, 0x4A, 0x34, 0x1C, + 0xF9, 0xC7, 0x3D, 0x29, 0x70, 0x8B, 0xF0, 0x99, + 0x61, 0xF5, 0x11, 0xFB, 0x82, 0xE2, 0x67, 0x35, + 0x60, 0x78, 0x47, 0x81, 0x2A, 0x74, 0x5E, 0x4D, + 0x48, 0xD3, 0x7C, 0x32, 0xCA, 0x1B, 0xD2, 0xA2, + 0x5C, 0x3A, 0x2F, 0xCE, 0xB4, 0x6C, 0x3A, 0x6A, + 0x8F, 0x67, 0x46, 0x12, 0xE7, 0xAE, 0x6A, 0x3B, + 0x99, 0x04, 0x5E, 0x96, 0xD0, 0xB9, 0x84, 0xF6, + 0xA7, 0x64, 0x11, 0xE8, 0x0C, 0x51, 0xFD, 0x3F, + 0x18, 0xFA, 0xE8, 0x52, 0xD9, 0x4B, 0x99, 0x7A, + 0x25, 0x2B, 0x1B, 0x21, 0xAD, 0x8C, 0xFE, 0x0D, + 0x34, 0x51, 0x91, 0x75, 0x55, 0x6F, 0xEB, 0x9F, + 0x42, 0xDC, 0x73, 0x7D, 0x31, 0x0A, 0x74, 0x13, + 0x80, 0xB8, 0xC3, 0xED, 0x73, 0x9D, 0x79, 0x42, + 0xC0, 0x33, 0xAB, 0xC1, 0xCB, 0xB9, 0xD0, 0xBE, + 0xA0, 0x78, 0xB8, 0x3B, 0xEB, 0x3D, 0x1A, 0x3F, + 0xFB, 0x9B, 0xAA, 0x8F, 0x89, 0xF9, 0xD1, 0x22, + 0x82, 0xE6, 0x66, 0xEE, 0x2A, 0xFD, 0x9F, 0xF8, + 0x92, 0x7E, 0x10, 0xF5, 0xD5, 0x23, 0x0B, 0xB1, + 0xD6, 0xF1, 0x7A, 0x3D, 0x73, 0xE9, 0xCE, 0x7F, + 0xE6, 0x0B, 0x17, 0xBC, 0x23, 0xAE, 0x72, 0xB6, + 0xFA, 0x19, 0x46, 0xBB, 0xFE, 0xA4, 0xC1, 0x64, + 0xA8, 0x5E, 0xE8, 0xBB, 0x63, 0x58, 0x19, 0x50, + 0xAA, 0x36, 0xC2, 0x4B, 0x38, 0x24, 0xD1, 0x2E, + 0xAE, 0xAD, 0x6E, 0x34, 0x64, 0xA8, 0xC8, 0xF2, + 0x4E, 0x74, 0x5C, 0x98, 0xE8, 0xDF, 0x99, 0x8C, + 0x41, 0x79, 0x60, 0x2D, 0xD5, 0xF4, 0xE3, 0xE9, + 0x1D, 0xF6, 0x5A, 0xA9, 0x69, 0x8E, 0xA1, 0x4F, + 0xD3, 0x1B, 0x09, 0xA8, 0x7A, 0xD1, 0xE1, 0xCF, + 0xAC, 0xBA, 0xD0, 0xD1, 0x34, 0x34, 0x8F, 0xC1, + 0xD1, 0xA8, 0xAF, 0x6E, 0x92, 0xE0, 0xB0, 0xF6, + 0xF9, 0x08, 0xA0, 0xCB, 0x58, 0x34, 0xF6, 0x68, + 0xA2, 0xBF, 0x05, 0x39, 0x63, 0xBA, 0x4F, 0xEF, + 0xE3, 0x95, 0x69, 0xD5, 0x89, 0x7C, 0x64, 0x07, + 0x13, 0x42, 0x14, 0xF1, 0xA6, 0x9B, 0x87, 0xE5, + 0xF4, 0x49, 0xAE, 0x67, 0x65, 0xCC, 0xF2, 0x26, + 0xF8, 0x31, 0xBD, 0x33, 0x6A, 0x87, 0x77, 0x4E, + 0xB1, 0xEE, 0xA4, 0xA2, 0xC8, 0xA0, 0x4A, 0xC1, + 0xDF, 0x55, 0xE0, 0xDE, 0x53, 0x15, 0x3B, 0xEC, + 0x55, 0x32, 0xCA, 0x06, 0xE4, 0x78, 0x59, 0x63, + 0x10, 0x68, 0xA9, 0x46, 0x1B, 0xEF, 0x73, 0x6D, + 0x1A, 0x02, 0x64, 0x12, 0x76, 0x9B, 0xDB, 0x7C, + 0x03, 0x35, 0x19, 0xE1, 0x58, 0x7A, 0x87, 0x0C, + 0x76, 0xDC, 0xFC, 0xC0, 0x28, 0xE4, 0xA2, 0x07, + 0x9C, 0x28, 0x05, 0x21, 0x13, 0x58, 0xEF, 0x05, + 0xBB, 0xAB, 0x94, 0xA2, 0x93, 0xBC, 0x31, 0x61, + 0x26, 0x39, 0x38, 0x0C, 0xC4, 0x67, 0xDA, 0xA5, + 0xE4, 0x1E, 0x1B, 0xB6, 0xE5, 0x73, 0xD6, 0x6C, + 0xEE, 0xBC, 0x9D, 0xB9, 0xE7, 0xD9, 0x45, 0x2F, + 0xF2, 0xB6, 0x92, 0x54, 0x41, 0x05, 0xB7, 0xB7, + 0xFC, 0x37, 0x63, 0x6A, 0xB4, 0xBE, 0xB8, 0x3E, + 0xD8, 0x53, 0x3B, 0xF8, 0x7D, 0x9A, 0x05, 0xDF, + 0x20, 0x02, 0x27, 0x64, 0x38, 0xFA, 0x7D, 0xAF, + 0x7F, 0xFA, 0xD1, 0xB7, 0x32, 0xC5, 0x74, 0x3E, + 0x04, 0xA2, 0x67, 0x79, 0x02, 0x2E, 0x6F, 0xA1, + 0x27, 0x87, 0x07, 0xB5, 0x9F, 0x0A, 0x7D, 0x5E, + 0x14, 0xA0, 0x31, 0x46, 0x3F, 0xA9, 0xDE, 0x98, + 0xB9, 0x89, 0xA0, 0x4A, 0x7A, 0xBD, 0x15, 0xAE, + 0x2D, 0x0B, 0x38, 0x9A, 0xD8, 0x0E, 0xD2, 0xBA, + 0x6D, 0xA1, 0x04, 0x1E, 0x4E, 0x39, 0x87, 0x4B, + 0xC8, 0x3C, 0x74, 0x35, 0x4D, 0xC8, 0x1B, 0x42, + 0x06, 0x5B, 0x73, 0xB7, 0x33, 0x86, 0x4A, 0x10, + 0x2A, 0x10, 0x16, 0x28, 0x6F, 0x2A, 0xE3, 0x86, + 0xDE, 0xA3, 0x44, 0x23, 0xE2, 0x90, 0xC4, 0x20, + 0x90, 0xE0, 0xB8, 0xE6, 0xA7, 0xB6, 0xD6, 0x92, + 0xF4, 0xF8, 0x8A, 0xBC, 0xAC, 0x31, 0x47, 0x8F, + 0xAA, 0xE0, 0xD9, 0xF7, 0xE3, 0xCB, 0x11, 0xA4, + 0x6B, 0x05, 0xB3, 0xB8, 0x72, 0x69, 0xE6, 0xDD, + 0x75, 0x0F, 0x20, 0x1D, 0x3F, 0xC6, 0x96, 0xA0, + 0x18, 0xB6, 0x24, 0xA1, 0xA6, 0xFD, 0x0C, 0x80, + 0x1E, 0xD2, 0x28, 0xA2, 0x1A, 0x27, 0xF4, 0x23, + 0x59, 0x1A, 0xCC, 0x0F, 0xD4, 0x99, 0xD0, 0xB4, + 0x1E, 0x91, 0xC7, 0xD8, 0x8F, 0x8C, 0x5B, 0xEB, + 0xB5, 0x9F, 0xFF, 0x4F, 0xD0, 0xD5, 0xB7, 0x60, + 0xCC, 0x0A, 0x10, 0x38, 0xBF, 0xA8, 0x2E, 0xCC, + 0xEB, 0x26, 0xB0, 0x78, 0xB3, 0xE0, 0x40, 0xAF, + 0xCD, 0x12, 0xC5, 0x3A, 0x24, 0xD8, 0xEE, 0x3A, + 0x64, 0x83, 0x2E, 0xD9, 0x25, 0x21, 0x66, 0xA5, + 0x28, 0xD1, 0xE1, 0x84, 0x25, 0x1B, 0x20, 0xB8, + 0xF5, 0x76, 0xB6, 0x3E, 0x4B, 0xC6, 0xEC, 0xC1, + 0xC7, 0xAC, 0xC4, 0xAD, 0xCE, 0xF0, 0xB4, 0x0F, + 0x35, 0x1E, 0xCE, 0x4E, 0xE3, 0x57, 0x30, 0xFC, + 0xF4, 0x9B, 0x86, 0xB0, 0xDD, 0x3F, 0x2F, 0xB6, + 0x10, 0x20, 0xE4, 0x24, 0x17, 0x1C, 0x24, 0xC6, + 0x89, 0xE4, 0x14, 0xAD, 0x2E, 0x41, 0x08, 0x33, + 0x88, 0xB1, 0x6F, 0x11, 0x85, 0xAF, 0x58, 0x17, + 0xE3, 0x91, 0xB4, 0x72, 0xA2, 0x7F, 0xA3, 0x98, + 0xAF, 0xB7, 0x6B, 0x58, 0x76, 0xA3, 0x11, 0x1C, + 0x8A, 0x1A, 0xE6, 0x58, 0x54, 0xB0, 0xB9, 0x6E, + 0x46, 0xCB, 0x16, 0xC0, 0x63, 0x0C, 0xEE, 0xA2, + 0xAE, 0xF6, 0x71, 0xEF, 0xD1, 0xB9, 0x3D, 0xB7, + 0x76, 0xCE, 0x5B, 0x84, 0x66, 0x7C, 0x7D, 0xF1, + 0x96, 0x60, 0x34, 0xF6, 0xD1, 0x64, 0x27, 0xD9, + 0xF3, 0x78, 0x8B, 0xF4, 0xC3, 0x1D, 0x37, 0xC0, + 0xF4, 0x4A, 0xD0, 0xA5, 0x9A, 0xEB, 0xDD, 0x79, + 0x54, 0x5D, 0xEB, 0x04, 0xC1, 0xA4, 0xBC, 0xED, + 0xE3, 0x74, 0xC3, 0xB9, 0x9A, 0x6A, 0xAA, 0x06, + 0xD1, 0xF0, 0x0F, 0xC5, 0xEF, 0x7E, 0x0B, 0xC8, + 0xF4, 0x94, 0x4E, 0x69, 0x0E, 0x36, 0x00, 0x13, + 0x45, 0xCE, 0x68, 0x13, 0xFE, 0x7F, 0x29, 0xA2, + 0x1D, 0x79, 0xDF, 0xF2, 0x27, 0xFB, 0xAE, 0x52, + 0x05, 0x78, 0xD7, 0xB9, 0xF7, 0x38, 0x68, 0xD5, + 0xBA, 0xD7, 0xCF, 0x09, 0xC6, 0xD2, 0x5B, 0xC6, + 0x98, 0xE4, 0xEC, 0xD5, 0xE9, 0xC2, 0xA5, 0x1A, + 0x52, 0xC8, 0xA7, 0xBA, 0x3D, 0x74, 0x75, 0x00, + 0xAA, 0xDD, 0x6A, 0x3F, 0xB6, 0x2F, 0x08, 0xB7, + 0x1C, 0x6B, 0x52, 0x0C, 0xC9, 0xE4, 0x4D, 0xF4, + 0xC5, 0x26, 0x1F, 0x35, 0x41, 0x25, 0x68, 0x17, + 0xA8, 0x81, 0x75, 0xF4, 0x66, 0x41, 0xB5, 0xE4, + 0x1D, 0x92, 0xEE, 0xDA, 0x0F, 0x56, 0x76, 0xC6, + 0xAA, 0x0F, 0xA8, 0x63, 0x8D, 0xF0, 0x69, 0x63, + 0x93, 0x45, 0xBC, 0x76, 0x40, 0xBE, 0xA9, 0x96, + 0x36, 0xAF, 0x2F, 0x6B, 0x3E, 0xAB, 0xF3, 0xC0, + 0xD7, 0xD5, 0xB1, 0x23, 0x23, 0xA2, 0xA0, 0xC4, + 0xC5, 0x70, 0xEF, 0x66, 0x79, 0x15, 0xF0, 0xD0, + 0x40, 0x0A, 0x33, 0x0C, 0xF3, 0x32, 0x6D, 0x8D, + 0xB4, 0x44, 0x46, 0x78, 0x3F, 0x8D, 0x75, 0x40, + 0xA5, 0x60, 0xBC, 0x9B, 0x76, 0xDF, 0x25, 0xF4, + 0xE9, 0xED, 0xAC, 0x74, 0x2F, 0x9A, 0x00, 0xC4, + 0x2B, 0x52, 0x26, 0x79, 0x09, 0x19, 0x57, 0x89, + 0x60, 0x14, 0xBE, 0x65, 0xBD, 0x7B, 0x4D, 0x7D, + 0x9B, 0x8B, 0x9E, 0x72, 0x6C, 0x0C, 0x57, 0xC7, + 0x00, 0x08, 0x38, 0x7C, 0x37, 0x45, 0x9D, 0x55, + 0xA2, 0x62, 0x5E, 0x34, 0x19, 0x99, 0x31, 0x16, + 0xF1, 0x14, 0x44, 0x2D, 0xE3, 0x7E, 0x22, 0xE1, + 0xA2, 0xB8, 0x9A, 0x9F, 0xE0, 0x37, 0x29, 0xBB, + 0xCD, 0x46, 0xEE, 0x0A, 0x62, 0x2B, 0x98, 0x34, + 0xBA, 0x9E, 0x54, 0x1B, 0xB1, 0x5C, 0x4F, 0xE9, + 0xAA, 0xE4, 0x95, 0x8C, 0xA4, 0xEF, 0xC2, 0xB1, + 0x7F, 0xF9, 0x80, 0xDA, 0x55, 0x95, 0x92, 0xC0, + 0x86, 0xF4, 0x2D, 0x99, 0x3E, 0x17, 0xDC, 0x55, + 0xA6, 0x33, 0x85, 0x90, 0x31, 0xC8, 0xFF, 0x58, + 0x83, 0xC5, 0xBA, 0x60, 0x20, 0x5F, 0x87, 0x29, + 0x20, 0x5A, 0x7D, 0x44, 0x2B, 0xA0, 0xE2, 0x99, + 0xC8, 0x70, 0xBE, 0x89, 0xC5, 0xBC, 0xF6, 0x0D, + 0x04, 0xC0, 0x96, 0xD1, 0x5C, 0xD1, 0x90, 0x43, + 0xD3, 0x7B, 0x73, 0x52, 0x30, 0xB6, 0xA9, 0x7C, + 0x0A, 0xA3, 0x24, 0x0E, 0x80, 0xFE, 0xBE, 0x31, + 0xFD, 0xB5, 0x96, 0x04, 0x2B, 0xCF, 0x0B, 0x28, + 0x1F, 0x7A, 0xCF, 0xC4, 0x82, 0x78, 0x52, 0x30, + 0xB1, 0x34, 0x12, 0x50, 0x03, 0x09, 0x1C, 0x8B, + 0x80, 0x60, 0xE3, 0xA1, 0xE5, 0x61, 0xF7, 0xD7, + 0xB6, 0x74, 0xBE, 0xD6, 0x58, 0x03, 0xD3, 0xE5, + 0xF7, 0xAC, 0x07, 0x60, 0xB7, 0x8A, 0xEC, 0xFA, + 0xC6, 0x0F, 0xF0, 0x20, 0x04, 0x6B, 0x8F, 0x61, + 0x09, 0x92, 0x03, 0xFB, 0x85, 0x99, 0x94, 0x9D, + 0x2E, 0x6A, 0xC2, 0x9F, 0x20, 0x46, 0x2A, 0x96, + 0xED, 0x42, 0x7D, 0x64, 0xA9, 0xE4, 0x1B, 0xDE, + 0x11, 0x20, 0x12, 0x93, 0xE6, 0x2B, 0xE5, 0x93, + 0x48, 0x37, 0x8C, 0x5A, 0x54, 0x0D, 0xEB, 0xF0, + 0x9F, 0x9D, 0xE4, 0xA5, 0xC4, 0x93, 0x6F, 0x6A, + 0xE3, 0x99, 0x69, 0xD9, 0xFE, 0x0C, 0x4E, 0xEC, + 0x8B, 0x30, 0x1F, 0x7A, 0xB8, 0xC8, 0x5B, 0x61, + 0x8E, 0xC2, 0x10, 0x90, 0x57, 0xB4, 0x72, 0x58, + 0x7F, 0x41, 0x29, 0x7E, 0xF9, 0xBE, 0x40, 0xC3, + 0x6F, 0xA9, 0xE3, 0x00, 0xE9, 0xC8, 0xFD, 0x4B, + 0xFD, 0x3F, 0xE3, 0x3F, 0x25, 0x22, 0xFD, 0xB7, + 0x2D, 0x57, 0xEF, 0x91, 0x08, 0xF0, 0x20, 0x56, + 0x30, 0xFA, 0x83, 0x69, 0xFD, 0x56, 0x5A, 0x9B, + 0xCE, 0xF8, 0x28, 0x02, 0xB4, 0x91, 0x35, 0x75, + 0x9E, 0x63, 0x99, 0x48, 0xCF, 0x35, 0xF5, 0x58, + 0x0C, 0x48, 0x8F, 0x0A, 0x2D, 0x9A, 0xE6, 0x40, + 0xF6, 0x21, 0xB5, 0x69, 0xC1, 0x09, 0x31, 0x00, + 0xA3, 0xC1, 0x4C, 0x99, 0x70, 0x4F, 0x5A, 0x63, + 0x17, 0x90, 0xB8, 0xF8, 0x3A, 0x0E, 0xFD, 0x67, + 0xEA, 0x0E, 0xBA, 0x7B, 0x1E, 0xEF, 0x37, 0x84, + 0xD5, 0x51, 0x37, 0x01, 0xD6, 0x93, 0x15, 0xDF, + 0x56, 0x89, 0x0E, 0x54, 0xF5, 0x1D, 0xF7, 0xE5, + 0xB7, 0xC1, 0xF1, 0xC2, 0xD9, 0x14, 0x6F, 0x40, + 0x55, 0x67, 0x50, 0x7C, 0x58, 0x35, 0x8B, 0x39, + 0xCB, 0xB5, 0x87, 0xF5, 0x55, 0x5E, 0x26, 0x8C, + 0x5B, 0x73, 0x0E, 0xBB, 0x25, 0x51, 0x0E, 0xAD, + 0x57, 0x72, 0x7B, 0x68, 0x83, 0x11, 0x1E, 0x3A, + 0x3D, 0xA4, 0x7C, 0x18, 0xB7, 0x70, 0x18, 0xBC, + 0x72, 0x03, 0x4A, 0xA1, 0xD2, 0xF9, 0xA9, 0x8A, + 0x25, 0x45, 0x19, 0xEE, 0x93, 0x06, 0xB5, 0x09, + 0x71, 0xC9, 0x2D, 0xFD, 0x2B, 0xF3, 0xC7, 0x64, + 0x5F, 0xCE, 0x71, 0x1D, 0x81, 0x96, 0x67, 0xBF, + 0x01, 0x39, 0x8C, 0xE7, 0xA2, 0xD0, 0x98, 0x57, + 0x5A, 0xFD, 0x21, 0xC7, 0x46, 0xAA, 0xB4, 0xE4, + 0x0E, 0xBE, 0xC6, 0x68, 0x3E, 0x38, 0xF5, 0xA2, + 0xED, 0x73, 0xCC, 0x53, 0x7E, 0x7E, 0x03, 0x32, + 0xDC, 0xB6, 0xC1, 0x03, 0x9E, 0xB3, 0x2A, 0xAD, + 0xC0, 0xC3, 0x6E, 0x47, 0xFB, 0x1E, 0xB7, 0x0D, + 0x86, 0x95, 0x09, 0xA6, 0x9D, 0x6F, 0x92, 0xFC, + 0xFF, 0x2C, 0x7D, 0x09, 0x16, 0x68, 0x50, 0x3E, + 0x4F, 0x23, 0x4C, 0x93, 0x95, 0x2A, 0xE1, 0x9B, + 0x16, 0xF0, 0x0F, 0xFF, 0x79, 0xA8, 0x06, 0xF9, + 0x70, 0x61, 0x72, 0x2C, 0xE8, 0x91, 0x01, 0x6D, + 0x45, 0xE5, 0x82, 0x5D, 0x26, 0x21, 0xAD, 0x3D, + 0x77, 0x73, 0x23, 0x04, 0x84, 0x27, 0xA3, 0x5D, + 0x6D, 0xA8, 0x99, 0xC1, 0xCE, 0x4F, 0xA9, 0xF7, + 0xAB, 0x5C, 0xDE, 0x01, 0xE6, 0x1E, 0xEF, 0xE6, + 0xFD, 0xE0, 0x68, 0x85, 0x3E, 0xEE, 0xBF, 0xF1, + 0x0D, 0x79, 0xF4, 0xA2, 0xB4, 0x14, 0xBC, 0x0C, + 0x49, 0x77, 0x03, 0x71, 0x08, 0x3E, 0x40, 0xA6, + 0xD7, 0x03, 0xFA, 0xE2, 0xFB, 0xC7, 0x59, 0x30, + 0x6E, 0x07, 0x06, 0x1C, 0x7C, 0x47, 0xE5, 0x4C, + 0x57, 0x0A, 0x91, 0x4A, 0x43, 0xE4, 0x8A, 0xCD, + 0x6E, 0x92, 0x01, 0xE2, 0x52, 0xC1, 0x92, 0x34, + 0x8E, 0x64, 0x0F, 0x39, 0x63, 0x53, 0xAB, 0xE5, + 0x44, 0xD5, 0xAA, 0xAA, 0xF6, 0x03, 0x89, 0xB9, + 0xDD, 0xB2, 0x2D, 0x56, 0x1A, 0xE0, 0x72, 0x5A, + 0x52, 0x19, 0x46, 0xEA, 0xB3, 0xCE, 0xB3, 0x59, + 0x46, 0x7A, 0xA7, 0x48, 0x37, 0x0C, 0x09, 0xBA, + 0x92, 0x70, 0x17, 0x7F, 0xF5, 0xD3, 0x60, 0x44, + 0xC4, 0xC6, 0xC6, 0x7D, 0xD2, 0x21, 0xAC, 0x3F, + 0x62, 0x6C, 0xE9, 0xBA, 0x4C, 0xF3, 0x82, 0x7E, + 0x6D, 0x3A, 0x92, 0xDC, 0x94, 0xE4, 0x5F, 0xA6, + 0x8B, 0x66, 0xA0, 0xDD, 0xE2, 0x97, 0x83, 0xED, + 0xF5, 0x9D, 0xDF, 0x74, 0x77, 0x23, 0x7D, 0xDA, + 0xC4, 0xFB, 0x92, 0x1A, 0xD9, 0x37, 0x36, 0xD2, + 0x88, 0xC9, 0xEA, 0x0F, 0x98, 0xBD, 0xC5, 0xF8, + 0xAA, 0x19, 0x75, 0x12, 0x6A, 0x41, 0xB5, 0xB3, + 0xB5, 0xA4, 0x96, 0xDC, 0x2B, 0x49, 0x86, 0x66, + 0x35, 0xD8, 0x4A, 0x62, 0xB4, 0xCB, 0x1E, 0x27, + 0xC1, 0xAD, 0x34, 0x0E, 0x26, 0x16, 0xF2, 0xC2, + 0x22, 0x52, 0x84, 0xD8, 0xD1, 0x32, 0xB8, 0x9C, + 0xFE, 0x64, 0x42, 0x9F, 0xE4, 0x69, 0xF0, 0xAE, + 0x3B, 0xD9, 0x2C, 0xA0, 0x14, 0xEB, 0x69, 0x74, + 0x7C, 0xE2, 0xA6, 0x60, 0xE1, 0x52, 0x1C, 0xCC, + 0xBF, 0xE6, 0xA1, 0x83, 0x20, 0x5D, 0x9E, 0xA3, + 0xFB, 0x84, 0x8B, 0x33, 0xE6, 0xC9, 0x32, 0x83, + 0xC0, 0x3F, 0x98, 0x1D, 0x6E, 0xC0, 0x50, 0x71, + 0x29, 0x60, 0x5F, 0x36, 0xB4, 0x68, 0x1D, 0xB9, + 0x76, 0x73, 0xC3, 0x80, 0xC5, 0xBC, 0x59, 0x7B, + 0x59, 0xB4, 0xE0, 0x6A, 0x80, 0xCD, 0x4D, 0x8C, + 0x9E, 0xE0, 0x0B, 0x45, 0x7D, 0x54, 0xD4, 0xC4, + 0x97, 0x6C, 0x54, 0xEF, 0x14, 0x64, 0xBD, 0x3B, + 0xD7, 0xEE, 0xF4, 0xD1, 0x41, 0x76, 0x3A, 0x24, + 0x7A, 0xC2, 0xCA, 0x68, 0x28, 0x53, 0x46, 0xF7, + 0x1B, 0xDA, 0x4B, 0x7A, 0x56, 0x75, 0x86, 0xFB, + 0x31, 0x2C, 0x27, 0xF9, 0x4D, 0x35, 0xA4, 0x82, + 0xE7, 0x2F, 0x41, 0xB4, 0xCA, 0xCE, 0x75, 0x94, + 0x08, 0x54, 0xE2, 0x9E, 0x99, 0xC9, 0x85, 0xDE, + 0x6F, 0x80, 0x95, 0x59, 0x3E, 0x54, 0x9F, 0x31, + 0xF8, 0xDE, 0xD0, 0xD7, 0xA6, 0xD4, 0xD3, 0xBB, + 0xD9, 0xC7, 0x55, 0xDD, 0xAE, 0xAD, 0x9E, 0x57, + 0x4A, 0x33, 0x5D, 0x7A, 0xA6, 0xA3, 0xCA, 0xF9, + 0x4C, 0x5B, 0x51, 0xCC, 0x22, 0xBB, 0x76, 0x44, + 0x17, 0xDE, 0x22, 0xA1, 0xDF, 0x80, 0x13, 0x7D, + 0xE5, 0x34, 0x7E, 0x75, 0x73, 0x10, 0x40, 0xFB, + 0x9A, 0x21, 0xCD, 0xD3, 0xD3, 0x84, 0xB6, 0x0C, + 0x31, 0x1E, 0xB5, 0x42, 0xF4, 0x34, 0x11, 0x7F, + 0x4A, 0x23, 0xA8, 0xA5, 0x8F, 0x20, 0xCD, 0xA9, + 0xF2, 0xE4, 0xEE, 0xFA, 0x57, 0xD1, 0x22, 0x1C, + 0xA5, 0xDC, 0x0B, 0x25, 0xFE, 0xC2, 0xA7, 0x7E, + 0x09, 0x2E, 0xDA, 0x40, 0x9F, 0x6C, 0xC8, 0x71, + 0x58, 0x91, 0x04, 0x25, 0xD0, 0x06, 0xEA, 0x1B, + 0xCD, 0x9D, 0x50, 0xD8, 0x40, 0x24, 0xAC, 0xC3, + 0xB4, 0x07, 0x6E, 0x76, 0xF4, 0x4C, 0xD8, 0x80, + 0xD0, 0x20, 0xF5, 0x15, 0x6A, 0x0A, 0x12, 0xF8, + 0x6B, 0x67, 0x77, 0x34, 0xAE, 0x60, 0x68, 0x13, + 0x5B, 0x8E, 0xFF, 0x5E, 0x7A, 0x77, 0x67, 0x0D, + 0xE6, 0x96, 0x43, 0x9F, 0x8F, 0x77, 0x5F, 0x97, + 0x23, 0x91, 0x33, 0x72, 0xC7, 0x8A, 0xC7, 0x80, + 0xCF, 0xE7, 0x71, 0x06, 0x25, 0xB7, 0x4B, 0x89, + 0x6A, 0x46, 0x67, 0x19, 0x49, 0x44, 0x03, 0x52, + 0x32, 0xB1, 0x8F, 0xE7, 0x9E, 0xDA, 0x03, 0x41, + 0xA3, 0xAC, 0xE5, 0xF3, 0x96, 0xE6, 0xAC, 0xFF, + 0xEC, 0x35, 0x4D, 0x83, 0xA9, 0xCE, 0x7C, 0x52, + 0xF2, 0x36, 0x97, 0xF0, 0x28, 0x36, 0x54, 0x59, + 0x96, 0xEA, 0xEE, 0xB2, 0xC1, 0xAB, 0xA4, 0x96, + 0x62, 0xD4, 0x3C, 0xF0, 0x1F, 0x2D, 0x65, 0x0E, + 0x46, 0x7E, 0x12, 0x31, 0x8F, 0xA7, 0x8D, 0x7A, + 0x4A, 0x41, 0x15, 0x57, 0x90, 0xF6, 0xF1, 0xE8, + 0xE8, 0xE3, 0x57, 0x7B, 0x55, 0x85, 0x95, 0x97, + 0xB3, 0x29, 0x3D, 0x02, 0x73, 0x1E, 0x87, 0x1F, + 0x01, 0x89, 0x06, 0x88, 0x9E, 0x8A, 0x2E, 0xE0, + 0x99, 0xFC, 0xF0, 0x48, 0x60, 0x37, 0x65, 0x25, + 0xDB, 0x89, 0xC9, 0x7A, 0x51, 0x7E, 0x35, 0x92, + 0x00, 0xC9, 0x61, 0x3F, 0x88, 0xE3, 0x20, 0x01, + 0x46, 0x5A, 0x2C, 0x37, 0x02, 0xC9, 0x15, 0x0F, + 0xB2, 0xEB, 0xC3, 0x55, 0x18, 0xF0, 0x15, 0x1A, + 0x08, 0x8E, 0xB8, 0x9D, 0x18, 0xE4, 0x9F, 0x34, + 0x10, 0x67, 0x68, 0x57, 0x60, 0x61, 0xEC, 0xD5, + 0xD9, 0xA8, 0x3A, 0xAB, 0xD6, 0xD2, 0x7A, 0x47, + 0x3D, 0xA4, 0x08, 0x7C, 0x3E, 0x4D, 0x76, 0x08, + 0x19, 0x22, 0xB2, 0x89, 0x57, 0x84, 0xC2, 0x98, + 0x72, 0xB8, 0x8B, 0xE0, 0x85, 0xA1, 0x3A, 0xC2, + 0xA0, 0x06, 0x43, 0x03, 0xCF, 0x4F, 0x27, 0x80, + 0x48, 0x9A, 0xBC, 0xB3, 0x3C, 0xC4, 0x5E, 0xAC, + 0x8B, 0x85, 0x6F, 0x21, 0xD6, 0xFE, 0x12, 0x90, + 0x53, 0x2F, 0x65, 0x32, 0x8D, 0x03, 0xFE, 0xFE, + 0x61, 0x04, 0x47, 0x24, 0x6A, 0xB5, 0x01, 0x98, + 0xB9, 0x27, 0x10, 0xE1, 0x32, 0x3D, 0x2A, 0xA0, + 0xC5, 0x70, 0xDE, 0x1E, 0x10, 0xD7, 0x01, 0x50, + 0x4F, 0x87, 0xCA, 0xD9, 0xBF, 0x12, 0xEA, 0x38, + 0x4C, 0x43, 0xD5, 0x5A, 0xEF, 0x3E, 0x21, 0x8E, + 0x59, 0x77, 0x23, 0xED, 0x51, 0x09, 0x99, 0x73, + 0xD2, 0x56, 0x04, 0xDD, 0x8F, 0x5F, 0xDF, 0x79, + 0xFF, 0x16, 0x8C, 0xB0, 0xBA, 0x8A, 0x1A, 0x56, + 0xAF, 0xCA, 0x19, 0xF2, 0x64, 0x1A, 0xF5, 0x1E, + 0xA7, 0xA7, 0x84, 0x3D, 0xAD, 0xC1, 0x0E, 0x22, + 0xA1, 0x45, 0xFC, 0xB4, 0x13, 0x91, 0x34, 0xB7, + 0x48, 0xEF, 0x9E, 0xD9, 0x0B, 0xE3, 0x82, 0x75, + 0x80, 0xC5, 0xD9, 0xA0, 0x77, 0xA3, 0xF9, 0xCC, + 0x67, 0xDD, 0xCB, 0x28, 0xC6, 0xE8, 0x2C, 0xB8, + 0xAC, 0x63, 0xBD, 0x3B, 0x28, 0x4A, 0xE9, 0x2D, + 0x29, 0x84, 0xD7, 0x4F, 0x61, 0x06, 0xE3, 0x37, + 0xC1, 0x58, 0x20, 0x5D, 0x0A, 0xE7, 0x45, 0x29, + 0x7D, 0xED, 0x0F, 0xCE, 0x00, 0x95, 0x2A, 0x62, + 0x38, 0xA8, 0x1A, 0x3A, 0x96, 0x0E, 0x56, 0xD9, + 0x18, 0xC2, 0x25, 0xA5, 0xAA, 0x27, 0x0A, 0x6E, + 0xDD, 0x1C, 0x35, 0x6C, 0xC1, 0x26, 0x90, 0xF6, + 0x43, 0x1B, 0x34, 0xDA, 0xE1, 0x5D, 0x09, 0x7C, + 0xBE, 0x0F, 0x40, 0xD3, 0x24, 0x82, 0x0B, 0xFF, + 0xE6, 0xB7, 0x10, 0xD6, 0x36, 0xD0, 0xE6, 0xC0, + 0xBE, 0x65, 0x4C, 0x83, 0xF1, 0xDA, 0xDE, 0xCE, + 0xE8, 0x5A, 0x80, 0x88, 0xFE, 0x9B, 0x79, 0x54, + 0xA3, 0xA4, 0x5A, 0x76, 0xD0, 0xE2, 0xCE, 0x92, + 0x53, 0x7D, 0x9C, 0xDA, 0xA1, 0xED, 0x9F, 0x56, + 0x05, 0x0A, 0xA4, 0x81, 0xC7, 0x82, 0x5B, 0xB8, + 0xC7, 0xA8, 0x95, 0x21, 0x99, 0x0B, 0x0F, 0xD2, + 0x66, 0x68, 0xC3, 0x07, 0x53, 0x93, 0x8C, 0x68, + 0x5A, 0xF5, 0x6F, 0x5E, 0x07, 0x68, 0x70, 0xF2, + 0x6A, 0x78, 0xA8, 0xDB, 0x24, 0x6F, 0xD9, 0x74, + 0x38, 0x29, 0xBF, 0x50, 0xCC, 0xC8, 0x34, 0x82, + 0x69, 0x1A, 0xF4, 0x1A, 0x95, 0xE1, 0x31, 0x39, + 0x00, 0xBE, 0xF6, 0x25, 0x19, 0x5A, 0xF6, 0xA7, + 0xB7, 0xC7, 0xAC, 0xFA, 0x52, 0x6D, 0xC7, 0xEA, + 0xFF, 0xC4, 0xB0, 0x84, 0x41, 0x2D, 0x6B, 0x22, + 0x57, 0x1F, 0x8A, 0x97, 0x71, 0xEF, 0x54, 0xA7, + 0xA9, 0xB5, 0xA0, 0x7A, 0xFF, 0x52, 0xBC, 0x78, + 0x56, 0x8C, 0x9C, 0x2A, 0xFB, 0x31, 0xA6, 0xC8, + 0x87, 0xC8, 0x82, 0x72, 0x00, 0x68, 0x27, 0xDE, + 0x54, 0x67, 0x4C, 0x36, 0x4E, 0xE2, 0x77, 0x7F, + 0xAD, 0x15, 0x1B, 0xC9, 0x07, 0xC4, 0x84, 0x50, + 0x84, 0x45, 0xB5, 0x1D, 0xCE, 0x3F, 0x7C, 0xDF, + 0x73, 0x88, 0x3E, 0x6D, 0xAC, 0x18, 0x67, 0x2C, + 0x1D, 0x31, 0xD5, 0x41, 0x6E, 0xFC, 0x3D, 0xFE, + 0x5C, 0x6D, 0x3B, 0xCB, 0x67, 0x05, 0x44, 0x8B, + 0x02, 0xB7, 0xF5, 0x05, 0xFD, 0x1D, 0x7E, 0x13, + 0x90, 0xCE, 0xED, 0xD2, 0xAB, 0x08, 0xFF, 0xC3, + 0x91, 0x2C, 0x79, 0x06, 0xDB, 0x54, 0xAB, 0xFF, + 0xF6, 0x9D, 0xBB, 0xDC, 0x3C, 0xCD, 0x03, 0xE7, + 0xD8, 0x1A, 0x4E, 0x7F, 0xCB, 0x1B, 0xA3, 0xCA, + 0xDC, 0x14, 0xC5, 0xFE, 0x46, 0x38, 0x07, 0x82, + 0x5E, 0x9A, 0x77, 0x9C, 0xB4, 0x44, 0x26, 0xBA, + 0xC2, 0x27, 0xD0, 0xF4, 0x75, 0x67, 0x4A, 0x15, + 0x2A, 0x55, 0x2A, 0x61, 0x87, 0x55, 0xA0, 0xFB, + 0xE0, 0x93, 0xA5, 0xD7, 0xDF, 0x0D, 0x97, 0xD3, + 0x93, 0x39, 0x0B, 0x5A, 0xC6, 0x86, 0x17, 0x7D, + 0x6A, 0xA4, 0x07, 0x9C, 0xB9, 0x0F, 0x42, 0x15, + 0x46, 0x71, 0x45, 0x70, 0x75, 0x4B, 0xD5, 0x80, + 0x64, 0x62, 0x92, 0x50, 0xBC, 0x10, 0xA1, 0x68, + 0x60, 0x5A, 0x6F, 0x31, 0x26, 0xCA, 0xB1, 0x48, + 0xFB, 0xC6, 0xF4, 0x3A, 0xCA, 0x52, 0x20, 0x2F, + 0x7D, 0xEC, 0x19, 0xF8, 0xAA, 0x27, 0xC2, 0x23, + 0xD0, 0x8F, 0x60, 0xFC, 0x7F, 0xA0, 0xCB, 0xDC, + 0xA2, 0xC6, 0xC4, 0xC5, 0x13, 0x00, 0xF3, 0x43, + 0x48, 0x6D, 0xFD, 0x7D, 0xB9, 0xA8, 0x14, 0xB0, + 0x0C, 0x72, 0x82, 0x2F, 0x99, 0x64, 0x41, 0x2B, + 0xB3, 0x34, 0x73, 0x73, 0xF4, 0x26, 0x1D, 0x06, + 0xDF, 0x6E, 0xF4, 0x20, 0x1E, 0x31, 0xE3, 0x8A, + 0x01, 0x6C, 0xDB, 0x3C, 0xE3, 0xC6, 0xC4, 0xC5, + 0xB8, 0x20, 0x51, 0xF1, 0xD6, 0xB0, 0x30, 0xB7, + 0x2D, 0xDA, 0x95, 0x01, 0x0D, 0xED, 0xEE, 0x6F, + 0x69, 0xFD, 0xCF, 0x9D, 0xDD, 0x05, 0xD6, 0xC0, + 0xFE, 0x11, 0x67, 0xAF, 0x53, 0x94, 0x60, 0xFC, + 0x56, 0xBA, 0x0C, 0x5F, 0xA7, 0x7E, 0xDA, 0x65 + }, + + .len = 4096 << 3 + }, + .validCipherLenInBits = { + .len = 4096 << 3 + }, + .cipher = { + .len_bits = 4096, + .offset_bits = 0 + }, + .dataunit_len = 4096 +}; + +static struct mlx5_test_data mlx5_test_case_cipher_aes_xts_4 = { + .key = { + .data = { + 0x94, 0xD8, 0xAE, 0xD7, 0x74, 0xBF, 0x51, 0xF1, + 0x2D, 0x19, 0x0B, 0x9A, 0xB3, 0xDC, 0x1E, 0xA9, + 0x5C, 0xC5, 0x47, 0xC1, 0xBB, 0x94, 0xDB, 0xFA, + 0x2B, 0x1E, 0x54, 0xA5, 0x76, 0xAD, 0x07, 0x4B, + 0x46, 0xE0, 0xAA, 0xF9, 0xF7, 0x5C, 0x64, 0x2D, + 0x8F, 0x04, 0x17, 0x72, 0xE5, 0x8D, 0xFE, 0x9C + }, + .len = 48 + }, + .cipher_iv = { + .data = { + 0x4F, 0xAE, 0xF7, 0x11, 0x7C, 0xDA, 0x59, 0xC6, + 0x6E, 0x4B, 0x92, 0x01, 0x3E, 0x76, 0x8A, 0xD5 + }, + .len = 16 + }, + .plaintext = { + .data = { + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, + 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79, 0x79 + }, + .len = 4096 << 3 + }, + .ciphertext = { + .data = { + 0xD3, 0x94, 0x3E, 0xC9, 0xD3, 0x43, 0x13, 0xD3, + 0x07, 0x7E, 0x51, 0x11, 0x97, 0xA5, 0xB1, 0xB2, + 0xB0, 0x55, 0xF8, 0xD6, 0xA2, 0x10, 0x78, 0x76, + 0x37, 0x95, 0x4F, 0x25, 0x99, 0xCC, 0x12, 0xD6, + 0xBB, 0x2A, 0x24, 0xF4, 0x2D, 0x82, 0xFF, 0x56, + 0xEC, 0x55, 0x3B, 0xDD, 0xF9, 0xE1, 0xC8, 0x48, + 0x68, 0x16, 0xD2, 0x2B, 0x95, 0x7C, 0x1F, 0xCB, + 0x32, 0xD6, 0x8F, 0x9A, 0x2E, 0xF7, 0x9B, 0xBE, + 0x72, 0xFC, 0x55, 0x33, 0x27, 0x21, 0x2F, 0x69, + 0xCA, 0x76, 0xA5, 0x9B, 0x21, 0x2E, 0x40, 0x57, + 0x2C, 0x1C, 0x98, 0x41, 0x9A, 0x5E, 0x55, 0x38, + 0xDE, 0xC2, 0x09, 0x57, 0x32, 0xA3, 0x34, 0x40, + 0xC1, 0x8D, 0xCF, 0x70, 0x15, 0xF2, 0x6F, 0x4F, + 0x6A, 0x04, 0xA6, 0x6D, 0xFF, 0x53, 0x25, 0x6A, + 0x0E, 0xD0, 0x87, 0x63, 0xA1, 0x6C, 0xB1, 0x99, + 0x4A, 0x42, 0xF5, 0xF6, 0xEA, 0xA4, 0xEB, 0x6D, + 0x70, 0x9B, 0x0F, 0x85, 0xE2, 0x43, 0x6C, 0x27, + 0x54, 0x57, 0x52, 0x1C, 0xCF, 0x72, 0x11, 0x83, + 0xC3, 0xF5, 0xC6, 0xB6, 0x07, 0xEC, 0x1A, 0xF5, + 0xAC, 0xA1, 0xF2, 0x3A, 0x01, 0x53, 0x0C, 0xA4, + 0x40, 0x19, 0xAF, 0x7B, 0x2D, 0xE7, 0x56, 0x8B, + 0x71, 0x5F, 0x8D, 0x96, 0xF1, 0x5D, 0x33, 0xAC, + 0xE7, 0xAD, 0x96, 0xDB, 0xBB, 0xF7, 0xF5, 0x3F, + 0x0E, 0x02, 0x2E, 0x80, 0xE9, 0xA2, 0x30, 0xD2, + 0x82, 0x65, 0xEC, 0x1A, 0xB2, 0xE8, 0x98, 0x23, + 0x42, 0x79, 0x43, 0x85, 0x3B, 0xF9, 0xFB, 0xBB, + 0xC5, 0x29, 0xAD, 0x95, 0xAE, 0x22, 0x5B, 0x26, + 0xDF, 0x76, 0x65, 0x37, 0x17, 0xBC, 0x58, 0xBB, + 0x1B, 0x0A, 0x71, 0xDC, 0x68, 0xFF, 0x90, 0x36, + 0x37, 0xB7, 0x49, 0x5E, 0x49, 0x4E, 0xE4, 0x1E, + 0x24, 0x39, 0x48, 0xC7, 0x68, 0x19, 0xED, 0x94, + 0xB5, 0xF6, 0x85, 0xFA, 0xE6, 0xB8, 0x2B, 0x9E, + 0x07, 0x9E, 0xFB, 0x1D, 0x61, 0x06, 0x47, 0x2A, + 0x3E, 0x1E, 0xD8, 0x52, 0xFB, 0xB7, 0xE3, 0xB4, + 0x0D, 0xA8, 0x15, 0x1E, 0x98, 0x02, 0xBD, 0x89, + 0x89, 0xE3, 0x38, 0x2C, 0xAB, 0x50, 0x25, 0x30, + 0xB4, 0x5E, 0xA5, 0xCD, 0xA8, 0x9B, 0xA4, 0x2A, + 0xED, 0x19, 0x3B, 0xC3, 0x05, 0x07, 0x57, 0xB5, + 0x52, 0x11, 0x74, 0x95, 0x51, 0x5A, 0xD8, 0xED, + 0xDF, 0x37, 0x91, 0x83, 0x27, 0xD5, 0x08, 0x82, + 0xB3, 0x42, 0x08, 0xC7, 0x81, 0x35, 0x5F, 0x58, + 0x28, 0x69, 0x0D, 0x97, 0x1D, 0x28, 0xE6, 0xB0, + 0x58, 0x93, 0xCE, 0x2A, 0xB4, 0x7D, 0x4B, 0x83, + 0x20, 0x1E, 0x08, 0xEF, 0x68, 0x51, 0xEB, 0xB4, + 0xFA, 0x78, 0xB5, 0xE5, 0x2D, 0x93, 0x07, 0x99, + 0xBB, 0xDD, 0x9A, 0x4E, 0xD6, 0xE7, 0x31, 0x9F, + 0x4D, 0xB4, 0x05, 0x45, 0x89, 0x59, 0x42, 0xFF, + 0x0C, 0xD4, 0xC2, 0xF3, 0xA0, 0xF7, 0xD3, 0xCE, + 0x26, 0x2F, 0x9B, 0xCB, 0x98, 0x24, 0xDE, 0x50, + 0xF6, 0x81, 0x24, 0xA3, 0xB4, 0x99, 0x94, 0xDB, + 0xF5, 0xF4, 0x74, 0xAC, 0xED, 0xF1, 0xD6, 0x2F, + 0x89, 0x48, 0x41, 0x24, 0xBB, 0xA7, 0x64, 0x48, + 0xAE, 0x36, 0xE3, 0xF3, 0xA8, 0x37, 0x4F, 0xE6, + 0x24, 0x92, 0xC4, 0x27, 0x86, 0x68, 0xC4, 0xA2, + 0x2F, 0xB2, 0xA8, 0xFF, 0x27, 0x5F, 0x59, 0xB9, + 0x86, 0x89, 0x5A, 0x09, 0xB8, 0xDA, 0xD6, 0xBA, + 0x26, 0x86, 0x51, 0xC2, 0xDF, 0xCF, 0x07, 0x49, + 0x1C, 0xB7, 0x1A, 0x80, 0xD4, 0x97, 0x13, 0xAA, + 0xE8, 0x38, 0xA7, 0x2C, 0xB7, 0xE2, 0xE0, 0x33, + 0xF9, 0xC3, 0x21, 0xF6, 0x2A, 0xAA, 0x1A, 0xF7, + 0x39, 0x1F, 0x2F, 0x7E, 0x38, 0xC8, 0x82, 0x3F, + 0xCC, 0xD8, 0xE2, 0xD9, 0x02, 0xE6, 0x8E, 0x31, + 0xA7, 0x37, 0xE8, 0x59, 0x3F, 0xE1, 0x60, 0x1C, + 0x74, 0x52, 0xE4, 0x49, 0xF7, 0x21, 0x60, 0x11, + 0x8F, 0xD0, 0xA1, 0x4B, 0xE3, 0xF1, 0x18, 0x3D, + 0x90, 0x89, 0x54, 0x27, 0xA0, 0xF9, 0x32, 0x09, + 0x3D, 0x9D, 0x9A, 0x09, 0x53, 0xC6, 0x7E, 0x95, + 0x85, 0x53, 0x98, 0x4C, 0x23, 0xEA, 0x54, 0xBD, + 0x6F, 0x50, 0xBC, 0x4C, 0xCF, 0x37, 0xC5, 0x7B, + 0x4B, 0xCE, 0x84, 0xAF, 0xE2, 0xE2, 0x55, 0x49, + 0xBC, 0xBF, 0x92, 0xCA, 0x1E, 0x5E, 0x10, 0xDF, + 0x60, 0x87, 0x09, 0xA1, 0x4C, 0x1D, 0x7E, 0x1E, + 0x59, 0xE9, 0xCF, 0xDA, 0x45, 0x3F, 0xE2, 0x0F, + 0x53, 0x8D, 0x8B, 0x79, 0xBD, 0xD8, 0xB0, 0xE3, + 0x5B, 0x7C, 0x55, 0x4A, 0x84, 0xF0, 0x1E, 0xF9, + 0xE8, 0xF3, 0x09, 0x4D, 0x0B, 0xD7, 0x77, 0xCC, + 0x3F, 0x70, 0x22, 0x7D, 0x17, 0x27, 0x48, 0x57, + 0xE2, 0x36, 0xA0, 0x84, 0x3B, 0xDE, 0x05, 0x34, + 0xEF, 0x55, 0x12, 0xF4, 0x9A, 0x99, 0x0D, 0x28, + 0x86, 0x28, 0x99, 0x6B, 0x22, 0xEE, 0x63, 0xF0, + 0x68, 0x9C, 0xE1, 0x70, 0xF6, 0x26, 0xD8, 0x3B, + 0xF9, 0x57, 0x18, 0x3D, 0xAD, 0x66, 0xF0, 0xCF, + 0x7B, 0x0C, 0x28, 0x4D, 0xB8, 0xEB, 0x7B, 0x04, + 0x1E, 0x7D, 0x40, 0x5F, 0x5A, 0x1E, 0x7E, 0x08, + 0x7F, 0x4C, 0x1E, 0x18, 0xE5, 0x3E, 0x6C, 0x90, + 0x3C, 0x89, 0x13, 0x2A, 0xC4, 0x2A, 0x94, 0xB5, + 0x3E, 0x18, 0x1C, 0x4C, 0xBA, 0xEA, 0x86, 0xD2, + 0x05, 0xA9, 0x59, 0x9C, 0x80, 0xC2, 0x45, 0xAD, + 0x30, 0x99, 0x18, 0x6A, 0x2F, 0x73, 0x8C, 0xF0, + 0xFE, 0xA4, 0xBD, 0x44, 0x3E, 0xEB, 0x98, 0x75, + 0x48, 0x08, 0x57, 0x45, 0xD8, 0x41, 0xDE, 0x61, + 0x6D, 0x06, 0x93, 0xC4, 0x99, 0x1B, 0x23, 0xB5, + 0x12, 0x22, 0x5C, 0xC7, 0x9E, 0x18, 0xEA, 0x64, + 0xDB, 0xCE, 0x1A, 0xAC, 0x5D, 0x9B, 0x80, 0xE2, + 0xBF, 0x3E, 0xC2, 0xA4, 0x78, 0x4F, 0xF1, 0xE3, + 0x7D, 0x2A, 0x20, 0x94, 0x13, 0xCD, 0xF3, 0x1C, + 0x33, 0x9C, 0xC1, 0x59, 0x85, 0x52, 0xCB, 0xDB, + 0x03, 0xDF, 0x11, 0xE6, 0xAB, 0x95, 0x82, 0x65, + 0x7A, 0x88, 0x73, 0xEE, 0xBA, 0x21, 0x1C, 0x2F, + 0xCD, 0xD7, 0xC6, 0xE5, 0x13, 0xDE, 0x7A, 0x9E, + 0xEE, 0x83, 0x8D, 0xC6, 0x47, 0x63, 0xE0, 0xC7, + 0xC4, 0xBE, 0x19, 0x25, 0xEE, 0xCC, 0x0A, 0x13, + 0x18, 0x9D, 0x34, 0x5B, 0x55, 0x6C, 0xC1, 0x6E, + 0xBF, 0x5A, 0xC5, 0x61, 0x75, 0x77, 0x49, 0x8C, + 0x67, 0x61, 0xE8, 0x72, 0x87, 0xE8, 0xCA, 0xBE, + 0x6E, 0xC3, 0xD7, 0x81, 0x8C, 0x78, 0x79, 0xC8, + 0x72, 0xDA, 0x1A, 0x40, 0x7D, 0x60, 0xE2, 0x5A, + 0x47, 0x38, 0xA1, 0x21, 0x22, 0x6B, 0x54, 0x74, + 0xDD, 0xF3, 0xBC, 0x96, 0x28, 0x7E, 0xC2, 0x8B, + 0x13, 0xEE, 0x53, 0xBC, 0x34, 0x67, 0x07, 0x87, + 0xD5, 0x6B, 0x93, 0x22, 0x21, 0xB9, 0xED, 0x17, + 0xE4, 0xA1, 0x96, 0xB2, 0xC6, 0xFF, 0x79, 0xA0, + 0xA7, 0xF9, 0xDD, 0x92, 0x78, 0xF7, 0xE3, 0x16, + 0x79, 0xEF, 0xEF, 0x31, 0x4D, 0x1D, 0x75, 0xF9, + 0xCF, 0x5A, 0x1B, 0x68, 0x16, 0x7F, 0xAF, 0x5F, + 0x30, 0xB7, 0xEF, 0xF8, 0x94, 0x63, 0x73, 0x3D, + 0xB3, 0x63, 0xE4, 0xE6, 0xD8, 0xAD, 0xF4, 0x80, + 0x5E, 0x82, 0xA3, 0xFB, 0x3D, 0x0F, 0xCF, 0x59, + 0xB8, 0x76, 0xAF, 0x27, 0x83, 0xE3, 0x2D, 0x6F, + 0xE0, 0xF3, 0x11, 0xD5, 0xAE, 0x82, 0x14, 0x1D, + 0x78, 0x95, 0xBF, 0x31, 0x22, 0x1B, 0x80, 0x12, + 0x02, 0xD7, 0x4D, 0x1B, 0x92, 0xE3, 0x15, 0xBA, + 0x67, 0xD6, 0x8F, 0xD4, 0xDA, 0xBF, 0xD4, 0x62, + 0xAD, 0x76, 0xDA, 0x04, 0xA2, 0xEA, 0x98, 0xD3, + 0xC3, 0x6E, 0x5F, 0x26, 0x3C, 0x5E, 0xD9, 0xEA, + 0x09, 0xF0, 0x02, 0xFC, 0xD2, 0x11, 0xF8, 0xA8, + 0x7E, 0xFF, 0x06, 0x28, 0x5B, 0xE5, 0x6F, 0x9A, + 0x00, 0xE7, 0x7F, 0xB9, 0xFB, 0x59, 0xBB, 0xDD, + 0x85, 0xF3, 0x40, 0xCE, 0xA3, 0x5E, 0x2E, 0x2E, + 0x34, 0x91, 0x58, 0x41, 0x00, 0xB5, 0xE1, 0x88, + 0x24, 0x51, 0xC7, 0xB6, 0xF3, 0x21, 0x52, 0x6E, + 0xE7, 0xFC, 0x42, 0xE7, 0x9C, 0xCC, 0x1E, 0x51, + 0x45, 0x39, 0xBE, 0x09, 0xFE, 0x1A, 0xC4, 0xF0, + 0x79, 0xF4, 0x05, 0xC7, 0xA7, 0xF8, 0x0F, 0xB6, + 0x5A, 0x7B, 0xD7, 0xE1, 0x6F, 0xF0, 0x9D, 0x67, + 0xA3, 0xE3, 0x3E, 0x2E, 0xB9, 0x8C, 0x83, 0x9E, + 0xFD, 0x2E, 0xA2, 0x92, 0x99, 0x3C, 0xC0, 0x99, + 0x01, 0xAB, 0x0D, 0xFA, 0x55, 0x96, 0x04, 0x60, + 0x1A, 0xAD, 0x4C, 0xBB, 0x3D, 0xBB, 0x7D, 0x8B, + 0x9F, 0x28, 0x85, 0x7D, 0xB9, 0xE4, 0x05, 0x79, + 0x7B, 0x63, 0xDD, 0x7F, 0x4D, 0xE7, 0x50, 0xD9, + 0x41, 0xFF, 0x53, 0xB1, 0xCE, 0x42, 0x7B, 0xD6, + 0x05, 0x1B, 0x4E, 0xAF, 0xC4, 0x8C, 0x17, 0xC8, + 0x52, 0xBD, 0x03, 0x3B, 0x92, 0x57, 0x4E, 0xA8, + 0x15, 0xC3, 0x26, 0x1C, 0x55, 0xC1, 0xFF, 0xAE, + 0xA9, 0x26, 0x2D, 0xA7, 0x8E, 0x3A, 0x7F, 0xA3, + 0x48, 0xA5, 0xBC, 0x14, 0x84, 0xF2, 0x90, 0xCE, + 0x35, 0x0F, 0x64, 0x6B, 0xD8, 0x1C, 0x12, 0xFE, + 0x5A, 0x4F, 0x0E, 0xCE, 0x81, 0x4E, 0x79, 0x6B, + 0xCF, 0x56, 0xA7, 0xDB, 0x24, 0xBC, 0xB0, 0x84, + 0x4C, 0xB0, 0xDA, 0xBE, 0xE6, 0x8F, 0xD7, 0x8E, + 0x0E, 0xA0, 0xD3, 0x55, 0xC2, 0x4A, 0x34, 0x1C, + 0xF9, 0xC7, 0x3D, 0x29, 0x70, 0x8B, 0xF0, 0x99, + 0x61, 0xF5, 0x11, 0xFB, 0x82, 0xE2, 0x67, 0x35, + 0x60, 0x78, 0x47, 0x81, 0x2A, 0x74, 0x5E, 0x4D, + 0x48, 0xD3, 0x7C, 0x32, 0xCA, 0x1B, 0xD2, 0xA2, + 0x5C, 0x3A, 0x2F, 0xCE, 0xB4, 0x6C, 0x3A, 0x6A, + 0x8F, 0x67, 0x46, 0x12, 0xE7, 0xAE, 0x6A, 0x3B, + 0x99, 0x04, 0x5E, 0x96, 0xD0, 0xB9, 0x84, 0xF6, + 0xA7, 0x64, 0x11, 0xE8, 0x0C, 0x51, 0xFD, 0x3F, + 0x18, 0xFA, 0xE8, 0x52, 0xD9, 0x4B, 0x99, 0x7A, + 0x25, 0x2B, 0x1B, 0x21, 0xAD, 0x8C, 0xFE, 0x0D, + 0x34, 0x51, 0x91, 0x75, 0x55, 0x6F, 0xEB, 0x9F, + 0x42, 0xDC, 0x73, 0x7D, 0x31, 0x0A, 0x74, 0x13, + 0x80, 0xB8, 0xC3, 0xED, 0x73, 0x9D, 0x79, 0x42, + 0xC0, 0x33, 0xAB, 0xC1, 0xCB, 0xB9, 0xD0, 0xBE, + 0xA0, 0x78, 0xB8, 0x3B, 0xEB, 0x3D, 0x1A, 0x3F, + 0xFB, 0x9B, 0xAA, 0x8F, 0x89, 0xF9, 0xD1, 0x22, + 0x82, 0xE6, 0x66, 0xEE, 0x2A, 0xFD, 0x9F, 0xF8, + 0x92, 0x7E, 0x10, 0xF5, 0xD5, 0x23, 0x0B, 0xB1, + 0xD6, 0xF1, 0x7A, 0x3D, 0x73, 0xE9, 0xCE, 0x7F, + 0xE6, 0x0B, 0x17, 0xBC, 0x23, 0xAE, 0x72, 0xB6, + 0xFA, 0x19, 0x46, 0xBB, 0xFE, 0xA4, 0xC1, 0x64, + 0xA8, 0x5E, 0xE8, 0xBB, 0x63, 0x58, 0x19, 0x50, + 0xAA, 0x36, 0xC2, 0x4B, 0x38, 0x24, 0xD1, 0x2E, + 0xAE, 0xAD, 0x6E, 0x34, 0x64, 0xA8, 0xC8, 0xF2, + 0x4E, 0x74, 0x5C, 0x98, 0xE8, 0xDF, 0x99, 0x8C, + 0x41, 0x79, 0x60, 0x2D, 0xD5, 0xF4, 0xE3, 0xE9, + 0x1D, 0xF6, 0x5A, 0xA9, 0x69, 0x8E, 0xA1, 0x4F, + 0xD3, 0x1B, 0x09, 0xA8, 0x7A, 0xD1, 0xE1, 0xCF, + 0xAC, 0xBA, 0xD0, 0xD1, 0x34, 0x34, 0x8F, 0xC1, + 0xD1, 0xA8, 0xAF, 0x6E, 0x92, 0xE0, 0xB0, 0xF6, + 0xF9, 0x08, 0xA0, 0xCB, 0x58, 0x34, 0xF6, 0x68, + 0xA2, 0xBF, 0x05, 0x39, 0x63, 0xBA, 0x4F, 0xEF, + 0xE3, 0x95, 0x69, 0xD5, 0x89, 0x7C, 0x64, 0x07, + 0x13, 0x42, 0x14, 0xF1, 0xA6, 0x9B, 0x87, 0xE5, + 0xF4, 0x49, 0xAE, 0x67, 0x65, 0xCC, 0xF2, 0x26, + 0xF8, 0x31, 0xBD, 0x33, 0x6A, 0x87, 0x77, 0x4E, + 0xB1, 0xEE, 0xA4, 0xA2, 0xC8, 0xA0, 0x4A, 0xC1, + 0xDF, 0x55, 0xE0, 0xDE, 0x53, 0x15, 0x3B, 0xEC, + 0x55, 0x32, 0xCA, 0x06, 0xE4, 0x78, 0x59, 0x63, + 0x10, 0x68, 0xA9, 0x46, 0x1B, 0xEF, 0x73, 0x6D, + 0x1A, 0x02, 0x64, 0x12, 0x76, 0x9B, 0xDB, 0x7C, + 0x03, 0x35, 0x19, 0xE1, 0x58, 0x7A, 0x87, 0x0C, + 0x76, 0xDC, 0xFC, 0xC0, 0x28, 0xE4, 0xA2, 0x07, + 0x9C, 0x28, 0x05, 0x21, 0x13, 0x58, 0xEF, 0x05, + 0xBB, 0xAB, 0x94, 0xA2, 0x93, 0xBC, 0x31, 0x61, + 0x26, 0x39, 0x38, 0x0C, 0xC4, 0x67, 0xDA, 0xA5, + 0xE4, 0x1E, 0x1B, 0xB6, 0xE5, 0x73, 0xD6, 0x6C, + 0xEE, 0xBC, 0x9D, 0xB9, 0xE7, 0xD9, 0x45, 0x2F, + 0xF2, 0xB6, 0x92, 0x54, 0x41, 0x05, 0xB7, 0xB7, + 0xFC, 0x37, 0x63, 0x6A, 0xB4, 0xBE, 0xB8, 0x3E, + 0xD8, 0x53, 0x3B, 0xF8, 0x7D, 0x9A, 0x05, 0xDF, + 0x20, 0x02, 0x27, 0x64, 0x38, 0xFA, 0x7D, 0xAF, + 0x7F, 0xFA, 0xD1, 0xB7, 0x32, 0xC5, 0x74, 0x3E, + 0x04, 0xA2, 0x67, 0x79, 0x02, 0x2E, 0x6F, 0xA1, + 0x27, 0x87, 0x07, 0xB5, 0x9F, 0x0A, 0x7D, 0x5E, + 0x14, 0xA0, 0x31, 0x46, 0x3F, 0xA9, 0xDE, 0x98, + 0xB9, 0x89, 0xA0, 0x4A, 0x7A, 0xBD, 0x15, 0xAE, + 0x2D, 0x0B, 0x38, 0x9A, 0xD8, 0x0E, 0xD2, 0xBA, + 0x6D, 0xA1, 0x04, 0x1E, 0x4E, 0x39, 0x87, 0x4B, + 0xC8, 0x3C, 0x74, 0x35, 0x4D, 0xC8, 0x1B, 0x42, + 0x06, 0x5B, 0x73, 0xB7, 0x33, 0x86, 0x4A, 0x10, + 0x2A, 0x10, 0x16, 0x28, 0x6F, 0x2A, 0xE3, 0x86, + 0xDE, 0xA3, 0x44, 0x23, 0xE2, 0x90, 0xC4, 0x20, + 0x90, 0xE0, 0xB8, 0xE6, 0xA7, 0xB6, 0xD6, 0x92, + 0xF4, 0xF8, 0x8A, 0xBC, 0xAC, 0x31, 0x47, 0x8F, + 0xAA, 0xE0, 0xD9, 0xF7, 0xE3, 0xCB, 0x11, 0xA4, + 0x6B, 0x05, 0xB3, 0xB8, 0x72, 0x69, 0xE6, 0xDD, + 0x75, 0x0F, 0x20, 0x1D, 0x3F, 0xC6, 0x96, 0xA0, + 0x18, 0xB6, 0x24, 0xA1, 0xA6, 0xFD, 0x0C, 0x80, + 0x1E, 0xD2, 0x28, 0xA2, 0x1A, 0x27, 0xF4, 0x23, + 0x59, 0x1A, 0xCC, 0x0F, 0xD4, 0x99, 0xD0, 0xB4, + 0x1E, 0x91, 0xC7, 0xD8, 0x8F, 0x8C, 0x5B, 0xEB, + 0xB5, 0x9F, 0xFF, 0x4F, 0xD0, 0xD5, 0xB7, 0x60, + 0xCC, 0x0A, 0x10, 0x38, 0xBF, 0xA8, 0x2E, 0xCC, + 0xEB, 0x26, 0xB0, 0x78, 0xB3, 0xE0, 0x40, 0xAF, + 0xCD, 0x12, 0xC5, 0x3A, 0x24, 0xD8, 0xEE, 0x3A, + 0x64, 0x83, 0x2E, 0xD9, 0x25, 0x21, 0x66, 0xA5, + 0x28, 0xD1, 0xE1, 0x84, 0x25, 0x1B, 0x20, 0xB8, + 0xF5, 0x76, 0xB6, 0x3E, 0x4B, 0xC6, 0xEC, 0xC1, + 0xC7, 0xAC, 0xC4, 0xAD, 0xCE, 0xF0, 0xB4, 0x0F, + 0x35, 0x1E, 0xCE, 0x4E, 0xE3, 0x57, 0x30, 0xFC, + 0xF4, 0x9B, 0x86, 0xB0, 0xDD, 0x3F, 0x2F, 0xB6, + 0x10, 0x20, 0xE4, 0x24, 0x17, 0x1C, 0x24, 0xC6, + 0x89, 0xE4, 0x14, 0xAD, 0x2E, 0x41, 0x08, 0x33, + 0x88, 0xB1, 0x6F, 0x11, 0x85, 0xAF, 0x58, 0x17, + 0xE3, 0x91, 0xB4, 0x72, 0xA2, 0x7F, 0xA3, 0x98, + 0xAF, 0xB7, 0x6B, 0x58, 0x76, 0xA3, 0x11, 0x1C, + 0x8A, 0x1A, 0xE6, 0x58, 0x54, 0xB0, 0xB9, 0x6E, + 0x46, 0xCB, 0x16, 0xC0, 0x63, 0x0C, 0xEE, 0xA2, + 0xAE, 0xF6, 0x71, 0xEF, 0xD1, 0xB9, 0x3D, 0xB7, + 0x76, 0xCE, 0x5B, 0x84, 0x66, 0x7C, 0x7D, 0xF1, + 0x96, 0x60, 0x34, 0xF6, 0xD1, 0x64, 0x27, 0xD9, + 0xF3, 0x78, 0x8B, 0xF4, 0xC3, 0x1D, 0x37, 0xC0, + 0xF4, 0x4A, 0xD0, 0xA5, 0x9A, 0xEB, 0xDD, 0x79, + 0x54, 0x5D, 0xEB, 0x04, 0xC1, 0xA4, 0xBC, 0xED, + 0xE3, 0x74, 0xC3, 0xB9, 0x9A, 0x6A, 0xAA, 0x06, + 0xD1, 0xF0, 0x0F, 0xC5, 0xEF, 0x7E, 0x0B, 0xC8, + 0xF4, 0x94, 0x4E, 0x69, 0x0E, 0x36, 0x00, 0x13, + 0x45, 0xCE, 0x68, 0x13, 0xFE, 0x7F, 0x29, 0xA2, + 0x1D, 0x79, 0xDF, 0xF2, 0x27, 0xFB, 0xAE, 0x52, + 0x05, 0x78, 0xD7, 0xB9, 0xF7, 0x38, 0x68, 0xD5, + 0xBA, 0xD7, 0xCF, 0x09, 0xC6, 0xD2, 0x5B, 0xC6, + 0x98, 0xE4, 0xEC, 0xD5, 0xE9, 0xC2, 0xA5, 0x1A, + 0x52, 0xC8, 0xA7, 0xBA, 0x3D, 0x74, 0x75, 0x00, + 0xAA, 0xDD, 0x6A, 0x3F, 0xB6, 0x2F, 0x08, 0xB7, + 0x1C, 0x6B, 0x52, 0x0C, 0xC9, 0xE4, 0x4D, 0xF4, + 0xC5, 0x26, 0x1F, 0x35, 0x41, 0x25, 0x68, 0x17, + 0xA8, 0x81, 0x75, 0xF4, 0x66, 0x41, 0xB5, 0xE4, + 0x1D, 0x92, 0xEE, 0xDA, 0x0F, 0x56, 0x76, 0xC6, + 0xAA, 0x0F, 0xA8, 0x63, 0x8D, 0xF0, 0x69, 0x63, + 0x93, 0x45, 0xBC, 0x76, 0x40, 0xBE, 0xA9, 0x96, + 0x36, 0xAF, 0x2F, 0x6B, 0x3E, 0xAB, 0xF3, 0xC0, + 0xD7, 0xD5, 0xB1, 0x23, 0x23, 0xA2, 0xA0, 0xC4, + 0xC5, 0x70, 0xEF, 0x66, 0x79, 0x15, 0xF0, 0xD0, + 0x40, 0x0A, 0x33, 0x0C, 0xF3, 0x32, 0x6D, 0x8D, + 0xB4, 0x44, 0x46, 0x78, 0x3F, 0x8D, 0x75, 0x40, + 0xA5, 0x60, 0xBC, 0x9B, 0x76, 0xDF, 0x25, 0xF4, + 0xE9, 0xED, 0xAC, 0x74, 0x2F, 0x9A, 0x00, 0xC4, + 0x2B, 0x52, 0x26, 0x79, 0x09, 0x19, 0x57, 0x89, + 0x60, 0x14, 0xBE, 0x65, 0xBD, 0x7B, 0x4D, 0x7D, + 0x9B, 0x8B, 0x9E, 0x72, 0x6C, 0x0C, 0x57, 0xC7, + 0x00, 0x08, 0x38, 0x7C, 0x37, 0x45, 0x9D, 0x55, + 0xA2, 0x62, 0x5E, 0x34, 0x19, 0x99, 0x31, 0x16, + 0xF1, 0x14, 0x44, 0x2D, 0xE3, 0x7E, 0x22, 0xE1, + 0xA2, 0xB8, 0x9A, 0x9F, 0xE0, 0x37, 0x29, 0xBB, + 0xCD, 0x46, 0xEE, 0x0A, 0x62, 0x2B, 0x98, 0x34, + 0xBA, 0x9E, 0x54, 0x1B, 0xB1, 0x5C, 0x4F, 0xE9, + 0xAA, 0xE4, 0x95, 0x8C, 0xA4, 0xEF, 0xC2, 0xB1, + 0x7F, 0xF9, 0x80, 0xDA, 0x55, 0x95, 0x92, 0xC0, + 0x86, 0xF4, 0x2D, 0x99, 0x3E, 0x17, 0xDC, 0x55, + 0xA6, 0x33, 0x85, 0x90, 0x31, 0xC8, 0xFF, 0x58, + 0x83, 0xC5, 0xBA, 0x60, 0x20, 0x5F, 0x87, 0x29, + 0x20, 0x5A, 0x7D, 0x44, 0x2B, 0xA0, 0xE2, 0x99, + 0xC8, 0x70, 0xBE, 0x89, 0xC5, 0xBC, 0xF6, 0x0D, + 0x04, 0xC0, 0x96, 0xD1, 0x5C, 0xD1, 0x90, 0x43, + 0xD3, 0x7B, 0x73, 0x52, 0x30, 0xB6, 0xA9, 0x7C, + 0x0A, 0xA3, 0x24, 0x0E, 0x80, 0xFE, 0xBE, 0x31, + 0xFD, 0xB5, 0x96, 0x04, 0x2B, 0xCF, 0x0B, 0x28, + 0x1F, 0x7A, 0xCF, 0xC4, 0x82, 0x78, 0x52, 0x30, + 0xB1, 0x34, 0x12, 0x50, 0x03, 0x09, 0x1C, 0x8B, + 0x80, 0x60, 0xE3, 0xA1, 0xE5, 0x61, 0xF7, 0xD7, + 0xB6, 0x74, 0xBE, 0xD6, 0x58, 0x03, 0xD3, 0xE5, + 0xF7, 0xAC, 0x07, 0x60, 0xB7, 0x8A, 0xEC, 0xFA, + 0xC6, 0x0F, 0xF0, 0x20, 0x04, 0x6B, 0x8F, 0x61, + 0x09, 0x92, 0x03, 0xFB, 0x85, 0x99, 0x94, 0x9D, + 0x2E, 0x6A, 0xC2, 0x9F, 0x20, 0x46, 0x2A, 0x96, + 0xED, 0x42, 0x7D, 0x64, 0xA9, 0xE4, 0x1B, 0xDE, + 0x11, 0x20, 0x12, 0x93, 0xE6, 0x2B, 0xE5, 0x93, + 0x48, 0x37, 0x8C, 0x5A, 0x54, 0x0D, 0xEB, 0xF0, + 0x9F, 0x9D, 0xE4, 0xA5, 0xC4, 0x93, 0x6F, 0x6A, + 0xE3, 0x99, 0x69, 0xD9, 0xFE, 0x0C, 0x4E, 0xEC, + 0x8B, 0x30, 0x1F, 0x7A, 0xB8, 0xC8, 0x5B, 0x61, + 0x8E, 0xC2, 0x10, 0x90, 0x57, 0xB4, 0x72, 0x58, + 0x7F, 0x41, 0x29, 0x7E, 0xF9, 0xBE, 0x40, 0xC3, + 0x6F, 0xA9, 0xE3, 0x00, 0xE9, 0xC8, 0xFD, 0x4B, + 0xFD, 0x3F, 0xE3, 0x3F, 0x25, 0x22, 0xFD, 0xB7, + 0x2D, 0x57, 0xEF, 0x91, 0x08, 0xF0, 0x20, 0x56, + 0x30, 0xFA, 0x83, 0x69, 0xFD, 0x56, 0x5A, 0x9B, + 0xCE, 0xF8, 0x28, 0x02, 0xB4, 0x91, 0x35, 0x75, + 0x9E, 0x63, 0x99, 0x48, 0xCF, 0x35, 0xF5, 0x58, + 0x0C, 0x48, 0x8F, 0x0A, 0x2D, 0x9A, 0xE6, 0x40, + 0xF6, 0x21, 0xB5, 0x69, 0xC1, 0x09, 0x31, 0x00, + 0xA3, 0xC1, 0x4C, 0x99, 0x70, 0x4F, 0x5A, 0x63, + 0x17, 0x90, 0xB8, 0xF8, 0x3A, 0x0E, 0xFD, 0x67, + 0xEA, 0x0E, 0xBA, 0x7B, 0x1E, 0xEF, 0x37, 0x84, + 0xD5, 0x51, 0x37, 0x01, 0xD6, 0x93, 0x15, 0xDF, + 0x56, 0x89, 0x0E, 0x54, 0xF5, 0x1D, 0xF7, 0xE5, + 0xB7, 0xC1, 0xF1, 0xC2, 0xD9, 0x14, 0x6F, 0x40, + 0x55, 0x67, 0x50, 0x7C, 0x58, 0x35, 0x8B, 0x39, + 0xCB, 0xB5, 0x87, 0xF5, 0x55, 0x5E, 0x26, 0x8C, + 0x5B, 0x73, 0x0E, 0xBB, 0x25, 0x51, 0x0E, 0xAD, + 0x57, 0x72, 0x7B, 0x68, 0x83, 0x11, 0x1E, 0x3A, + 0x3D, 0xA4, 0x7C, 0x18, 0xB7, 0x70, 0x18, 0xBC, + 0x72, 0x03, 0x4A, 0xA1, 0xD2, 0xF9, 0xA9, 0x8A, + 0x25, 0x45, 0x19, 0xEE, 0x93, 0x06, 0xB5, 0x09, + 0x71, 0xC9, 0x2D, 0xFD, 0x2B, 0xF3, 0xC7, 0x64, + 0x5F, 0xCE, 0x71, 0x1D, 0x81, 0x96, 0x67, 0xBF, + 0x01, 0x39, 0x8C, 0xE7, 0xA2, 0xD0, 0x98, 0x57, + 0x5A, 0xFD, 0x21, 0xC7, 0x46, 0xAA, 0xB4, 0xE4, + 0x0E, 0xBE, 0xC6, 0x68, 0x3E, 0x38, 0xF5, 0xA2, + 0xED, 0x73, 0xCC, 0x53, 0x7E, 0x7E, 0x03, 0x32, + 0xDC, 0xB6, 0xC1, 0x03, 0x9E, 0xB3, 0x2A, 0xAD, + 0xC0, 0xC3, 0x6E, 0x47, 0xFB, 0x1E, 0xB7, 0x0D, + 0x86, 0x95, 0x09, 0xA6, 0x9D, 0x6F, 0x92, 0xFC, + 0xFF, 0x2C, 0x7D, 0x09, 0x16, 0x68, 0x50, 0x3E, + 0x4F, 0x23, 0x4C, 0x93, 0x95, 0x2A, 0xE1, 0x9B, + 0x16, 0xF0, 0x0F, 0xFF, 0x79, 0xA8, 0x06, 0xF9, + 0x70, 0x61, 0x72, 0x2C, 0xE8, 0x91, 0x01, 0x6D, + 0x45, 0xE5, 0x82, 0x5D, 0x26, 0x21, 0xAD, 0x3D, + 0x77, 0x73, 0x23, 0x04, 0x84, 0x27, 0xA3, 0x5D, + 0x6D, 0xA8, 0x99, 0xC1, 0xCE, 0x4F, 0xA9, 0xF7, + 0xAB, 0x5C, 0xDE, 0x01, 0xE6, 0x1E, 0xEF, 0xE6, + 0xFD, 0xE0, 0x68, 0x85, 0x3E, 0xEE, 0xBF, 0xF1, + 0x0D, 0x79, 0xF4, 0xA2, 0xB4, 0x14, 0xBC, 0x0C, + 0x49, 0x77, 0x03, 0x71, 0x08, 0x3E, 0x40, 0xA6, + 0xD7, 0x03, 0xFA, 0xE2, 0xFB, 0xC7, 0x59, 0x30, + 0x6E, 0x07, 0x06, 0x1C, 0x7C, 0x47, 0xE5, 0x4C, + 0x57, 0x0A, 0x91, 0x4A, 0x43, 0xE4, 0x8A, 0xCD, + 0x6E, 0x92, 0x01, 0xE2, 0x52, 0xC1, 0x92, 0x34, + 0x8E, 0x64, 0x0F, 0x39, 0x63, 0x53, 0xAB, 0xE5, + 0x44, 0xD5, 0xAA, 0xAA, 0xF6, 0x03, 0x89, 0xB9, + 0xDD, 0xB2, 0x2D, 0x56, 0x1A, 0xE0, 0x72, 0x5A, + 0x52, 0x19, 0x46, 0xEA, 0xB3, 0xCE, 0xB3, 0x59, + 0x46, 0x7A, 0xA7, 0x48, 0x37, 0x0C, 0x09, 0xBA, + 0x92, 0x70, 0x17, 0x7F, 0xF5, 0xD3, 0x60, 0x44, + 0xC4, 0xC6, 0xC6, 0x7D, 0xD2, 0x21, 0xAC, 0x3F, + 0x62, 0x6C, 0xE9, 0xBA, 0x4C, 0xF3, 0x82, 0x7E, + 0x6D, 0x3A, 0x92, 0xDC, 0x94, 0xE4, 0x5F, 0xA6, + 0x8B, 0x66, 0xA0, 0xDD, 0xE2, 0x97, 0x83, 0xED, + 0xF5, 0x9D, 0xDF, 0x74, 0x77, 0x23, 0x7D, 0xDA, + 0xC4, 0xFB, 0x92, 0x1A, 0xD9, 0x37, 0x36, 0xD2, + 0x88, 0xC9, 0xEA, 0x0F, 0x98, 0xBD, 0xC5, 0xF8, + 0xAA, 0x19, 0x75, 0x12, 0x6A, 0x41, 0xB5, 0xB3, + 0xB5, 0xA4, 0x96, 0xDC, 0x2B, 0x49, 0x86, 0x66, + 0x35, 0xD8, 0x4A, 0x62, 0xB4, 0xCB, 0x1E, 0x27, + 0xC1, 0xAD, 0x34, 0x0E, 0x26, 0x16, 0xF2, 0xC2, + 0x22, 0x52, 0x84, 0xD8, 0xD1, 0x32, 0xB8, 0x9C, + 0xFE, 0x64, 0x42, 0x9F, 0xE4, 0x69, 0xF0, 0xAE, + 0x3B, 0xD9, 0x2C, 0xA0, 0x14, 0xEB, 0x69, 0x74, + 0x7C, 0xE2, 0xA6, 0x60, 0xE1, 0x52, 0x1C, 0xCC, + 0xBF, 0xE6, 0xA1, 0x83, 0x20, 0x5D, 0x9E, 0xA3, + 0xFB, 0x84, 0x8B, 0x33, 0xE6, 0xC9, 0x32, 0x83, + 0xC0, 0x3F, 0x98, 0x1D, 0x6E, 0xC0, 0x50, 0x71, + 0x29, 0x60, 0x5F, 0x36, 0xB4, 0x68, 0x1D, 0xB9, + 0x76, 0x73, 0xC3, 0x80, 0xC5, 0xBC, 0x59, 0x7B, + 0x59, 0xB4, 0xE0, 0x6A, 0x80, 0xCD, 0x4D, 0x8C, + 0x9E, 0xE0, 0x0B, 0x45, 0x7D, 0x54, 0xD4, 0xC4, + 0x97, 0x6C, 0x54, 0xEF, 0x14, 0x64, 0xBD, 0x3B, + 0xD7, 0xEE, 0xF4, 0xD1, 0x41, 0x76, 0x3A, 0x24, + 0x7A, 0xC2, 0xCA, 0x68, 0x28, 0x53, 0x46, 0xF7, + 0x1B, 0xDA, 0x4B, 0x7A, 0x56, 0x75, 0x86, 0xFB, + 0x31, 0x2C, 0x27, 0xF9, 0x4D, 0x35, 0xA4, 0x82, + 0xE7, 0x2F, 0x41, 0xB4, 0xCA, 0xCE, 0x75, 0x94, + 0x08, 0x54, 0xE2, 0x9E, 0x99, 0xC9, 0x85, 0xDE, + 0x6F, 0x80, 0x95, 0x59, 0x3E, 0x54, 0x9F, 0x31, + 0xF8, 0xDE, 0xD0, 0xD7, 0xA6, 0xD4, 0xD3, 0xBB, + 0xD9, 0xC7, 0x55, 0xDD, 0xAE, 0xAD, 0x9E, 0x57, + 0x4A, 0x33, 0x5D, 0x7A, 0xA6, 0xA3, 0xCA, 0xF9, + 0x4C, 0x5B, 0x51, 0xCC, 0x22, 0xBB, 0x76, 0x44, + 0x17, 0xDE, 0x22, 0xA1, 0xDF, 0x80, 0x13, 0x7D, + 0xE5, 0x34, 0x7E, 0x75, 0x73, 0x10, 0x40, 0xFB, + 0x9A, 0x21, 0xCD, 0xD3, 0xD3, 0x84, 0xB6, 0x0C, + 0x31, 0x1E, 0xB5, 0x42, 0xF4, 0x34, 0x11, 0x7F, + 0x4A, 0x23, 0xA8, 0xA5, 0x8F, 0x20, 0xCD, 0xA9, + 0xF2, 0xE4, 0xEE, 0xFA, 0x57, 0xD1, 0x22, 0x1C, + 0xA5, 0xDC, 0x0B, 0x25, 0xFE, 0xC2, 0xA7, 0x7E, + 0x09, 0x2E, 0xDA, 0x40, 0x9F, 0x6C, 0xC8, 0x71, + 0x58, 0x91, 0x04, 0x25, 0xD0, 0x06, 0xEA, 0x1B, + 0xCD, 0x9D, 0x50, 0xD8, 0x40, 0x24, 0xAC, 0xC3, + 0xB4, 0x07, 0x6E, 0x76, 0xF4, 0x4C, 0xD8, 0x80, + 0xD0, 0x20, 0xF5, 0x15, 0x6A, 0x0A, 0x12, 0xF8, + 0x6B, 0x67, 0x77, 0x34, 0xAE, 0x60, 0x68, 0x13, + 0x5B, 0x8E, 0xFF, 0x5E, 0x7A, 0x77, 0x67, 0x0D, + 0xE6, 0x96, 0x43, 0x9F, 0x8F, 0x77, 0x5F, 0x97, + 0x23, 0x91, 0x33, 0x72, 0xC7, 0x8A, 0xC7, 0x80, + 0xCF, 0xE7, 0x71, 0x06, 0x25, 0xB7, 0x4B, 0x89, + 0x6A, 0x46, 0x67, 0x19, 0x49, 0x44, 0x03, 0x52, + 0x32, 0xB1, 0x8F, 0xE7, 0x9E, 0xDA, 0x03, 0x41, + 0xA3, 0xAC, 0xE5, 0xF3, 0x96, 0xE6, 0xAC, 0xFF, + 0xEC, 0x35, 0x4D, 0x83, 0xA9, 0xCE, 0x7C, 0x52, + 0xF2, 0x36, 0x97, 0xF0, 0x28, 0x36, 0x54, 0x59, + 0x96, 0xEA, 0xEE, 0xB2, 0xC1, 0xAB, 0xA4, 0x96, + 0x62, 0xD4, 0x3C, 0xF0, 0x1F, 0x2D, 0x65, 0x0E, + 0x46, 0x7E, 0x12, 0x31, 0x8F, 0xA7, 0x8D, 0x7A, + 0x4A, 0x41, 0x15, 0x57, 0x90, 0xF6, 0xF1, 0xE8, + 0xE8, 0xE3, 0x57, 0x7B, 0x55, 0x85, 0x95, 0x97, + 0xB3, 0x29, 0x3D, 0x02, 0x73, 0x1E, 0x87, 0x1F, + 0x01, 0x89, 0x06, 0x88, 0x9E, 0x8A, 0x2E, 0xE0, + 0x99, 0xFC, 0xF0, 0x48, 0x60, 0x37, 0x65, 0x25, + 0xDB, 0x89, 0xC9, 0x7A, 0x51, 0x7E, 0x35, 0x92, + 0x00, 0xC9, 0x61, 0x3F, 0x88, 0xE3, 0x20, 0x01, + 0x46, 0x5A, 0x2C, 0x37, 0x02, 0xC9, 0x15, 0x0F, + 0xB2, 0xEB, 0xC3, 0x55, 0x18, 0xF0, 0x15, 0x1A, + 0x08, 0x8E, 0xB8, 0x9D, 0x18, 0xE4, 0x9F, 0x34, + 0x10, 0x67, 0x68, 0x57, 0x60, 0x61, 0xEC, 0xD5, + 0xD9, 0xA8, 0x3A, 0xAB, 0xD6, 0xD2, 0x7A, 0x47, + 0x3D, 0xA4, 0x08, 0x7C, 0x3E, 0x4D, 0x76, 0x08, + 0x19, 0x22, 0xB2, 0x89, 0x57, 0x84, 0xC2, 0x98, + 0x72, 0xB8, 0x8B, 0xE0, 0x85, 0xA1, 0x3A, 0xC2, + 0xA0, 0x06, 0x43, 0x03, 0xCF, 0x4F, 0x27, 0x80, + 0x48, 0x9A, 0xBC, 0xB3, 0x3C, 0xC4, 0x5E, 0xAC, + 0x8B, 0x85, 0x6F, 0x21, 0xD6, 0xFE, 0x12, 0x90, + 0x53, 0x2F, 0x65, 0x32, 0x8D, 0x03, 0xFE, 0xFE, + 0x61, 0x04, 0x47, 0x24, 0x6A, 0xB5, 0x01, 0x98, + 0xB9, 0x27, 0x10, 0xE1, 0x32, 0x3D, 0x2A, 0xA0, + 0xC5, 0x70, 0xDE, 0x1E, 0x10, 0xD7, 0x01, 0x50, + 0x4F, 0x87, 0xCA, 0xD9, 0xBF, 0x12, 0xEA, 0x38, + 0x4C, 0x43, 0xD5, 0x5A, 0xEF, 0x3E, 0x21, 0x8E, + 0x59, 0x77, 0x23, 0xED, 0x51, 0x09, 0x99, 0x73, + 0xD2, 0x56, 0x04, 0xDD, 0x8F, 0x5F, 0xDF, 0x79, + 0xFF, 0x16, 0x8C, 0xB0, 0xBA, 0x8A, 0x1A, 0x56, + 0xAF, 0xCA, 0x19, 0xF2, 0x64, 0x1A, 0xF5, 0x1E, + 0xA7, 0xA7, 0x84, 0x3D, 0xAD, 0xC1, 0x0E, 0x22, + 0xA1, 0x45, 0xFC, 0xB4, 0x13, 0x91, 0x34, 0xB7, + 0x48, 0xEF, 0x9E, 0xD9, 0x0B, 0xE3, 0x82, 0x75, + 0x80, 0xC5, 0xD9, 0xA0, 0x77, 0xA3, 0xF9, 0xCC, + 0x67, 0xDD, 0xCB, 0x28, 0xC6, 0xE8, 0x2C, 0xB8, + 0xAC, 0x63, 0xBD, 0x3B, 0x28, 0x4A, 0xE9, 0x2D, + 0x29, 0x84, 0xD7, 0x4F, 0x61, 0x06, 0xE3, 0x37, + 0xC1, 0x58, 0x20, 0x5D, 0x0A, 0xE7, 0x45, 0x29, + 0x7D, 0xED, 0x0F, 0xCE, 0x00, 0x95, 0x2A, 0x62, + 0x38, 0xA8, 0x1A, 0x3A, 0x96, 0x0E, 0x56, 0xD9, + 0x18, 0xC2, 0x25, 0xA5, 0xAA, 0x27, 0x0A, 0x6E, + 0xDD, 0x1C, 0x35, 0x6C, 0xC1, 0x26, 0x90, 0xF6, + 0x43, 0x1B, 0x34, 0xDA, 0xE1, 0x5D, 0x09, 0x7C, + 0xBE, 0x0F, 0x40, 0xD3, 0x24, 0x82, 0x0B, 0xFF, + 0xE6, 0xB7, 0x10, 0xD6, 0x36, 0xD0, 0xE6, 0xC0, + 0xBE, 0x65, 0x4C, 0x83, 0xF1, 0xDA, 0xDE, 0xCE, + 0xE8, 0x5A, 0x80, 0x88, 0xFE, 0x9B, 0x79, 0x54, + 0xA3, 0xA4, 0x5A, 0x76, 0xD0, 0xE2, 0xCE, 0x92, + 0x53, 0x7D, 0x9C, 0xDA, 0xA1, 0xED, 0x9F, 0x56, + 0x05, 0x0A, 0xA4, 0x81, 0xC7, 0x82, 0x5B, 0xB8, + 0xC7, 0xA8, 0x95, 0x21, 0x99, 0x0B, 0x0F, 0xD2, + 0x66, 0x68, 0xC3, 0x07, 0x53, 0x93, 0x8C, 0x68, + 0x5A, 0xF5, 0x6F, 0x5E, 0x07, 0x68, 0x70, 0xF2, + 0x6A, 0x78, 0xA8, 0xDB, 0x24, 0x6F, 0xD9, 0x74, + 0x38, 0x29, 0xBF, 0x50, 0xCC, 0xC8, 0x34, 0x82, + 0x69, 0x1A, 0xF4, 0x1A, 0x95, 0xE1, 0x31, 0x39, + 0x00, 0xBE, 0xF6, 0x25, 0x19, 0x5A, 0xF6, 0xA7, + 0xB7, 0xC7, 0xAC, 0xFA, 0x52, 0x6D, 0xC7, 0xEA, + 0xFF, 0xC4, 0xB0, 0x84, 0x41, 0x2D, 0x6B, 0x22, + 0x57, 0x1F, 0x8A, 0x97, 0x71, 0xEF, 0x54, 0xA7, + 0xA9, 0xB5, 0xA0, 0x7A, 0xFF, 0x52, 0xBC, 0x78, + 0x56, 0x8C, 0x9C, 0x2A, 0xFB, 0x31, 0xA6, 0xC8, + 0x87, 0xC8, 0x82, 0x72, 0x00, 0x68, 0x27, 0xDE, + 0x54, 0x67, 0x4C, 0x36, 0x4E, 0xE2, 0x77, 0x7F, + 0xAD, 0x15, 0x1B, 0xC9, 0x07, 0xC4, 0x84, 0x50, + 0x84, 0x45, 0xB5, 0x1D, 0xCE, 0x3F, 0x7C, 0xDF, + 0x73, 0x88, 0x3E, 0x6D, 0xAC, 0x18, 0x67, 0x2C, + 0x1D, 0x31, 0xD5, 0x41, 0x6E, 0xFC, 0x3D, 0xFE, + 0x5C, 0x6D, 0x3B, 0xCB, 0x67, 0x05, 0x44, 0x8B, + 0x02, 0xB7, 0xF5, 0x05, 0xFD, 0x1D, 0x7E, 0x13, + 0x90, 0xCE, 0xED, 0xD2, 0xAB, 0x08, 0xFF, 0xC3, + 0x91, 0x2C, 0x79, 0x06, 0xDB, 0x54, 0xAB, 0xFF, + 0xF6, 0x9D, 0xBB, 0xDC, 0x3C, 0xCD, 0x03, 0xE7, + 0xD8, 0x1A, 0x4E, 0x7F, 0xCB, 0x1B, 0xA3, 0xCA, + 0xDC, 0x14, 0xC5, 0xFE, 0x46, 0x38, 0x07, 0x82, + 0x5E, 0x9A, 0x77, 0x9C, 0xB4, 0x44, 0x26, 0xBA, + 0xC2, 0x27, 0xD0, 0xF4, 0x75, 0x67, 0x4A, 0x15, + 0x2A, 0x55, 0x2A, 0x61, 0x87, 0x55, 0xA0, 0xFB, + 0xE0, 0x93, 0xA5, 0xD7, 0xDF, 0x0D, 0x97, 0xD3, + 0x93, 0x39, 0x0B, 0x5A, 0xC6, 0x86, 0x17, 0x7D, + 0x6A, 0xA4, 0x07, 0x9C, 0xB9, 0x0F, 0x42, 0x15, + 0x46, 0x71, 0x45, 0x70, 0x75, 0x4B, 0xD5, 0x80, + 0x64, 0x62, 0x92, 0x50, 0xBC, 0x10, 0xA1, 0x68, + 0x60, 0x5A, 0x6F, 0x31, 0x26, 0xCA, 0xB1, 0x48, + 0xFB, 0xC6, 0xF4, 0x3A, 0xCA, 0x52, 0x20, 0x2F, + 0x7D, 0xEC, 0x19, 0xF8, 0xAA, 0x27, 0xC2, 0x23, + 0xD0, 0x8F, 0x60, 0xFC, 0x7F, 0xA0, 0xCB, 0xDC, + 0xA2, 0xC6, 0xC4, 0xC5, 0x13, 0x00, 0xF3, 0x43, + 0x48, 0x6D, 0xFD, 0x7D, 0xB9, 0xA8, 0x14, 0xB0, + 0x0C, 0x72, 0x82, 0x2F, 0x99, 0x64, 0x41, 0x2B, + 0xB3, 0x34, 0x73, 0x73, 0xF4, 0x26, 0x1D, 0x06, + 0xDF, 0x6E, 0xF4, 0x20, 0x1E, 0x31, 0xE3, 0x8A, + 0x01, 0x6C, 0xDB, 0x3C, 0xE3, 0xC6, 0xC4, 0xC5, + 0xB8, 0x20, 0x51, 0xF1, 0xD6, 0xB0, 0x30, 0xB7, + 0x2D, 0xDA, 0x95, 0x01, 0x0D, 0xED, 0xEE, 0x6F, + 0x69, 0xFD, 0xCF, 0x9D, 0xDD, 0x05, 0xD6, 0xC0, + 0xFE, 0x11, 0x67, 0xAF, 0x53, 0x94, 0x60, 0xFC, + 0x56, 0xBA, 0x0C, 0x5F, 0xA7, 0x7E, 0xDA, 0x65 + }, + .len = 4096 << 3 + }, + .validCipherLenInBits = { + .len = 4096 << 3 + }, + .cipher = { + .len_bits = 4096, + .offset_bits = 0 + }, + .dataunit_len = 0 +}; + + + + +#endif /*TEST_CRYPTODEV_MLX5_TEST_VECTORS_H_*/ From patchwork Thu Jul 1 13:26:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95144 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 928D9A0A0C; Thu, 1 Jul 2021 15:28:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A9B2841339; Thu, 1 Jul 2021 15:27:27 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by mails.dpdk.org (Postfix) with ESMTP id D36954132C for ; Thu, 1 Jul 2021 15:27:24 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Gy8YGYiNKY/MW57TETShrz8ZIv4keitpBnnaxjfRmJfYAk8QY1jyujp2nKJc6qpb9jDkVSjiuOoZ+nCA/XTDH15U+lxRjvNzY2Ls08Pu3r5Jcotsi2w4dv+Bqek/3i4Zox92qzrLUo2O11n0twsKcEMdcnW5zuqgxPj6BWmg8VCZFo658yGo9SEN4/aGPNzHzWhptBbdEyyPyPVB/gnBdIOdITcApOCKMxaStlKhrkMhHzgqqDbGga30Av6YsosfS4zwwmYNAsVh3HK1VDZ+ISLUf9zh1AYX9uubbVIrw+S4xpMEruBG3D13ssfQAREyFhIHssLWp5+H/SwgwDqU4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I7vSkIeD6SePoUmOZZj2jwOrC8YCW1XCuSd+iJGGhb4=; b=Hiht5Zn6ISu9d1pZ+nErkTIBlNtr6knpk+2luUwDZD5gIN89W+jHYSXarHyEQPRaUlu78hTA3DDEbmrReYtNElEq1X8I+FW+Ta0lK8uUS5dTCvc4yJzBv+q1mzG26mdS/nDwkyRXTzOzLNvzc8qVL+90T50GCLlbeHsxTT6x2ybWDXCVT1tX/EU91LmO8CY9ISzHX79DdGOXabjHQBgXWjCM0oWPoVTx4DXxgJWU+CLqgRh6Dw3ka7mzHmpHHe4dZnEgIjMECEaEHEnm8qTqlIxZV65mqsh5u/gtufcnO9lS+IF6v7EH3LaZK/2mYuM9j/eR4nAq9pJ5FmJWuyRgzg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I7vSkIeD6SePoUmOZZj2jwOrC8YCW1XCuSd+iJGGhb4=; b=fpCs8UWKLeYDdyooJk5AiiDG1ES5JNDrEH0ddskse1kb9BBFFu5/5vJKvvulhzlJ2EzDQFzdXTt5mHf+tQVEC1Ap0nRaM0vkoSyo3YvrwFPsA44LZLXQoqB2sMUBYY47d1gNKKYfkUHKbGrN6F9sfqLXv4nj3IYcEZWEjLAup3AsrsOtT8dIYsGQoCx96w78eFJENgVqvulDYa59O7a/Lt3bZPHx+qBBHjU8mnJMYYE/HsbXyy2p72Wx4064aKkf7+cN1plTKsJtRZCErl9PPdWP2rDHJSuyk82vthYP9VS/Il2bkcv7FuX+837OmCvdC6J2Y7uu6pv7GAwu+owZdg== Received: from BN9PR03CA0477.namprd03.prod.outlook.com (2603:10b6:408:139::32) by CY4PR1201MB0103.namprd12.prod.outlook.com (2603:10b6:910:17::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.20; Thu, 1 Jul 2021 13:27:23 +0000 Received: from BN8NAM11FT055.eop-nam11.prod.protection.outlook.com (2603:10b6:408:139:cafe::84) by BN9PR03CA0477.outlook.office365.com (2603:10b6:408:139::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.21 via Frontend Transport; Thu, 1 Jul 2021 13:27:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT055.mail.protection.outlook.com (10.13.177.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 13:27:22 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:27:13 +0000 From: Shiri Kuzin To: CC: , , Date: Thu, 1 Jul 2021 16:26:09 +0300 Message-ID: <20210701132609.53727-16-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210701132609.53727-1-shirik@nvidia.com> References: <20210509160507.224644-1-matan@nvidia.com> <20210701132609.53727-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b939331b-2c17-47b2-e7b3-08d93c93f5a6 X-MS-TrafficTypeDiagnostic: CY4PR1201MB0103: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3276; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wy/Vcy8oPntxaZ8OshnnI5qKRDIhvpCVaD9/GgCx6qOAOaX1OXXNgs3hGfrcL79uM4Q1zgcyk4mRFen3BigD4R2iEc+qUrqiAggqH/pjkhO+BhbreT8A+P4yYyxw3uEBd3nlLC3vmIMviWn/h5NfFGNuyP89x9JwBETOxhza5To2zSVYm8RJcACDxv3UZWrE0Iqk+014cZMkz4Aolke0CwSorzp8zKdtfUfWY9wnHLiCWWYSea1rGugPKuEd8hUqReu7cRkW9xwu3Hr51E9QEgrfvTQPjax9QXYGP3UV57I/KCqVgyzZE/vTJYpQVv5hnj786+i13sovrWWmORE21WtgdpuCU+MhqdYirfK6ZIhtzXH7Rq4rA3uNvD+Gw2Y06c1Q+qImYg2mm+RV4PdwRRDZioMCFmHHXAKlc4EwzRKn7JH0ULQP/LIyHTOpL9vKlzkWjFui7aCTgFIu9XmfKYbL6vmYQFERgOiHA2m++8t0EOMaJge/TDucEE110lkae+g/YjIPHEVHa2y0Ka8BXk/GsCj6k4XjTalISWPlgKX5wMN7yaIf0pnyM8CBOqVORJVJEba1N1MwTtfnlKzqDVjP8ATsOCQUMkK/EeSnYawbhvsFnHWJFo0V411G+68xQI8RTFTMctKmrGHKeEcwDKy03tShqCyAVN6LBm+CsNYcD7uMbD9Vn9VwsGH2TNVbw7i5GfYauc5NjXY/tRG78Q== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(36840700001)(46966006)(82740400003)(6666004)(2906002)(47076005)(7636003)(70206006)(83380400001)(186003)(70586007)(16526019)(26005)(7696005)(8936002)(107886003)(2616005)(36756003)(6286002)(426003)(54906003)(316002)(336012)(4326008)(478600001)(30864003)(55016002)(36860700001)(1076003)(6916009)(5660300002)(356005)(82310400003)(86362001)(8676002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 13:27:22.6852 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b939331b-2c17-47b2-e7b3-08d93c93f5a6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT055.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1201MB0103 Subject: [dpdk-dev] [PATCH v5 15/15] test/crypto: add mlx5 multi segment tests X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The crypto mlx5 driver supports multi segment encryption and decryption operations. Added mlx5 multi segment encryption function and multi segment decryption function that will both use the mlx5 vectors. The added tests will test both data integrity and correct stat values. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- app/test/test_cryptodev.c | 277 ++++++++++++++++++++ app/test/test_cryptodev_mlx5_test_vectors.h | 3 - 2 files changed, 277 insertions(+), 3 deletions(-) diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c index 8dbe324b81..4d27a9444c 100644 --- a/app/test/test_cryptodev.c +++ b/app/test/test_cryptodev.c @@ -6681,6 +6681,219 @@ test_mlx5_decryption(const struct mlx5_test_data *tdata) return 0; } +static int +test_mlx5_encryption_sgl(const struct mlx5_test_data *tdata) +{ + struct crypto_testsuite_params *ts_params = &testsuite_params; + struct crypto_unittest_params *ut_params = &unittest_params; + struct rte_cryptodev_sym_capability_idx cap_idx; + struct rte_cryptodev_info dev_info; + struct rte_cryptodev_stats stats; + uint8_t buffer[10000]; + const uint8_t *ciphertext; + const uint8_t *reference_ciphertext; + uint64_t feat_flags; + unsigned int plaintext_pad_len; + unsigned int plaintext_len; + int retval; + + /* Verify the capabilities */ + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_AES_XTS; + if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0], + &cap_idx) == NULL) + return TEST_SKIPPED; + rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info); + feat_flags = dev_info.feature_flags; + if (!(feat_flags & RTE_CRYPTODEV_FF_IN_PLACE_SGL)) { + printf("Device doesn't support in-place scatter-gather. " + "Test Skipped.\n"); + return TEST_SKIPPED; + } + if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) && + (!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) { + printf("Device doesn't support RAW data-path APIs.\n"); + return TEST_SKIPPED; + } + if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) + return TEST_SKIPPED; + /* Create mlx5 session */ + retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0], + RTE_CRYPTO_CIPHER_OP_ENCRYPT, + RTE_CRYPTO_CIPHER_AES_XTS, + tdata->key.data, tdata->key.len, + tdata->cipher_iv.len, 0); + if (retval < 0) + return retval; + plaintext_len = ceil_byte_length(tdata->plaintext.len); + /* Append data which is padded to a multiple */ + /* of the algorithms block size */ + plaintext_pad_len = RTE_ALIGN_CEIL(plaintext_len, 8); + ut_params->ibuf = create_segmented_mbuf(ts_params->mbuf_pool, + plaintext_pad_len, 8, 0); + if (unlikely(ut_params->ibuf == NULL)) + return -ENOMEM; + pktmbuf_write(ut_params->ibuf, 0, plaintext_len, tdata->plaintext.data); + debug_hexdump(stdout, "plaintext:", tdata->plaintext.data, + plaintext_len); + /* Create mlx5 operation */ + retval = create_wireless_algo_cipher_operation(tdata->cipher_iv.data, + tdata->cipher_iv.len, (tdata->cipher.len_bits), + (tdata->cipher.offset_bits)); + if (retval < 0) + return retval; + if (global_api_test_type == CRYPTODEV_RAW_API_TEST) + process_sym_raw_dp_op(ts_params->valid_devs[0], 0, + ut_params->op, 1, 0, 1, tdata->cipher_iv.len); + else + ut_params->op = process_crypto_request(ts_params->valid_devs[0], + ut_params->op); + TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf"); + ut_params->obuf = ut_params->op->sym->m_dst; + if (ut_params->obuf) + ciphertext = rte_pktmbuf_read(ut_params->obuf, 0, + plaintext_len, buffer); + else + ciphertext = rte_pktmbuf_read(ut_params->ibuf, + tdata->cipher.offset_bits, + plaintext_len, buffer); + if (unlikely(ciphertext == NULL)) + return -ENOMEM; + /* Validate obuf */ + debug_hexdump(stdout, "ciphertext:", ciphertext, plaintext_len); + reference_ciphertext = tdata->ciphertext.data + + tdata->cipher.offset_bits; + /* Validate obuf */ + TEST_ASSERT_BUFFERS_ARE_EQUAL_BIT( + ciphertext, + reference_ciphertext, + tdata->validCipherLenInBits.len, + "MLX5 Ciphertext data not as expected"); + /* Validate stats */ + TEST_ASSERT_SUCCESS(rte_cryptodev_stats_get(ts_params->valid_devs[0], + &stats), + "rte_cryptodev_stats_get failed"); + TEST_ASSERT((stats.enqueued_count == 1), + "rte_cryptodev_stats_get returned unexpected enqueued stat"); + TEST_ASSERT((stats.dequeued_count == 1), + "rte_cryptodev_stats_get returned unexpected dequeued stat"); + TEST_ASSERT((stats.enqueue_err_count == 0), + "rte_cryptodev_stats_get returned error enqueued stat"); + TEST_ASSERT((stats.dequeue_err_count == 0), + "rte_cryptodev_stats_get returned error dequeued stat"); + return 0; +} + +static int +test_mlx5_decryption_sgl(const struct mlx5_test_data *tdata) +{ + struct crypto_testsuite_params *ts_params = &testsuite_params; + struct crypto_unittest_params *ut_params = &unittest_params; + struct rte_cryptodev_sym_capability_idx cap_idx; + struct rte_cryptodev_stats stats; + struct rte_cryptodev_info dev_info; + uint8_t *ciphertext; + const uint8_t *plaintext; + const uint8_t *reference_plaintext; + uint8_t buffer[10000]; + uint64_t feat_flags; + unsigned int ciphertext_pad_len; + unsigned int ciphertext_len; + int retval; + + rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info); + feat_flags = dev_info.feature_flags; + if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) && + (!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) { + printf("Device doesn't support RAW data-path APIs.\n"); + return -ENOTSUP; + } + if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) + return -ENOTSUP; + /* Verify the capabilities */ + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_AES_XTS; + if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0], + &cap_idx) == NULL) + return -ENOTSUP; + /* Create mlx5 session */ + retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0], + RTE_CRYPTO_CIPHER_OP_DECRYPT, + RTE_CRYPTO_CIPHER_AES_XTS, + tdata->key.data, tdata->key.len, + tdata->cipher_iv.len, 0); + if (retval < 0) + return retval; + ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + if (unlikely(ut_params->ibuf == NULL)) + return -ENOMEM; + /* Clear mbuf payload */ + memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0, + rte_pktmbuf_tailroom(ut_params->ibuf)); + ciphertext_len = ceil_byte_length(tdata->ciphertext.len); + /* Append data which is padded to a multiple */ + /* of the algorithms block size */ + ciphertext_pad_len = RTE_ALIGN_CEIL(ciphertext_len, 8); + ciphertext = (uint8_t *)rte_pktmbuf_append(ut_params->ibuf, + ciphertext_pad_len); + if (unlikely(ciphertext == NULL)) + return -ENOMEM; + ut_params->ibuf = create_segmented_mbuf(ts_params->mbuf_pool, + ciphertext_pad_len, 8, 0); + if (unlikely(ut_params->ibuf == NULL)) + return -ENOMEM; + pktmbuf_write(ut_params->ibuf, 0, ciphertext_len, + tdata->ciphertext.data); + memcpy(ciphertext, tdata->ciphertext.data, ciphertext_len); + if (unlikely(ciphertext == NULL)) + return -ENOMEM; + debug_hexdump(stdout, "ciphertext:", ciphertext, ciphertext_len); + /* Create mlx5 operation */ + retval = create_wireless_algo_cipher_operation(tdata->cipher_iv.data, + tdata->cipher_iv.len, (tdata->cipher.len_bits), + (tdata->cipher.offset_bits)); + if (retval < 0) + return retval; + if (global_api_test_type == CRYPTODEV_RAW_API_TEST) + process_sym_raw_dp_op(ts_params->valid_devs[0], 0, + ut_params->op, 1, 0, 1, 0); + else + ut_params->op = process_crypto_request(ts_params->valid_devs[0], + ut_params->op); + TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf"); + ut_params->obuf = ut_params->op->sym->m_dst; + if (ut_params->obuf) + plaintext = rte_pktmbuf_read(ut_params->obuf, 0, + ciphertext_len, buffer); + else + plaintext = rte_pktmbuf_read(ut_params->ibuf, + tdata->cipher.offset_bits, + ciphertext_len, buffer); + debug_hexdump(stdout, "plaintext:", plaintext, ciphertext_len); + reference_plaintext = tdata->plaintext.data + + (tdata->cipher.offset_bits); + /* Validate obuf */ + TEST_ASSERT_BUFFERS_ARE_EQUAL_BIT( + plaintext, + reference_plaintext, + tdata->validCipherLenInBits.len, + "MLX5 Plaintext data not as expected"); + /* Validate stats */ + TEST_ASSERT_SUCCESS(rte_cryptodev_stats_get(ts_params->valid_devs[0], + &stats), + "rte_cryptodev_stats_get failed"); + TEST_ASSERT((stats.enqueued_count == 1), + "rte_cryptodev_stats_get returned unexpected enqueued stat"); + TEST_ASSERT((stats.dequeued_count == 1), + "rte_cryptodev_stats_get returned unexpected dequeued stat"); + TEST_ASSERT((stats.enqueue_err_count == 0), + "rte_cryptodev_stats_get returned error enqueued stat"); + TEST_ASSERT((stats.dequeue_err_count == 0), + "rte_cryptodev_stats_get returned error dequeued stat"); + return 0; +} + + static int test_kasumi_encryption_test_case_1(void) { @@ -7345,6 +7558,54 @@ test_mlx5_decryption_test_case_4(void) return test_mlx5_decryption(&mlx5_test_case_cipher_aes_xts_4); } +static int +test_mlx5_encryption_test_case_1_sgl(void) +{ + return test_mlx5_encryption_sgl(&mlx5_test_case_cipher_aes_xts_1); +} + +static int +test_mlx5_encryption_test_case_2_sgl(void) +{ + return test_mlx5_encryption_sgl(&mlx5_test_case_cipher_aes_xts_2); +} + +static int +test_mlx5_encryption_test_case_3_sgl(void) +{ + return test_mlx5_encryption_sgl(&mlx5_test_case_cipher_aes_xts_3); +} + +static int +test_mlx5_encryption_test_case_4_sgl(void) +{ + return test_mlx5_encryption_sgl(&mlx5_test_case_cipher_aes_xts_4); +} + +static int +test_mlx5_decryption_test_case_1_sgl(void) +{ + return test_mlx5_decryption_sgl(&mlx5_test_case_cipher_aes_xts_1); +} + +static int +test_mlx5_decryption_test_case_2_sgl(void) +{ + return test_mlx5_decryption_sgl(&mlx5_test_case_cipher_aes_xts_2); +} + +static int +test_mlx5_decryption_test_case_3_sgl(void) +{ + return test_mlx5_decryption_sgl(&mlx5_test_case_cipher_aes_xts_3); +} + +static int +test_mlx5_decryption_test_case_4_sgl(void) +{ + return test_mlx5_decryption_sgl(&mlx5_test_case_cipher_aes_xts_4); +} + static int test_mixed_check_if_unsupported(const struct mixed_cipher_auth_test_data *tdata) { @@ -14747,6 +15008,22 @@ static struct unit_test_suite cryptodev_mlx5_testsuite = { test_mlx5_encryption_test_case_3), TEST_CASE_ST(ut_setup, ut_teardown, test_mlx5_encryption_test_case_4), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_decryption_test_case_1_sgl), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_decryption_test_case_2_sgl), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_decryption_test_case_3_sgl), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_decryption_test_case_4_sgl), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_encryption_test_case_1_sgl), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_encryption_test_case_2_sgl), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_encryption_test_case_3_sgl), + TEST_CASE_ST(ut_setup, ut_teardown, + test_mlx5_encryption_test_case_4_sgl), TEST_CASES_END() } }; diff --git a/app/test/test_cryptodev_mlx5_test_vectors.h b/app/test/test_cryptodev_mlx5_test_vectors.h index 2a05aa4626..db7e5fc744 100644 --- a/app/test/test_cryptodev_mlx5_test_vectors.h +++ b/app/test/test_cryptodev_mlx5_test_vectors.h @@ -2496,7 +2496,4 @@ static struct mlx5_test_data mlx5_test_case_cipher_aes_xts_4 = { .dataunit_len = 0 }; - - - #endif /*TEST_CRYPTODEV_MLX5_TEST_VECTORS_H_*/