From patchwork Mon Sep 25 06:06:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 131880 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2EA3042633; Mon, 25 Sep 2023 08:08:18 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1261C40C35; Mon, 25 Sep 2023 08:07:51 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2100.outbound.protection.outlook.com [40.107.236.100]) by mails.dpdk.org (Postfix) with ESMTP id A3A2040C35 for ; Mon, 25 Sep 2023 08:07:45 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aDPsXC+Hy6CiOEt3TWxYoxXy0d+TS1lHLW+NnRWNHyRicUhA9wxyIEobtCjE78TPt8z2aSkFJEo2hf5icnLFXsXIfdxkVQSMnf+oLbTQ6qnAKIuI2cNmxJ2lHVN7vklc8uj5fB0fobG+1KAQr7Mc5+bDOmyJzAVvzzGbPUjhOrzWhqSCh9B3RV2Q6v3rKR0sQlGEMFJ4WFfaVUKYUSYNtqZ/CHAIzPOexrrDOiIuBpbrziSDAXC6q22hO/jnNPfPxpdUFDn7yLHZkdfNZbJYL8uCbBP0MxU0uM2A0G+wbaVw/9EBG/eo4yaRsuObrY68TVNU1Kmpt8fWHbHXYjlJUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oZ82ZZJPoaqsFD6dhi6piR39+zuJPk/9I251siRXyyk=; b=lN/96P05g9VjnbVf0c7IvQbbFKZNjMnBFW+LUQOFGA3KJMgZwLnrriKcoaXBR/Hvi2qJB3NcuaYfcrewYe0Dsp/ej0Qc54mAXZOAAC1CtiwSS2R7s+OgS+4YpC+vp0Uxuv5EKc0tSo8odciXrQ8Bba0NgzRgJTxsv97A5bg4gQS44kwfDX/MYAMv1JutvHyJ15/LwLA/iprSpBCT3ojw0R8WdgVP+6vqAUKlCiUM2VqY9y03ZWXnDSBWF7WlbsiJKNjh6Vji9OEo5CGdl8MFQWQWh+Jf5BI+T7PZBEW6iZ2M+e9ugK0Q+38qMnKssw8AhG05Y0sGqY1FQZ2/v98rug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oZ82ZZJPoaqsFD6dhi6piR39+zuJPk/9I251siRXyyk=; b=NmEgNdjddJqX7QUg1aEMaXGeGMDyrH58VbwQRgl9iXSdfQq639xL+XhxPQt/Jbv62IZuNODGuMB+e8I6Xv0kI9+OK7YOB0JSbkvipWEP6o749hGNA855Vm4v8H7YPuye/FMIBc3LYE8ZvOX9wK37bxuSluV30azONm+P7u2qNqk= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB5568.namprd13.prod.outlook.com (2603:10b6:510:12b::16) by DM8PR13MB5080.namprd13.prod.outlook.com (2603:10b6:8:23::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.28; Mon, 25 Sep 2023 06:07:44 +0000 Received: from PH0PR13MB5568.namprd13.prod.outlook.com ([fe80::b070:92e1:931e:fee7]) by PH0PR13MB5568.namprd13.prod.outlook.com ([fe80::b070:92e1:931e:fee7%4]) with mapi id 15.20.6792.026; Mon, 25 Sep 2023 06:07:43 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Chang Miao , Shihong Wang , Chaoyong He Subject: [PATCH 07/10] net/nfp: create security session Date: Mon, 25 Sep 2023 14:06:41 +0800 Message-Id: <20230925060644.1458598-8-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230925060644.1458598-1-chaoyong.he@corigine.com> References: <20230925060644.1458598-1-chaoyong.he@corigine.com> X-ClientProxiedBy: BY5PR04CA0001.namprd04.prod.outlook.com (2603:10b6:a03:1d0::11) To PH0PR13MB5568.namprd13.prod.outlook.com (2603:10b6:510:12b::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR13MB5568:EE_|DM8PR13MB5080:EE_ X-MS-Office365-Filtering-Correlation-Id: 923beb4f-b64e-4814-f2d3-08dbbd8db9f5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ud39BHyTQo1GGThVHZwt0aBMlXk7LhiXsB66Fzx4CKXLjA1rpcP23b5ClyHsZpLCRde1fqLA+Qe5mDHfmfRiO+Mka445OYw4gt4BcTT+Lg8V+XL+lPCjjcNrzxF8QJ+ukiU4z/lZ2M0V5nT5tu7tITOKU6Tg8zpZ6kZh2akAbLb1rhGN9esq+E6kaK8+T9AgJI5df4P/P6ztoHj3nwQxTbzBiWo1/dk/2PN+bZnzXjnGUZorqxt66PrhtD6rw/RmJYAD+FSabjYMG2Z9mp6sU/LB2kHnSNxg29nnVh/3q6WefoJBOQrKmk75CVDzrOxnGK+YQE+pM35AZ0RCCNk8snDgmusco3duIowwOTKujn886LVhq2/ODQOBj84jZcrL6N+Osb+tFCObERVlbNPVoDVBtCGopvzdxZWQeFXR7VTBL7bOrXFvO4T/NDE9r9Gb1PXuuxUijTMq5MlZNMueG4oLqgXahUtAVgy9EgkpHatr8eGwurimnxEOOHBYJEyTd1c81QHtaJIJou0ABP3qsK6uFfLB4w/6PdVGzp7d+XLZhgWbB0pyM5QZjcwQUv1KNP+hRvOif6W65dkw9WcgPQ2fY6x4Xt3pOWqUWnCt+t5RiYqtcuGQIDRq5qa9mZzYQG9DXu+ORXr+bWthup0VH4SX2dMndcRYltREu74eMJw= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR13MB5568.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366004)(396003)(346002)(39830400003)(376002)(136003)(230922051799003)(186009)(1800799009)(451199024)(15650500001)(30864003)(2906002)(5660300002)(66946007)(66476007)(44832011)(66556008)(54906003)(478600001)(6512007)(52116002)(2616005)(107886003)(1076003)(26005)(6486002)(6506007)(8936002)(8676002)(4326008)(316002)(6916009)(41300700001)(83380400001)(38350700002)(86362001)(36756003)(38100700002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: fUFuYcjRKKwmARqpzJifuSmsLENknsyIr7Ux53Re85ESfMkjgY+RSbdIf2f4IFvfI2Px+UmjezFvygNRMY73ZpgqgkZZ2cK28dZxbSIi8McMeru1SowxxnrMzB7CcYcTfsNbgmwE4VS7G/BP+GvSk8UCJ4HFYsfc10pv2MovQboRHRDaTL4VYjXDJRxRZIrJzvuTTnG5m4huhuR48EvHfHyEjTEzZL3Jcy9t0Vzi+azqxiNYpsMDBrXK1V6inni7NU6T8PtLht6Ntfz93C2Ije4Pj1Qsi2sM6E1gd/6Q4ItUknkLJ7wz+h1jVGpWtvX1J9PtTbnRT6CNBPHEQYd6/q6s7a0l3VXyvWQHbEdMD50dVhsxo6+aAJuGhGpMIb4/cjZXL9BIeRGHfH4P/ikpBB2iD8+vdjYqsqkkclAobPZ4eo+4xqcHb7UtEdUaa3ZpvOVQw2fNozRw0dm7alyQUrEi7Tj1rXhPaWt4KBF9rDMqYt+ivGVOmcac0eqxlGGNc8/ogQ+tHgV831ldSqes8brtny27LeCXuxaYWBKsKSBhWKtl7fP2744GoT6yPfPD1mCR3aY7IhnOr8d/OjWh6eRUwg3jJbi5IjH3hNFrfsHkKlhlp+9OobThSQ2BKVjsv/h9EMd0du2UlkRjKBLHztNHxrX55zlg4ucPk3E9YVwHwPJEIQDJzii3vITC//AGYG1Vm9ta1xoCiCM7qclcIthpXXD5wNZdbu0OD2yyKpzf/MpCoso4HnemhwI8A/7QwSlS9N5pEvdoUd+gu6eWJtk9oMKy7PQugAgX4QMbYY23BjOWb6qmE/lW+tSJ5+6IvGUabHkP/D9TmZWCoxLDDkfRt9TQ6zYWq0G543fltOnfnXaRqfrTLH9dvgBtWZ+RmVqEg7zYXPXdM65e0ddHIqLsTLEVVsbR87Z3T9pxiXAYVtj7SyKeDihAENgnxU4FuNsNWdxUALK+WN+OTTW31A0HiMK4iX2l9pSAZr7XMk5md7t0G/nUHIYFoDZXAUlElwmYdBgKKMeWlwNBTFShSX0SOU76LzSVoQZAf38Q04h6A0YIceU9IOYF2Z0WwasrOoBCeGkK2yLCnmVyCdmzxLkhTC6ReHUK/u+4syMUm11EjgeKSIIi6u8NoxfwUCiwoX3xork/7LAbBUS7AtEURWdg3fJVj9ULKkdyeiLJd+Vrr9pi+Kh48yxgxcDKVFsUedYaWgGM/ygQrw6Lph/wkwGY++JEBKEblv3qAZobznd1eORp+DLqDw+PRXX/Aul/gk39CM2Q7w60bLUVKLSpLy+FcALdLsKSzRDz6MxQNLTYppx3A+hjL5NVtj+uvXyH06uNj26Nohs7c768vHxxiSRFp2GmRZmXq6hUi9w16JCAH9MDZp8xW8+usBkNLAykrrX1jcuoxXG6oCtSmkuaSOnHpZzvwa8b3kNcWPYQxn2pjhn1B5Ozty+pDgTLnfJbmCFQqIKSFuMlidGne6oejC31s/mhm0CYCttqvPVQGe3uYpH2EAkc7TcH5OBktusanCo4JR+l16EFHvuju7NbqrxLc9dw8Bq9sBu/qQz5Ffhm8agE5R9zwlM3HhRaVpiUCL21ZeDGz8ZPwWlVQozosA== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 923beb4f-b64e-4814-f2d3-08dbbd8db9f5 X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB5568.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2023 06:07:43.8773 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Tzf3zquT9VGsL4yXA9SqWxcEjkLaAKDEgdW9PzIDRj9gR+UXb4uYMQIKkVeK5dYcakpItAeYQQLfzx83rlB8Hj09STJP+v1N/99o4TH0M2U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR13MB5080 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Chang Miao Create security session to manage IPsec protocol specific message and crypto parameters. This session support full protocol offload and inline crypto operation with NIC devices. Signed-off-by: Chang Miao Signed-off-by: Shihong Wang Reviewed-by: Chaoyong He --- drivers/net/nfp/nfp_ipsec.c | 711 ++++++++++++++++++++++++++++++++++++ 1 file changed, 711 insertions(+) diff --git a/drivers/net/nfp/nfp_ipsec.c b/drivers/net/nfp/nfp_ipsec.c index d4292c3b90..d3fdef2d76 100644 --- a/drivers/net/nfp/nfp_ipsec.c +++ b/drivers/net/nfp/nfp_ipsec.c @@ -17,6 +17,8 @@ #include "nfp_logs.h" #include "nfp_rxtx.h" +#define NFP_UDP_ESP_PORT 4500 + static const struct rte_cryptodev_capabilities nfp_crypto_caps[] = { { .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, @@ -375,6 +377,62 @@ enum nfp_ipsec_cfg_msg_rsp_codes { NFP_IPSEC_CFG_MSG_SA_INVALID_CMD }; +enum nfp_ipsec_mode { + NFP_IPSEC_MODE_TRANSPORT, + NFP_IPSEC_MODE_TUNNEL, +}; + +enum nfp_ipsec_protocal { + NFP_IPSEC_PROTOCOL_AH, + NFP_IPSEC_PROTOCOL_ESP, +}; + +/* Cipher modes */ +enum nfp_ipsec_cimode { + NFP_IPSEC_CIMODE_ECB, + NFP_IPSEC_CIMODE_CBC, + NFP_IPSEC_CIMODE_CFB, + NFP_IPSEC_CIMODE_OFB, + NFP_IPSEC_CIMODE_CTR, +}; + +/* Hash types */ +enum nfp_ipsec_hash_type { + NFP_IPSEC_HASH_NONE, + NFP_IPSEC_HASH_MD5_96, + NFP_IPSEC_HASH_SHA1_96, + NFP_IPSEC_HASH_SHA256_96, + NFP_IPSEC_HASH_SHA384_96, + NFP_IPSEC_HASH_SHA512_96, + NFP_IPSEC_HASH_MD5_128, + NFP_IPSEC_HASH_SHA1_80, + NFP_IPSEC_HASH_SHA256_128, + NFP_IPSEC_HASH_SHA384_192, + NFP_IPSEC_HASH_SHA512_256, + NFP_IPSEC_HASH_GF128_128, + NFP_IPSEC_HASH_POLY1305_128, +}; + +/* Cipher types */ +enum nfp_ipsec_cipher_type { + NFP_IPSEC_CIPHER_NULL, + NFP_IPSEC_CIPHER_3DES, + NFP_IPSEC_CIPHER_AES128, + NFP_IPSEC_CIPHER_AES192, + NFP_IPSEC_CIPHER_AES256, + NFP_IPSEC_CIPHER_AES128_NULL, + NFP_IPSEC_CIPHER_AES192_NULL, + NFP_IPSEC_CIPHER_AES256_NULL, + NFP_IPSEC_CIPHER_CHACHA20, +}; + +/* Don't Fragment types */ +enum nfp_ipsec_df_type { + NFP_IPSEC_DF_CLEAR, + NFP_IPSEC_DF_SET, + NFP_IPSEC_DF_COPY, +}; + static int nfp_ipsec_cfg_cmd_issue(struct nfp_net_hw *hw, struct nfp_ipsec_msg *msg) @@ -427,6 +485,658 @@ nfp_ipsec_cfg_cmd_issue(struct nfp_net_hw *hw, return ret; } +/** + * Get valid SA index from SA table + * + * @param data + * SA table pointer + * @param sa_idx + * SA table index pointer + * + * @return + * Negative number on full or repeat, 0 on success + * + * Note: multiple sockets may create same SA session. + */ +static void +nfp_get_sa_entry(struct nfp_net_ipsec_data *data, + int *sa_idx) +{ + uint32_t i; + + for (i = 0; i < NFP_NET_IPSEC_MAX_SA_CNT; i++) { + if (data->sa_entries[i] == NULL) { + *sa_idx = i; + break; + } + } +} + +static void +nfp_aesgcm_iv_update(struct ipsec_add_sa *cfg, + uint16_t iv_len, + const char *iv_string) +{ + int i; + char *save; + char *iv_b; + char *iv_str; + uint8_t *cfg_iv; + + iv_str = strdup(iv_string); + cfg_iv = (uint8_t *)cfg->aesgcm_fields.iv; + + for (i = 0; i < iv_len; i++) { + iv_b = strtok_r(i ? NULL : iv_str, ",", &save); + if (iv_b == NULL) + break; + + cfg_iv[i] = strtoul(iv_b, NULL, 0); + } + + *(uint32_t *)cfg_iv = rte_be_to_cpu_32(*(uint32_t *)cfg_iv); + *(uint32_t *)&cfg_iv[4] = rte_be_to_cpu_32(*(uint32_t *)&cfg_iv[4]); + + free(iv_str); +} + +static int +set_aes_keylen(uint32_t key_length, + struct ipsec_add_sa *cfg) +{ + switch (key_length << 3) { + case 128: + cfg->ctrl_word.cipher = NFP_IPSEC_CIPHER_AES128; + break; + case 192: + cfg->ctrl_word.cipher = NFP_IPSEC_CIPHER_AES192; + break; + case 256: + cfg->ctrl_word.cipher = NFP_IPSEC_CIPHER_AES256; + break; + default: + PMD_DRV_LOG(ERR, "AES cipher key length is illegal!"); + return -EINVAL; + } + + return 0; +} + +/* Map rte_security_session_conf aead algo to NFP aead algo */ +static int +nfp_aead_map(struct rte_eth_dev *eth_dev, + struct rte_crypto_aead_xform *aead, + uint32_t key_length, + struct ipsec_add_sa *cfg) +{ + int ret; + uint32_t i; + uint32_t index; + uint16_t iv_len; + uint32_t offset; + uint32_t device_id; + const char *iv_str; + const uint32_t *key; + struct nfp_net_hw *hw; + + hw = NFP_NET_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); + device_id = hw->device_id; + offset = 0; + + switch (aead->algo) { + case RTE_CRYPTO_AEAD_AES_GCM: + if (aead->digest_length != 16) { + PMD_DRV_LOG(ERR, "ICV must be 128bit with RTE_CRYPTO_AEAD_AES_GCM!"); + return -EINVAL; + } + + cfg->ctrl_word.cimode = NFP_IPSEC_CIMODE_CTR; + cfg->ctrl_word.hash = NFP_IPSEC_HASH_GF128_128; + + ret = set_aes_keylen(key_length, cfg); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to set AES_GCM key length!"); + return -EINVAL; + } + + break; + case RTE_CRYPTO_AEAD_CHACHA20_POLY1305: + if (device_id != PCI_DEVICE_ID_NFP3800_PF_NIC) { + PMD_DRV_LOG(ERR, "Unsupported aead CHACHA20_POLY1305 algorithm!"); + return -EINVAL; + } + + if (aead->digest_length != 16) { + PMD_DRV_LOG(ERR, "ICV must be 128bit with RTE_CRYPTO_AEAD_CHACHA20_POLY1305"); + return -EINVAL; + } + + /* Aead->alg_key_len includes 32-bit salt */ + if (key_length != 32) { + PMD_DRV_LOG(ERR, "Unsupported CHACHA20 key length"); + return -EINVAL; + } + + /* The CHACHA20's mode is not configured */ + cfg->ctrl_word.hash = NFP_IPSEC_HASH_POLY1305_128; + cfg->ctrl_word.cipher = NFP_IPSEC_CIPHER_CHACHA20; + break; + default: + PMD_DRV_LOG(ERR, "Unsupported aead algorithm!"); + return -EINVAL; + } + + key = (const uint32_t *)(aead->key.data); + + /* + * The CHACHA20's key order needs to be adjusted based on hardware design. + * Unadjusted order: {K0, K1, K2, K3, K4, K5, K6, K7} + * Adjusted order: {K4, K5, K6, K7, K0, K1, K2, K3} + */ + if (aead->algo == RTE_CRYPTO_AEAD_CHACHA20_POLY1305) + offset = key_length / sizeof(cfg->cipher_key[0]) << 1; + + for (i = 0; i < key_length / sizeof(cfg->cipher_key[0]); i++) { + index = (i + offset) % (key_length / sizeof(cfg->cipher_key[0])); + cfg->cipher_key[index] = rte_cpu_to_be_32(*key++); + } + + /* + * The iv of the FW is equal to ESN by default. Reading the + * iv of the configuration information is not supported. + */ + iv_str = getenv("ETH_SEC_IV_OVR"); + if (iv_str != NULL) { + iv_len = aead->iv.length; + nfp_aesgcm_iv_update(cfg, iv_len, iv_str); + } + + return 0; +} + +/* Map rte_security_session_conf cipher algo to NFP cipher algo */ +static int +nfp_cipher_map(struct rte_eth_dev *eth_dev, + struct rte_crypto_cipher_xform *cipher, + uint32_t key_length, + struct ipsec_add_sa *cfg) +{ + int ret; + uint32_t i; + uint32_t device_id; + const uint32_t *key; + struct nfp_net_hw *hw; + + hw = NFP_NET_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); + device_id = hw->device_id; + + switch (cipher->algo) { + case RTE_CRYPTO_CIPHER_NULL: + cfg->ctrl_word.cimode = NFP_IPSEC_CIMODE_CBC; + cfg->ctrl_word.cipher = NFP_IPSEC_CIPHER_NULL; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + if (device_id == PCI_DEVICE_ID_NFP3800_PF_NIC) { + PMD_DRV_LOG(ERR, "Unsupported 3DESCBC encryption algorithm!"); + return -EINVAL; + } + + cfg->ctrl_word.cimode = NFP_IPSEC_CIMODE_CBC; + cfg->ctrl_word.cipher = NFP_IPSEC_CIPHER_3DES; + break; + case RTE_CRYPTO_CIPHER_AES_CBC: + cfg->ctrl_word.cimode = NFP_IPSEC_CIMODE_CBC; + ret = set_aes_keylen(key_length, cfg); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to set cipher key length!"); + return -EINVAL; + } + + break; + default: + PMD_DRV_LOG(ERR, "Unsupported cipher alg!"); + return -EINVAL; + } + + key = (const uint32_t *)(cipher->key.data); + if (key_length > sizeof(cfg->cipher_key)) { + PMD_DRV_LOG(ERR, "Insufficient space for offloaded key"); + return -EINVAL; + } + + for (i = 0; i < key_length / sizeof(cfg->cipher_key[0]); i++) + cfg->cipher_key[i] = rte_cpu_to_be_32(*key++); + + return 0; +} + +static void +set_md5hmac(struct ipsec_add_sa *cfg, + uint32_t *digest_length) +{ + switch (*digest_length) { + case 96: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_MD5_96; + break; + case 128: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_MD5_128; + break; + default: + *digest_length = 0; + } +} + +static void +set_sha1hmac(struct ipsec_add_sa *cfg, + uint32_t *digest_length) +{ + switch (*digest_length) { + case 96: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_SHA1_96; + break; + case 80: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_SHA1_80; + break; + default: + *digest_length = 0; + } +} + +static void +set_sha2_256hmac(struct ipsec_add_sa *cfg, + uint32_t *digest_length) +{ + switch (*digest_length) { + case 96: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_SHA256_96; + break; + case 128: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_SHA256_128; + break; + default: + *digest_length = 0; + } +} + +static void +set_sha2_384hmac(struct ipsec_add_sa *cfg, + uint32_t *digest_length) +{ + switch (*digest_length) { + case 96: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_SHA384_96; + break; + case 192: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_SHA384_192; + break; + default: + *digest_length = 0; + } +} + +static void +set_sha2_512hmac(struct ipsec_add_sa *cfg, + uint32_t *digest_length) +{ + switch (*digest_length) { + case 96: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_SHA512_96; + break; + case 256: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_SHA512_256; + break; + default: + *digest_length = 0; + } +} + +/* Map rte_security_session_conf auth algo to NFP auth algo */ +static int +nfp_auth_map(struct rte_eth_dev *eth_dev, + struct rte_crypto_auth_xform *auth, + uint32_t digest_length, + struct ipsec_add_sa *cfg) +{ + uint32_t i; + uint8_t key_length; + uint32_t device_id; + const uint32_t *key; + struct nfp_net_hw *hw; + + if (digest_length == 0) { + PMD_DRV_LOG(ERR, "Auth digest length is illegal!"); + return -EINVAL; + } + + hw = NFP_NET_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); + device_id = hw->device_id; + digest_length = digest_length << 3; + + switch (auth->algo) { + case RTE_CRYPTO_AUTH_NULL: + cfg->ctrl_word.hash = NFP_IPSEC_HASH_NONE; + digest_length = 1; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + if (device_id == PCI_DEVICE_ID_NFP3800_PF_NIC) { + PMD_DRV_LOG(ERR, "Unsupported MD5HMAC authentication algorithm!"); + return -EINVAL; + } + + set_md5hmac(cfg, &digest_length); + break; + case RTE_CRYPTO_AUTH_SHA1_HMAC: + set_sha1hmac(cfg, &digest_length); + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + set_sha2_256hmac(cfg, &digest_length); + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + set_sha2_384hmac(cfg, &digest_length); + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + set_sha2_512hmac(cfg, &digest_length); + break; + default: + PMD_DRV_LOG(ERR, "Unsupported auth alg!"); + return -EINVAL; + } + + if (digest_length == 0) { + PMD_DRV_LOG(ERR, "Unsupported authentication algorithm digest length"); + return -EINVAL; + } + + key = (const uint32_t *)(auth->key.data); + key_length = auth->key.length; + if (key_length > sizeof(cfg->auth_key)) { + PMD_DRV_LOG(ERR, "Insufficient space for offloaded auth key!"); + return -EINVAL; + } + + for (i = 0; i < key_length / sizeof(cfg->auth_key[0]); i++) + cfg->auth_key[i] = rte_cpu_to_be_32(*key++); + + return 0; +} + +static int +nfp_crypto_msg_build(struct rte_eth_dev *eth_dev, + struct rte_security_session_conf *conf, + struct nfp_ipsec_msg *msg) +{ + int ret; + struct ipsec_add_sa *cfg; + struct rte_crypto_sym_xform *cur; + struct rte_crypto_sym_xform *next; + enum rte_security_ipsec_sa_direction direction; + + cur = conf->crypto_xform; + if (cur == NULL) { + PMD_DRV_LOG(ERR, "Unsupported crypto_xform is NULL!"); + return -EINVAL; + } + + next = cur->next; + direction = conf->ipsec.direction; + cfg = &msg->cfg_add_sa; + + switch (cur->type) { + case RTE_CRYPTO_SYM_XFORM_AEAD: + /* Aead transforms can be used for either inbound/outbound IPsec SAs */ + if (next != NULL) { + PMD_DRV_LOG(ERR, "Next crypto_xform type should be NULL!"); + return -EINVAL; + } + + ret = nfp_aead_map(eth_dev, &cur->aead, cur->aead.key.length, cfg); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to map aead alg!"); + return ret; + } + + cfg->aesgcm_fields.salt = rte_cpu_to_be_32(conf->ipsec.salt); + break; + case RTE_CRYPTO_SYM_XFORM_AUTH: + /* Only support Auth + Cipher for inbound */ + if (direction != RTE_SECURITY_IPSEC_SA_DIR_INGRESS) { + PMD_DRV_LOG(ERR, "Direction should be INGRESS, but it is not!"); + return -EINVAL; + } + + if (next == NULL || next->type != RTE_CRYPTO_SYM_XFORM_CIPHER) { + PMD_DRV_LOG(ERR, "Next crypto_xfrm should be cipher, but it is not!"); + return -EINVAL; + } + + ret = nfp_auth_map(eth_dev, &cur->auth, cur->auth.digest_length, cfg); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to map auth alg!"); + return ret; + } + + ret = nfp_cipher_map(eth_dev, &next->cipher, next->cipher.key.length, cfg); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to map cipher alg!"); + return ret; + } + + break; + case RTE_CRYPTO_SYM_XFORM_CIPHER: + /* Only support Cipher + Auth for outbound */ + if (direction != RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { + PMD_DRV_LOG(ERR, "Direction should be EGRESS, but it is not!"); + return -EINVAL; + } + + if (next == NULL || next->type != RTE_CRYPTO_SYM_XFORM_AUTH) { + PMD_DRV_LOG(ERR, "Next crypto_xfrm should be auth, but it is not!"); + return -EINVAL; + } + + ret = nfp_cipher_map(eth_dev, &cur->cipher, cur->cipher.key.length, cfg); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to map cipher alg!"); + return ret; + } + + ret = nfp_auth_map(eth_dev, &next->auth, next->auth.digest_length, cfg); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to map auth alg!"); + return ret; + } + + break; + default: + PMD_DRV_LOG(ERR, "Unsupported crypto_xform type!"); + return -EINVAL; + } + + return 0; +} + +static int +nfp_ipsec_msg_build(struct rte_eth_dev *eth_dev, + struct rte_security_session_conf *conf, + struct nfp_ipsec_msg *msg) +{ + int ret; + uint32_t i; + uint32_t *src_ip; + uint32_t *dst_ip; + struct ipsec_add_sa *cfg; + enum rte_security_ipsec_tunnel_type type; + + cfg = &msg->cfg_add_sa; + cfg->spi = conf->ipsec.spi; + cfg->pmtu_limit = 0xffff; + + /* + * UDP encapsulation + * + * 1: Do UDP encapsulation/decapsulation + * 0: No UDP encapsulation + */ + if (conf->ipsec.options.udp_encap == 1) { + cfg->udp_enable = 1; + cfg->natt_dst_port = NFP_UDP_ESP_PORT; + cfg->natt_src_port = NFP_UDP_ESP_PORT; + } + + if (conf->ipsec.options.copy_df == 1) + cfg->df_ctrl = NFP_IPSEC_DF_COPY; + else if (conf->ipsec.tunnel.ipv4.df != 0) + cfg->df_ctrl = NFP_IPSEC_DF_SET; + else + cfg->df_ctrl = NFP_IPSEC_DF_CLEAR; + + switch (conf->action_type) { + case RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO: + cfg->ctrl_word.encap_dsbl = 1; + break; + case RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL: + cfg->ctrl_word.encap_dsbl = 0; + break; + default: + PMD_DRV_LOG(ERR, "Unsupported IPsec action for offload, action: %d", + conf->action_type); + return -EINVAL; + } + + switch (conf->ipsec.proto) { + case RTE_SECURITY_IPSEC_SA_PROTO_ESP: + cfg->ctrl_word.proto = NFP_IPSEC_PROTOCOL_ESP; + break; + case RTE_SECURITY_IPSEC_SA_PROTO_AH: + cfg->ctrl_word.proto = NFP_IPSEC_PROTOCOL_AH; + break; + default: + PMD_DRV_LOG(ERR, "Unsupported IPsec protocol for offload, protocol: %d", + conf->ipsec.proto); + return -EINVAL; + } + + switch (conf->ipsec.mode) { + case RTE_SECURITY_IPSEC_SA_MODE_TUNNEL: + type = conf->ipsec.tunnel.type; + cfg->ctrl_word.mode = NFP_IPSEC_MODE_TUNNEL; + if (type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) { + src_ip = &conf->ipsec.tunnel.ipv4.src_ip.s_addr; + dst_ip = &conf->ipsec.tunnel.ipv4.dst_ip.s_addr; + cfg->src_ip[0] = rte_cpu_to_be_32(*src_ip); + cfg->dst_ip[0] = rte_cpu_to_be_32(*dst_ip); + cfg->ipv6 = 0; + } else if (type == RTE_SECURITY_IPSEC_TUNNEL_IPV6) { + src_ip = conf->ipsec.tunnel.ipv6.src_addr.__in6_u.__u6_addr32; + dst_ip = conf->ipsec.tunnel.ipv6.dst_addr.__in6_u.__u6_addr32; + for (i = 0; i < 4; i++) { + cfg->src_ip[i] = rte_cpu_to_be_32(src_ip[i]); + cfg->dst_ip[i] = rte_cpu_to_be_32(dst_ip[i]); + } + cfg->ipv6 = 1; + } else { + PMD_DRV_LOG(ERR, "Unsupported address family!"); + return -EINVAL; + } + + break; + case RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT: + type = conf->ipsec.tunnel.type; + cfg->ctrl_word.mode = NFP_IPSEC_MODE_TRANSPORT; + if (type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) { + cfg->src_ip[0] = 0; + cfg->dst_ip[0] = 0; + cfg->ipv6 = 0; + } else if (type == RTE_SECURITY_IPSEC_TUNNEL_IPV6) { + for (i = 0; i < 4; i++) { + cfg->src_ip[i] = 0; + cfg->dst_ip[i] = 0; + } + cfg->ipv6 = 1; + } else { + PMD_DRV_LOG(ERR, "Unsupported address family!"); + return -EINVAL; + } + + break; + default: + PMD_DRV_LOG(ERR, "Unsupported IPsec mode for offload, mode: %d", + conf->ipsec.mode); + return -EINVAL; + } + + ret = nfp_crypto_msg_build(eth_dev, conf, msg); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to build auth/crypto/aead msg!"); + return ret; + } + + return 0; +} + +static int +nfp_crypto_create_session(void *device, + struct rte_security_session_conf *conf, + struct rte_security_session *session) +{ + int ret; + int sa_idx; + struct nfp_net_hw *hw; + struct nfp_ipsec_msg msg; + struct rte_eth_dev *eth_dev; + struct nfp_ipsec_session *priv_session; + + /* Only support IPsec at present */ + if (conf->protocol != RTE_SECURITY_PROTOCOL_IPSEC) { + PMD_DRV_LOG(ERR, "Unsupported non-IPsec offload!"); + return -EINVAL; + } + + sa_idx = -1; + eth_dev = device; + priv_session = SECURITY_GET_SESS_PRIV(session); + hw = NFP_NET_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); + + if (hw->ipsec_data->sa_free_cnt == 0) { + PMD_DRV_LOG(ERR, "No space in SA table, spi: %d", conf->ipsec.spi); + return -EINVAL; + } + + nfp_get_sa_entry(hw->ipsec_data, &sa_idx); + + if (sa_idx < 0) { + PMD_DRV_LOG(ERR, "Failed to get SA entry!"); + return -EINVAL; + } + + memset(&msg, 0, sizeof(msg)); + ret = nfp_ipsec_msg_build(eth_dev, conf, &msg); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to build IPsec msg!"); + return -EINVAL; + } + + msg.cmd = NFP_IPSEC_CFG_MSG_ADD_SA; + msg.sa_idx = sa_idx; + ret = nfp_ipsec_cfg_cmd_issue(hw, &msg); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to add SA to nic"); + return -EINVAL; + } + + priv_session->action = conf->action_type; + priv_session->ipsec = conf->ipsec; + priv_session->msg = msg.cfg_add_sa; + priv_session->sa_index = sa_idx; + priv_session->dev = eth_dev; + priv_session->user_data = conf->userdata; + + hw->ipsec_data->sa_free_cnt--; + hw->ipsec_data->sa_entries[sa_idx] = priv_session; + + return 0; +} + /** * Get discards packet statistics for each SA * @@ -517,6 +1227,7 @@ nfp_security_session_get_size(void *device __rte_unused) } static const struct rte_security_ops nfp_security_ops = { + .session_create = nfp_crypto_create_session, .session_get_size = nfp_security_session_get_size, .session_stats_get = nfp_security_session_get_stats, .capabilities_get = nfp_crypto_capabilities_get,