From patchwork Tue Apr 30 20:21:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Boyer X-Patchwork-Id: 139756 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 10AE343F56; Tue, 30 Apr 2024 22:23:16 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B82D4406B7; Tue, 30 Apr 2024 22:22:25 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2043.outbound.protection.outlook.com [40.107.237.43]) by mails.dpdk.org (Postfix) with ESMTP id 8335E402DD for ; Tue, 30 Apr 2024 22:22:22 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VQGlQ2Xf6veM1lkrVLYf1V4vnQIqPRahYGy6ZxX6x9U2SQDMe8tZQACgv5Xef8HgB1TgIteLYtJECiPlA7N3GLr3ZFJhnyLCsfxn6Sz89OlVUIX7MIcAwwwt8O48AgoQNFlwNzWQofvPCuQ48xukIb2dk8DOPT8iENWpBiO4qMe7jioRndLPL7KbVZhjRvFL76GoRLLPcOdYWc+IT+yl6YO7rtLIPrai2DIogK2i7cIHkbD92hCgS65rD/inoNPvhdm36fF5V+uolmvhKoBfdrqtb1MarS85wdCifnDzjlpNGrTX7g0D1SZEIRblN7mGPnNXya7TXqISEHUMc13p1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=H8NXA+KvI1WWPwcI25lho72EF5KfCm64kShmfqbk2TA=; b=e1nG5EraVD2AvfbQjP+Dii+ZxFAac3vUM64fzc6ou2e8XVqN8y8ZxNQeJ13xwLu2ei3BaiTJJFV2wAEpCQuvhGF0/9g65nKm2BcAaV5bqBYj7Vrbtj8r+XlbfqfrTWVYegE9fC7qBctkWTvUilNpiutpjfcrWOt8flxyi3rFf2Jd3GVKVWz2ozqFFE3ld7aSEckrpiL8d1e4qeo+217nIIBUtoRbyUqBER2GG2+Nc+eNoIJqToqSf5ehIbI2iLMK/DslzO19l8kT/9chtU5DS6EHh+boaGKeCt+p45SUN17vFeBUDZDlYtYDg0Zj/bs4/KPSoYlo1COE6mMNiqDF3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H8NXA+KvI1WWPwcI25lho72EF5KfCm64kShmfqbk2TA=; b=2FA4LoYYYmnNT/7mjhmJwPJje4BZNu7AMKuLKHp6rZAYjLeJORQu6JcDTB9wz/jwpvUaaNbkrRbXXeVfUM+y1xRAjvObeKqYWd10AK22llJavsGX+7THdPfEVF5EtB3WPt9Kv54VOIpu2b9MrxjKR7XuBeHMcRewjrTgZOJ2Uxo= Received: from SN7PR04CA0012.namprd04.prod.outlook.com (2603:10b6:806:f2::17) by SN7PR12MB8433.namprd12.prod.outlook.com (2603:10b6:806:2e5::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.34; Tue, 30 Apr 2024 20:22:13 +0000 Received: from SN1PEPF00026369.namprd02.prod.outlook.com (2603:10b6:806:f2:cafe::35) by SN7PR04CA0012.outlook.office365.com (2603:10b6:806:f2::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.36 via Frontend Transport; Tue, 30 Apr 2024 20:22:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SN1PEPF00026369.mail.protection.outlook.com (10.167.241.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7544.18 via Frontend Transport; Tue, 30 Apr 2024 20:22:13 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 30 Apr 2024 15:22:12 -0500 From: Andrew Boyer To: CC: Andrew Boyer Subject: [PATCH v2 8/9] crypto/ionic: add a watchdog operation Date: Tue, 30 Apr 2024 13:21:43 -0700 Message-ID: <20240430202144.49899-9-andrew.boyer@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240430202144.49899-1-andrew.boyer@amd.com> References: <20240419195310.21432-1-andrew.boyer@amd.com> <20240430202144.49899-1-andrew.boyer@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF00026369:EE_|SN7PR12MB8433:EE_ X-MS-Office365-Filtering-Correlation-Id: 10bfd282-7311-449c-ae5a-08dc695338a6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230031|1800799015|36860700004|82310400014|376005; X-Microsoft-Antispam-Message-Info: 9y40rLOsfsl9PL1eqhEoHcoypUG601lYUbAAAii/hUUNjobYqPPESSG70pk99Q0JJ1CCkG1KUrZxM6tqfNcHuJse0gvWhECP25j7LfxadEkxD5XkNqU4spHgIVUG/omRBecs1ym0nkK+/oTelryC1v6n4Wlafnqt2Vhq2qR0+18lDCP6j3+Ja2lOPaFmPpsTIMOa5ECwWwcZ+/XV1j3nqa7C5gAOISn0kkt+yArJ+NCboudFdqkxelf2RM31JqJ9gobrxqnBJXXKVVkXeKszgtV+1tkzi+fkNDJajc6BD17F7l36gRUmORGP4xikw6gSI8D4441hdhWOTeWZ7YFrYe6pEj6M3GXMTo5tGL4pthj8NuYe1kxT/F3P/JITZnPQQyRLjEhj99sHoeFAjrGluSM0gBfUk+8YH6hHGwC5kcwUwzA6nKKu66iV3kKceShuZA+Lc0njqO15tKE5++av6GRC75b7KkwB1DRmXG292DY6wIRxlA3b+oEqSuspS6dGmyMmaJ1EkkXfh/Aq/OnUFcgoRFTDi4lKK14R4q+u0XYO8PAN7xXLBdBAMWltIQhQ0Y8NzMkgqvYhoaZfeMRV85n6vxv6GmMXkqhIUu1QcV8LaNkmfDkqghJndy89PDsegt5tNITtwRtRsI7dSlC+ayldls1MZBpDn9ZXP2itjwxGuKfhlVh6vnYQT19wkeBYEj5sI6vyX4Zf4F3sHT9Ha74mOxhEQ8pB+EjH1eeBXknx4PDVBI9RjgD0t58mSH5TVeCMs1/NjNURhaIoPjr89vGf47wDULN9XnNYkvWQ18xG7PxLxB+jouQpLfmQthYPgBB3EKNUyIXM5MaHjhAgdGzutLFOloHg7Do21OdJxFarMP0XxtQE4tsrZ5UyUAlodA0AS3XPtcON0EvII5SKleMnfVfbP2vYxCSqLpqTujQpoVohsrISi7Qnt5iIncIlztl2aBh6OjL53reXAKAhYPaG+Flhv1vrkBcRg77RuT4fkD4YndeCatd0ojHD+bzy3EbiFspab2o/4lzvDG9u9wx12gcQKOZuck+8yP3T03gDVt2+ESYHy8k/YR1fA9OpN68OEfPX6/I5bM3DqpUdt5Uw7ovbyHvI4RxQf5BRWWPY+LcE1kcRstu4IYmRFS5B3sHJ+3RV+ffD0It2Z3PLAIjSf+mEazA4Qg4kMT6y1sC1KDEHtWGKuuvg4P0axAQ0olxvFaXmLYH+YKJWAQ5Z+dB1ZOt59lSA82z5CBWGE5sXZvxdmkGXy89dkifA9rDb1ZM+QmmE1RyXmSeOZVzxmkis2cmGurXaUmSE+meUytMR8DuzGWL7w6jNKA0EN0vZlMxUvUIV0Tb02n/mV1HPiJgInw+K+dE5u2TZOcHAFGifAAaOX0K6ZDZjm8vrVVtE X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(1800799015)(36860700004)(82310400014)(376005); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Apr 2024 20:22:13.2792 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 10bfd282-7311-449c-ae5a-08dc695338a6 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF00026369.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8433 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org If no progress has been made within the timeout, post a dummy operation using session 0. This will restart the queue in the rare case that a doorbell is lost inside the device. Signed-off-by: Andrew Boyer --- drivers/crypto/ionic/ionic_crypto.h | 16 ++++ drivers/crypto/ionic/ionic_crypto_main.c | 31 ++++++++ drivers/crypto/ionic/ionic_crypto_ops.c | 97 +++++++++++++++++++++++- 3 files changed, 142 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/ionic/ionic_crypto.h b/drivers/crypto/ionic/ionic_crypto.h index e05b458926..69c17887fb 100644 --- a/drivers/crypto/ionic/ionic_crypto.h +++ b/drivers/crypto/ionic/ionic_crypto.h @@ -89,6 +89,14 @@ struct iocpt_dev_bars { uint32_t num_bars; }; +/* Queue watchdog */ +#define IOCPT_Q_WDOG_SESS_IDX 0 +#define IOCPT_Q_WDOG_KEY_LEN 16 +#define IOCPT_Q_WDOG_IV_LEN 12 +#define IOCPT_Q_WDOG_PLD_LEN 4 +#define IOCPT_Q_WDOG_TAG_LEN 16 +#define IOCPT_Q_WDOG_OP_TYPE RTE_CRYPTO_OP_TYPE_UNDEFINED + struct iocpt_qtype_info { uint8_t version; uint8_t supported; @@ -162,7 +170,15 @@ struct iocpt_crypto_q { IOCPT_COMMON_FIELDS; /* cacheline2 */ + uint64_t last_wdog_cycles; uint16_t flags; + + /* cacheline3 */ + uint64_t enqueued_wdogs; + uint64_t dequeued_wdogs; + uint8_t wdog_iv[IOCPT_Q_WDOG_IV_LEN]; + uint8_t wdog_pld[IOCPT_Q_WDOG_PLD_LEN]; + uint8_t wdog_tag[IOCPT_Q_WDOG_TAG_LEN]; }; #define IOCPT_S_F_INITED BIT(0) diff --git a/drivers/crypto/ionic/ionic_crypto_main.c b/drivers/crypto/ionic/ionic_crypto_main.c index 7693377831..113347c57a 100644 --- a/drivers/crypto/ionic/ionic_crypto_main.c +++ b/drivers/crypto/ionic/ionic_crypto_main.c @@ -172,6 +172,22 @@ iocpt_session_write(struct iocpt_session_priv *priv, return 0; } +static int +iocpt_session_wdog(struct iocpt_dev *dev) +{ + struct iocpt_session_priv priv = { + .dev = dev, + .index = IOCPT_Q_WDOG_SESS_IDX, + .type = IOCPT_SESS_AEAD_AES_GCM, + .key_len = IOCPT_Q_WDOG_KEY_LEN, + }; + + /* Reserve session 0 for queue watchdog */ + rte_bitmap_clear(dev->sess_bm, IOCPT_Q_WDOG_SESS_IDX); + + return iocpt_session_write(&priv, IOCPT_SESS_INIT); +} + int iocpt_session_init(struct iocpt_session_priv *priv) { @@ -491,6 +507,9 @@ iocpt_cryptoq_deinit(struct iocpt_crypto_q *cptq) IOCPT_PRINT(DEBUG, "Deinit queue %u returned %d after %u ms", cptq->q.index, err, sleep_cnt * 100); + IOCPT_PRINT(DEBUG, "Queue %u watchdog: enq %"PRIu64" deq %"PRIu64, + cptq->q.index, cptq->enqueued_wdogs, cptq->dequeued_wdogs); + cptq->flags &= ~IOCPT_Q_F_INITED; } @@ -659,9 +678,21 @@ iocpt_init(struct iocpt_dev *dev) if (err != 0) return err; + /* Write the queue watchdog key */ + err = iocpt_session_wdog(dev); + if (err != 0) { + IOCPT_PRINT(ERR, "Cannot setup watchdog session"); + goto err_out_adminq_deinit; + } + dev->state |= IOCPT_DEV_F_INITED; return 0; + +err_out_adminq_deinit: + iocpt_adminq_deinit(dev); + + return err; } void diff --git a/drivers/crypto/ionic/ionic_crypto_ops.c b/drivers/crypto/ionic/ionic_crypto_ops.c index 28b099dea2..0330fd76ad 100644 --- a/drivers/crypto/ionic/ionic_crypto_ops.c +++ b/drivers/crypto/ionic/ionic_crypto_ops.c @@ -58,7 +58,8 @@ iocpt_op_info_get(struct rte_cryptodev *cdev, struct rte_cryptodev_info *info) info->max_nb_queue_pairs = dev->max_qps; info->feature_flags = dev->features; info->capabilities = iocpt_get_caps(info->feature_flags); - info->sym.max_nb_sessions = dev->max_sessions; + /* Reserve one session for watchdog */ + info->sym.max_nb_sessions = dev->max_sessions - 1; info->driver_id = dev->driver_id; info->min_mbuf_headroom_req = 0; info->min_mbuf_tailroom_req = 0; @@ -380,12 +381,72 @@ iocpt_enqueue_sym(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) count++; } - if (likely(count > 0)) + if (likely(count > 0)) { iocpt_q_flush(&cptq->q); + /* Restart timer if ops are being enqueued */ + cptq->last_wdog_cycles = rte_get_timer_cycles(); + } + return count; } +static void +iocpt_enqueue_wdog(struct iocpt_crypto_q *cptq) +{ + struct iocpt_queue *q = &cptq->q; + struct iocpt_crypto_desc *desc, *desc_base = q->base; + struct iocpt_crypto_sg_desc *sg_desc, *sg_desc_base = q->sg_base; + struct iocpt_crypto_sg_elem *src; + struct rte_crypto_op *wdog_op; + rte_iova_t iv_addr, pld_addr, tag_addr; + uint8_t nsge_src = 0; + uint16_t avail; + + avail = iocpt_q_space_avail(&cptq->q); + if (avail < 1) + goto out_flush; + + wdog_op = rte_zmalloc_socket("iocpt", sizeof(*wdog_op), + RTE_CACHE_LINE_SIZE, rte_socket_id()); + if (wdog_op == NULL) + goto out_flush; + + wdog_op->type = IOCPT_Q_WDOG_OP_TYPE; + wdog_op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; + + desc = &desc_base[q->head_idx]; + sg_desc = &sg_desc_base[q->head_idx]; + src = sg_desc->src_elems; + + /* Fill the first SGE with the IV / Nonce */ + iv_addr = rte_mem_virt2iova(cptq->wdog_iv); + iocpt_fill_sge(src, nsge_src++, iv_addr, IOCPT_Q_WDOG_IV_LEN); + + /* Fill the second SGE with the payload segment */ + pld_addr = rte_mem_virt2iova(cptq->wdog_pld); + iocpt_fill_sge(src, nsge_src++, pld_addr, IOCPT_Q_WDOG_PLD_LEN); + + /* AEAD AES-GCM: digest == authentication tag */ + tag_addr = rte_mem_virt2iova(cptq->wdog_tag); + iocpt_fill_sge(src, nsge_src++, tag_addr, IOCPT_Q_WDOG_TAG_LEN); + + desc->opcode = IOCPT_DESC_OPCODE_GCM_AEAD_ENCRYPT; + desc->flags = 0; + desc->num_src_dst_sgs = iocpt_encode_nsge_src_dst(nsge_src, 0); + desc->session_tag = rte_cpu_to_le_32(IOCPT_Q_WDOG_SESS_IDX); + + q->info[q->head_idx] = wdog_op; + q->head_idx = Q_NEXT_TO_POST(q, 1); + + IOCPT_PRINT(DEBUG, "Queue %u wdog enq %p", + q->index, wdog_op); + cptq->enqueued_wdogs++; + +out_flush: + iocpt_q_flush(q); +} + static uint16_t iocpt_dequeue_sym(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) { @@ -395,6 +456,7 @@ iocpt_dequeue_sym(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) struct rte_crypto_op *op; struct iocpt_crypto_comp *cq_desc_base = cq->base; volatile struct iocpt_crypto_comp *cq_desc; + uint64_t then, now, hz, delta; uint16_t count = 0; cq_desc = &cq_desc_base[cq->tail_idx]; @@ -442,6 +504,17 @@ iocpt_dequeue_sym(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) op->status == RTE_CRYPTO_OP_STATUS_NOT_PROCESSED) break; + /* Handle watchdog operations */ + if (unlikely(op->type == IOCPT_Q_WDOG_OP_TYPE)) { + IOCPT_PRINT(DEBUG, "Queue %u wdog deq %p st %d", + q->index, op, op->status); + q->info[q->tail_idx] = NULL; + q->tail_idx = Q_NEXT_TO_SRVC(q, 1); + cptq->dequeued_wdogs++; + rte_free(op); + continue; + } + ops[count] = op; q->info[q->tail_idx] = NULL; @@ -449,6 +522,26 @@ iocpt_dequeue_sym(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) count++; } + if (!count) { + /* + * Ring the doorbell again if no work was dequeued and work + * is still pending after the deadline. + */ + if (q->head_idx != q->tail_idx) { + then = cptq->last_wdog_cycles; + now = rte_get_timer_cycles(); + hz = rte_get_timer_hz(); + delta = (now - then) * 1000; + + if (delta >= hz * IONIC_Q_WDOG_MS) { + iocpt_enqueue_wdog(cptq); + cptq->last_wdog_cycles = now; + } + } + } else + /* Restart timer if the queue is making progress */ + cptq->last_wdog_cycles = rte_get_timer_cycles(); + return count; }