From patchwork Thu Sep 21 08:34:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Hu X-Patchwork-Id: 131761 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 92C9542606; Thu, 21 Sep 2023 10:35:37 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7C8F340A6C; Thu, 21 Sep 2023 10:35:36 +0200 (CEST) Received: from BN3PR00CU001.outbound.protection.outlook.com (mail-eastus2azon11020016.outbound.protection.outlook.com [52.101.56.16]) by mails.dpdk.org (Postfix) with ESMTP id 7EEF14014F; Thu, 21 Sep 2023 10:35:34 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ALsRcLjxSmQmhZnwzNyuaJ5hl91UFkKWvkWdAmzLHkBfJRZetC1FvG9ZSNVn85IQ0Lj2d+otN5sTw5RuThUPhD1qvFc9NEQRQ12xRp6EaU7YsV8CZZ/NkJxkSk/J5F29YtQBqvi216z6l/nJGfgRIOX0t7t+8IEgfFyzYT03NM8L8SEInEfOS+8uuPRbHVBXjgN2x0i9oyNEe7+87DeDdRGdNoOOcJLBX09n/LNcK+2ghKlb2IcpJe8uNr8w1AQovpEkyKfMolbufDr7HGSuzAg9M9uUguiUtktPBm5Cd61D4lxctIPbAIDvnvyYflQ1n9FmzO/wJ0KdF4VMhSWaFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7SUbOK+U9W7fEH5UmIfFvY8OKNtiQLwnBEMAETcVVDQ=; b=XcyewNkm/Q0agLVWNnue93Q8Z80BHRsC89Dl07nKm1eGi2k1pJql0qIlODbXT5OWEyl0Rn2zLrcRHVqirerMOR8xa4ngcrcIgRqn9BB+qyoE76CxwgifuzNXrtxNKN5f5kUXQF/GH8Zda8vx77Ld+mOlpAaleHeJYLAETTxifaa7DTNyCVGGQ2XkJouB3oQNbD8cjLPREPNzRnsScrKu56u1vouiRRLzOr5qPIB3QZcYQTM39RjffcfH0ktKsJ1GxUsIIKEtPfKOyc+mZyEOmPY8qldBkkGBpC9M+83dV34XZCUdWAt99xH9ko+MAoYubJnSltv6MKLF3nPzM8M8tg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7SUbOK+U9W7fEH5UmIfFvY8OKNtiQLwnBEMAETcVVDQ=; b=PvnpBRerTrW+6xxYD8g50KqD1Ow2w+J7WZxwVuUZis4M9kbdVlP9UP7fZ2MuZaxMUMGiPWkzKgk2/IDaQ+XmPmVdQA26Q7kwofGiTtjLw83SBipGCS/Ydo1Ovn6uOxBGn7bUsS+A5MDIW46NlXhSfoHHLsrmztgtWxyiA+ZEIVU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=microsoft.com; Received: from MN2PR21MB1454.namprd21.prod.outlook.com (2603:10b6:208:208::11) by LV2PR21MB3086.namprd21.prod.outlook.com (2603:10b6:408:17c::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.4; Thu, 21 Sep 2023 08:35:32 +0000 Received: from MN2PR21MB1454.namprd21.prod.outlook.com ([fe80::cbd6:26c1:4d57:b9fc]) by MN2PR21MB1454.namprd21.prod.outlook.com ([fe80::cbd6:26c1:4d57:b9fc%5]) with mapi id 15.20.6838.010; Thu, 21 Sep 2023 08:35:32 +0000 From: Wei Hu To: dev@dpdk.org, ferruh.yigit@amd.com, ktraynor@redhat.com, xuemingl@nvidia.com, bluca@debian.org, Long Li Cc: Wei Hu , stable@dpdk.org Subject: [PATCH 1/2] net/mana: enable 32 bit build for mana driver Date: Thu, 21 Sep 2023 08:34:41 +0000 Message-Id: <20230921083442.2104530-2-weh@microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230921083442.2104530-1-weh@microsoft.com> References: <20230921083442.2104530-1-weh@microsoft.com> X-ClientProxiedBy: PH0P220CA0020.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:d3::34) To MN2PR21MB1454.namprd21.prod.outlook.com (2603:10b6:208:208::11) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR21MB1454:EE_|LV2PR21MB3086:EE_ X-MS-Office365-Filtering-Correlation-Id: aed9b59a-4df6-4d8b-1cd7-08dbba7db7cd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: p65Y2EefYYNx1Qh3CneCPFCbcrY/SYbRy8p1KNRRIZS0nf4PPdXeaLvpyQU/iXLVj8js0BIGy4f3UBqv5DisgM9HdULX+7T562SQFAMbvLZ5wAB++IiQEv8NerI7V57eBccOVIxjLsaUFS/nD/FadcM1HoOVILGGvNK7OwfHMWzNM866CTGdrOQ9X4t2MVV6XH/xZ9r+2dGStlwrMFa3P4BNfghcuY/xB27N9aRAj3BAUT5PQXgNqF2ZKAuU4EroZgtzgJUWbZEfN3N5nXpQfzy/HqmyiuZ/kjNz3GwKZqDMWqvlPfOh98ngHs0OVokfkVb07DtEJWJ7j70pRiST1KH+G8I75aCXbB8PiCQ7K3J4Yb3Hz/RvVNJjqRqMf8Olu/EXam14MIedWwHOSiTXblE66xReWi6U0vJCs84QzLbVAHeAzXypff297ZGU7n/B34vviUa3e9XttR5HLJXmrbbo8ESGjUlV4EagpBiNYl3L1V0p/gSEHx0eEezE2nUo1qfl29SRtSXZ56Q7jqMYPBfUGPv3QJjD+6ES0Zm1Ez53oB9mgrYj7MpbyrsqjMFvckW1Xd+sCcmc7bMrcn6Z6fmKAMwNVIMb6HvkBaiBusltq76lNPnEj6EXlO4ZHzvMwJQAUXrV0Br6DE5sXvUQvGua+VGX7Y2XuiuvMZUmWOA= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR21MB1454.namprd21.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(136003)(396003)(366004)(39860400002)(376002)(346002)(186009)(451199024)(1800799009)(36756003)(86362001)(8676002)(26005)(66946007)(82950400001)(37006003)(38350700002)(478600001)(82960400001)(10290500003)(38100700002)(8936002)(41300700001)(6636002)(316002)(6506007)(2616005)(6512007)(52116002)(6666004)(66556008)(1076003)(4326008)(66476007)(2906002)(6862004)(6486002)(83380400001)(5660300002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: gu5qFiPfmmv1+bSac7vPDf+rX8UsszTho3ED4bmjBL0LE25xSqoDh8lv/ISvH+CziXyeWqbI7JrGudf2Z9GIDU2LGvpjFZmfAcQIOj2kOBZUWteCkWXbUbHLc+PLYc8A6WxsWBmg+m9P/hAaGNsxZw7YsI30zPBoi8YWZ6qs8J1FgLZZP1jpICAd3KhM911gSac+P9ItwkFAQ8HI3vea2tmd2PkEPclbCMFrRtn/2Otj/076v7PugibO5a/yjJHNl3frfaEY8t6ZFIVFdlCxcm1miMJwK2TRglVTy7+6G5f0ZWOzT1kJTNkcjZaNj9LXbsnDEYYgfnlWXCn9xAjqOFZ+SQE7M2faKYMs1R3FFICL0+Y1tkkTtvgOQIzrhE3kGy2k/svXoBnVR6AbaE6YYG1bs0cvKtrrk2DGsLOVxIrvU0xmfAJQirm0YECPf6mPlh+V08ictPw9xUMEQtGyKzxF0be0kvorLyjWyruCe17T8Mn6sAPWfgDZLLmzZPecCzKEDinBtxqZV1whNz2950FyCTwfMH95x1Edt4ndvEp2EnIilQ1q+tDm6lHtD2NF8D7KojtI+xlQvgXcI3EX0922xg6meQ2yrSwbh/DZEuEmbFjDsdwqVD1c56cmRKMVDS+VLGzxik++GOkpZ1QaZORE5s7O1Bf+dMORa8EbKSBYV2BCpQw62TtsE6cki0HeV8NYj8s7S92OYJVjvoMaO+X1vZLfB6sL+jMlXqThf97YAeh6yg0EaI9MpYs0U7iwVqXHWIB7vzeeyYdk6gkugHTT8J1wkWUZP+V4rTvb2bF22IjeNZbAkzlw5ueJsNIV42w7Gk7P66DDi1eTfRUm7QUjKwxrkRD4GDfAQmYk3DhigBs+f6LLlt+UqCjk0G70ZkD3C33IJSNfC4ryLgL2QuUDdRzhR8yYJW9Q1fiIFepOQ3sdzvsZp0g9FEjXz8bCBCTayp6LeQ9yn4iwPterrSzXxmCKmuoWrPFVdh/8aYhRguq4I0sL1enpuUk8c0wJ3Ehly5Kw1N4irCbdTOzAOLUl1EO4D9mJ3dmF5wuGguGM2RASMTxBcReHutDLlp8r6/2neBmAgIElYwXYl1y4iKXTLUsl34FKAuwcD4Eul1bLnY7LmKmdwFUDMjWmZuF74kT29RY4bKJO8VMLTYBkCWCBq+MVzqEz1aF6YbzsoBTnqsz8z1eGziBhthUPKkJcLBcnukc8FAbp/rdi0drpEIzOF5HqZ/6EZvm3d70sSwqaz5qytXJ/s6mq4N+fKcHDfFAFRcgiQSYdaRRWc7rbQ+0GbmGMlgBJ43qgvEJAAvLK/PIt1+zqw309qJVji8gvdSJLP1RxHXgKMBufjOkUGQAgsnAAXxIyp30qsdhp51qjlAvj8mR6vtnyuU85npOXfjTFfezmxzgISk4EYQnE3+Wfu4kvYZP0FUEcygosbuQdYG8k5km3uE+c5s9vp5l+qUVrw8Ogjof8KB7zYgZ8ucoKm1pbut52nG3ACx7+c+2f6oaIHYu+vtPDy/v0hsnk3rHmxtUG+tio7ux1himkDlrSBZRSlSL0We2DMot9FBdU7c/Pibr//Kx4eWlULroq X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: aed9b59a-4df6-4d8b-1cd7-08dbba7db7cd X-MS-Exchange-CrossTenant-AuthSource: MN2PR21MB1454.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2023 08:35:32.1697 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pxCdOFzqAC0+utHsJ72xoK1QJ1Zws4dTVo+cJv77bVOApfNJ7FqtXTZW6BjYjblgpiSrgcuUoCCjTcT1/39KiA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR21MB3086 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Enable 32 bit build on x86 Linux. Fixed build warnings and errors when building in 32 bit. With this patch, mana will be able to build into 32 bit. However, another patch for mana short doorbell support is needed to make mana fully functional for 32 bit applicatons. Cc: stable@dpdk.org Signed-off-by: Wei Hu Acked-by: Long Li --- drivers/net/mana/mana.c | 2 +- drivers/net/mana/meson.build | 4 ++-- drivers/net/mana/mr.c | 18 +++++++++--------- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/drivers/net/mana/mana.c b/drivers/net/mana/mana.c index 7630118d4f..896b53ed35 100644 --- a/drivers/net/mana/mana.c +++ b/drivers/net/mana/mana.c @@ -1260,7 +1260,7 @@ mana_probe_port(struct ibv_device *ibdev, struct ibv_device_attr_ex *dev_attr, /* Create a parent domain with the port number */ attr.pd = priv->ib_pd; attr.comp_mask = IBV_PARENT_DOMAIN_INIT_ATTR_PD_CONTEXT; - attr.pd_context = (void *)(uint64_t)port; + attr.pd_context = (void *)(uintptr_t)port; priv->ib_parent_pd = ibv_alloc_parent_domain(ctx, &attr); if (!priv->ib_parent_pd) { DRV_LOG(ERR, "ibv_alloc_parent_domain failed port %d", port); diff --git a/drivers/net/mana/meson.build b/drivers/net/mana/meson.build index 493f0d26d4..2d72eca5a8 100644 --- a/drivers/net/mana/meson.build +++ b/drivers/net/mana/meson.build @@ -1,9 +1,9 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022 Microsoft Corporation -if not is_linux or not dpdk_conf.has('RTE_ARCH_X86_64') +if not is_linux or not dpdk_conf.has('RTE_ARCH_X86') build = false - reason = 'only supported on x86_64 Linux' + reason = 'only supported on x86 Linux' subdir_done() endif diff --git a/drivers/net/mana/mr.c b/drivers/net/mana/mr.c index fec0dc961c..b8e6ea0bbf 100644 --- a/drivers/net/mana/mr.c +++ b/drivers/net/mana/mr.c @@ -53,7 +53,7 @@ mana_new_pmd_mr(struct mana_mr_btree *local_tree, struct mana_priv *priv, } DP_LOG(DEBUG, - "registering memory chunk start 0x%" PRIx64 " len %u", + "registering memory chunk start 0x%" PRIxPTR " len %u", ranges[i].start, ranges[i].len); if (rte_eal_process_type() == RTE_PROC_SECONDARY) { @@ -62,7 +62,7 @@ mana_new_pmd_mr(struct mana_mr_btree *local_tree, struct mana_priv *priv, ranges[i].len); if (ret) { DP_LOG(ERR, - "MR failed start 0x%" PRIx64 " len %u", + "MR failed start 0x%" PRIxPTR " len %u", ranges[i].start, ranges[i].len); return ret; } @@ -72,7 +72,7 @@ mana_new_pmd_mr(struct mana_mr_btree *local_tree, struct mana_priv *priv, ibv_mr = ibv_reg_mr(priv->ib_pd, (void *)ranges[i].start, ranges[i].len, IBV_ACCESS_LOCAL_WRITE); if (ibv_mr) { - DP_LOG(DEBUG, "MR lkey %u addr %p len %" PRIu64, + DP_LOG(DEBUG, "MR lkey %u addr %p len %zu", ibv_mr->lkey, ibv_mr->addr, ibv_mr->length); mr = rte_calloc("MANA MR", 1, sizeof(*mr), 0); @@ -99,7 +99,7 @@ mana_new_pmd_mr(struct mana_mr_btree *local_tree, struct mana_priv *priv, return ret; } } else { - DP_LOG(ERR, "MR failed at 0x%" PRIx64 " len %u", + DP_LOG(ERR, "MR failed at 0x%" PRIxPTR " len %u", ranges[i].start, ranges[i].len); return -errno; } @@ -141,7 +141,7 @@ mana_find_pmd_mr(struct mana_mr_btree *local_mr_btree, struct mana_priv *priv, mr = mana_mr_btree_lookup(local_mr_btree, &idx, (uintptr_t)mbuf->buf_addr, mbuf->buf_len); if (mr) { - DP_LOG(DEBUG, "Local mr lkey %u addr 0x%" PRIx64 " len %" PRIu64, + DP_LOG(DEBUG, "Local mr lkey %u addr 0x%" PRIxPTR " len %zu", mr->lkey, mr->addr, mr->len); return mr; } @@ -162,7 +162,7 @@ mana_find_pmd_mr(struct mana_mr_btree *local_mr_btree, struct mana_priv *priv, } DP_LOG(DEBUG, - "Added local MR key %u addr 0x%" PRIx64 " len %" PRIu64, + "Added local MR key %u addr 0x%" PRIxPTR " len %zu", mr->lkey, mr->addr, mr->len); return mr; } @@ -266,7 +266,7 @@ mana_mr_btree_lookup(struct mana_mr_btree *bt, uint16_t *idx, return &table[base]; DP_LOG(DEBUG, - "addr 0x%" PRIx64 " len %zu idx %u sum 0x%" PRIx64 " not found", + "addr 0x%" PRIxPTR " len %zu idx %u sum 0x%" PRIxPTR " not found", addr, len, *idx, addr + len); return NULL; @@ -316,7 +316,7 @@ mana_mr_btree_insert(struct mana_mr_btree *bt, struct mana_mr_cache *entry) uint16_t shift; if (mana_mr_btree_lookup(bt, &idx, entry->addr, entry->len)) { - DP_LOG(DEBUG, "Addr 0x%" PRIx64 " len %zu exists in btree", + DP_LOG(DEBUG, "Addr 0x%" PRIxPTR " len %zu exists in btree", entry->addr, entry->len); return 0; } @@ -340,7 +340,7 @@ mana_mr_btree_insert(struct mana_mr_btree *bt, struct mana_mr_cache *entry) bt->len++; DP_LOG(DEBUG, - "Inserted MR b-tree table %p idx %d addr 0x%" PRIx64 " len %zu", + "Inserted MR b-tree table %p idx %d addr 0x%" PRIxPTR " len %zu", table, idx, entry->addr, entry->len); return 0; From patchwork Thu Sep 21 08:34:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Hu X-Patchwork-Id: 131762 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A425E42606; Thu, 21 Sep 2023 10:35:46 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0230940A8B; Thu, 21 Sep 2023 10:35:40 +0200 (CEST) Received: from BN3PR00CU001.outbound.protection.outlook.com (mail-eastus2azon11020022.outbound.protection.outlook.com [52.101.56.22]) by mails.dpdk.org (Postfix) with ESMTP id C565C40A79; Thu, 21 Sep 2023 10:35:38 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kARQFq+oCVq2AyYBcP16QHApsDjMgRY5lloCP96r6RxuQ1Tdu2WWss6L/Fa4MlSSecKVD/CToBHZq6pgn5w5Bgyfn1c6uZWXdXy9/oA9VgtHExHtp+7QsqNCywcs2pIur9185IzZu26JHz7x6CRou07qrO9gRKArmfMVysoHOG7T2WEAbv3EjshM5S+Q8iVBq5y63w0WOOtfOrYpYZ953xuJKHROveWN/SPbwMdmVCxzastOdHfg1Wo7h76JTIgzvvgUe1JPBeGkF26dA0sP/wIWxj+IqXEAOFRn2Jyv6CJLVn1ZSYN6PEQ8bUDWJ+Ai2B61txHbFAYnZ7S6B+0Fgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yucAq9XOa+q+ZquQ7NqHPYIRDTI8EqsyjtoE/GGfqU4=; b=GUXB6GgXeA596lN8E7YP6nvr+DFqwCTDDwNRa3gcr7R1yqyuwOXKW5a/ju3mBNhlRTNFIJzR66bRWWmoE2x6C7g8Tcg/WTEyEDa1MFiJXafKNJVjwBUAU82DbPI3LWeqmt/rfLQV1+dRpw+rv3Nc7yR8foYEAwPjUyVpfOcCpkwTlo4cIZO7eSJo23p5sC7N7aIJUGEk1ZOVS6rBwP6aHHG4FM2tEgetiOrs50TmSp3kXpRbg3VkKkJ6dg9PlGsyd+2xEvq83vdJywRMc0qBiTP9qiwycqxMPnQ8T6Jjp7FMmfVaujQSFsHKZEYXpNX/vKJzg/RVpieBdGs4ekglWA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yucAq9XOa+q+ZquQ7NqHPYIRDTI8EqsyjtoE/GGfqU4=; b=IhAYltcNfqxFckxF+ZhYOxjyV/r20NCV36aZu92YtTex6PEthJsgL4HKGbKxz6bA4qD6NVCAoaHGd+xKiQ2eo06F8+GmBpH4PGyJSu1L5b1i9BJIeN60AU7nEm/UMjS3rrq9PnlJ81n4el/ogFkgFlE8mjZhabfKS/D2Dur5zRA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=microsoft.com; Received: from MN2PR21MB1454.namprd21.prod.outlook.com (2603:10b6:208:208::11) by LV2PR21MB3086.namprd21.prod.outlook.com (2603:10b6:408:17c::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.4; Thu, 21 Sep 2023 08:35:36 +0000 Received: from MN2PR21MB1454.namprd21.prod.outlook.com ([fe80::cbd6:26c1:4d57:b9fc]) by MN2PR21MB1454.namprd21.prod.outlook.com ([fe80::cbd6:26c1:4d57:b9fc%5]) with mapi id 15.20.6838.010; Thu, 21 Sep 2023 08:35:36 +0000 From: Wei Hu To: dev@dpdk.org, ferruh.yigit@amd.com, ktraynor@redhat.com, xuemingl@nvidia.com, bluca@debian.org, Long Li Cc: Wei Hu , stable@dpdk.org Subject: [PATCH 2/2] net/mana: add 32 bit short doorbell Date: Thu, 21 Sep 2023 08:34:42 +0000 Message-Id: <20230921083442.2104530-3-weh@microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230921083442.2104530-1-weh@microsoft.com> References: <20230921083442.2104530-1-weh@microsoft.com> X-ClientProxiedBy: PH0P220CA0020.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:d3::34) To MN2PR21MB1454.namprd21.prod.outlook.com (2603:10b6:208:208::11) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR21MB1454:EE_|LV2PR21MB3086:EE_ X-MS-Office365-Filtering-Correlation-Id: a55f3314-4a88-4e46-fd89-08dbba7dba76 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +KvJ6EgLTVsNaxjQjmLuZ5+5x/l3mih0ZLZqBj6rgOFoW+BGMlEsmk21rkZi88ALfgdpyjgdHmPVH/EAcqLyVumeUilfxbpnqOQL9gzeb6Yak4nfY0CjwqJau3OPgMnzB/zp3jwnJWIgyIiPUt/r1phedfR9WYZgbeMNJzuoAiOTxr2LDwvsvuEtT8lvtYdv13eHZumkblN9kjTniiWBfdSy6FiZGIlG28Aq/Bdtm98XzEu2e88wFKgpx+qyhvMuLHJ463iKMobkjj9xb10WLgwzOiRf/vxeAetJJukZiptNHSaDh48AtulWzATzrxf+Gr5coF7/vO1ZmlkzshJSupFy8swG2J1aiv5UTxGdwP4MTinazzuqRusA+VFJMrj1jUylQTjozBnPdVlkV1yDF/+mw610gGyjztt/69RZxlhJkExAGvqC2d+s0PlsItjsNRKzH9o9JJQhHSWkVFlKPNY1T/HPA1m7KWkOo+uesD9NTbjiULd3YaFqO22WMjwtVxgMTtatZO+DVUir/dcZZvDDjJkotSu6qpVgkQIa3ShCYSpcJI0ff3cCLi5/rxuodvfUWIc+URjVeDUNJcXU0AkATS9tjMv1MxfyuAzkYJEPYtaNd/RBof5VgFHURJ9mraNjCGQPXEYaUPatt0pDTJqptiKD97YlhYPfdO6aNk4= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR21MB1454.namprd21.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(136003)(396003)(366004)(39860400002)(376002)(346002)(186009)(451199024)(1800799009)(36756003)(86362001)(8676002)(26005)(66946007)(82950400001)(37006003)(38350700002)(478600001)(82960400001)(10290500003)(38100700002)(8936002)(41300700001)(6636002)(316002)(6506007)(2616005)(6512007)(52116002)(6666004)(66556008)(1076003)(4326008)(66476007)(2906002)(6862004)(6486002)(83380400001)(5660300002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: DwvqOxan3wlnHRXpuPkfjllPNIAyh7BXSWeb4G+qbVm8EgjGdhDCAIUBd1yw3nNEo7rtkVuDuJDI4Cp0UsrO+S1VkLS+SQe6gHr/cke7ANDHyBXPpo4uCy69mgwFsYP+WNqqqWrGJEonF+MwV9OEWRDG1zx8yJiFc3l99bUdpKZ2MA7IZ6/5KU1xz1131O1yFJci0aXOocC0/yKX+t+SCK6tg/YIv5tiH6WQlKo2btAZ+eR615GBCV9k9dXLuDPxI9coBSsgj1PJe0prm9KylO91ThyHzpRakgsfQ9m1k8Lcb5qKzFK6o/63fk9T4Co/uRvovzeTqBpfziLgNetk6XVYluyBWYk+JEBH+h4xokkiv5eCeOn0mI+3sq46kN0JrAGwtVfj0LjOhEvID0J4MjkeVi2EEluWbGoNm98MjDUEkQ/vGAOLH7UQOIYaOx5PahEmtddIeaI9PDHvrdcnW5ZQNbwkRdgZwPUg2lqSEPb1yKdnFNTpZhu67KUE6d9so0Foe9taebJyxM4FQJyXSjmixrrP8pQDWeAIZ9F5du1FHUxXGriitPT+u1doQygk+Xv4EIrW53IpSiqrHeBKi/HVvEZH9OMk31CmC2x+fSPTvo2EituyycoWNCGgqg8EQ893FM5W5vLwFRv7i2aNgCxmvvYcwxKtL6en6LewWhAdV+vWIpUojU1ZO04xo0ar6iF+Q6FIF9AchdnQUAcxnAnZoG5x/vkLZrgOU+siXjGbjgo0ryD9c316jJfxbkXLjdtSwP2W5zjtfnNUD5MwYh4ruQZDRZuYUFNtkNtGLCqzh3dKB0zK5jP0KTgW/LVX44FGrgG2os1UvsOXA1806w3N0UOHRJ0jaBJpoD7IrD1yKXu2wyFA5HbJKMq/HkvPtVUcOPkvRKPK95TmfIn6v1tIlRTsy1pXjQTws9J248dqHAFthQiA8CQLjBCx2I1vC4efbc5bPTd/LjXzIB77mU3uttKTkfAZ+Dtj9jmEOICnoy5v6pEgXNGebxmpY+fP8bsE4pHRfhsqf0EiGzKGQFa3d0QOmvKWEs7obHF7bXOgjcEUpjXUu8TPSIu1YLlbFVauegXhDeZVMKoSoJPy1l27XGEJKhdJBuI/JJ4vXU7wJK7lMo872WKmRsJjKfaAB8TXK+tGuuRkYaavBNxVVw83RvsD/S2PX0ctdgwlizj+X1yzUQGzxX6R86Lccl7ZdCSeL+0/1qkvuLznvg7fV8N22q+cs/oRrGwI9sF33zcLyHjw0gGLYu+14A4GYfR9zB2p3mZzMsS3v5cIYO4/KsuAbyQdSQC6R38tEp9kQP3G8nbnlVwCG9roRCCzgfaiEjgIQXzP8QFASAH7IETNAm3CYDW/kcDPBnQX+heKr91oLlzlXAmHJSvsqUcNdiRPZXEXc4Sv2xn2rWofRAdC88KYs1O2Atdb6WpWr/NE7AsCGnmax7KBW6O8dC385IFCgEHA1TSvgrWQA8FV+eHLJbL0MF+7pD/gEeZJaURKiH6yWbih4uSGxKtIi277sS+AIAnrAY8EvklfjJTDFtN3kxu2ay/ZTz4X5YfqQQllxOWJM0YNY1n0PbYBfoHnhha0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: a55f3314-4a88-4e46-fd89-08dbba7dba76 X-MS-Exchange-CrossTenant-AuthSource: MN2PR21MB1454.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2023 08:35:36.6254 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: alQ19dq0ZELTx3VMEVUQSWKBaMFUDy721xNV8A4Sd6DBtOkUi573NvKPBL4/KI0j3D15xIoT7StWv+bWzCqcXw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR21MB3086 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add 32 bit short doorbell support. Ring short doorbell when running in 32 bit applicactions. Both 32 bit and 64 bit doorbells are supported by mana hardware on same platform. 32 bit applications cannot use 64 bit doorbells. 64 bit applications can use 32 bit doorbells, however the performance would greatly suffer and it is not recommended. Cc: stable@dpdk.org Signed-off-by: Wei Hu Acked-by: Long Li --- drivers/net/mana/gdma.c | 92 +++++++++++++++++++++++++++++++++++++++++ drivers/net/mana/mana.h | 26 ++++++++++++ drivers/net/mana/rx.c | 45 ++++++++++++++++++++ drivers/net/mana/tx.c | 25 +++++++++++ 4 files changed, 188 insertions(+) diff --git a/drivers/net/mana/gdma.c b/drivers/net/mana/gdma.c index 65685fe236..7f66a7a7cf 100644 --- a/drivers/net/mana/gdma.c +++ b/drivers/net/mana/gdma.c @@ -166,6 +166,97 @@ gdma_post_work_request(struct mana_gdma_queue *queue, return 0; } +#ifdef RTE_ARCH_32 +union gdma_short_doorbell_entry { + uint32_t as_uint32; + + struct { + uint32_t tail_ptr_incr : 16; /* Number of CQEs */ + uint32_t id : 12; + uint32_t reserved : 3; + uint32_t arm : 1; + } cq; + + struct { + uint32_t tail_ptr_incr : 16; /* In number of bytes */ + uint32_t id : 12; + uint32_t reserved : 4; + } rq; + + struct { + uint32_t tail_ptr_incr : 16; /* In number of bytes */ + uint32_t id : 12; + uint32_t reserved : 4; + } sq; + + struct { + uint32_t tail_ptr_incr : 16; /* Number of EQEs */ + uint32_t id : 12; + uint32_t reserved : 3; + uint32_t arm : 1; + } eq; +}; /* HW DATA */ + +enum { + DOORBELL_SHORT_OFFSET_SQ = 0x10, + DOORBELL_SHORT_OFFSET_RQ = 0x410, + DOORBELL_SHORT_OFFSET_CQ = 0x810, + DOORBELL_SHORT_OFFSET_EQ = 0xFF0, +}; + +/* + * Write to hardware doorbell to notify new activity. + */ +int +mana_ring_short_doorbell(void *db_page, enum gdma_queue_types queue_type, + uint32_t queue_id, uint32_t tail_incr, uint8_t arm) +{ + uint8_t *addr = db_page; + union gdma_short_doorbell_entry e = {}; + + if ((queue_id & ~GDMA_SHORT_DB_QID_MASK) || + (tail_incr & ~GDMA_SHORT_DB_INC_MASK)) { + DP_LOG(ERR, "%s: queue_id %u or " + "tail_incr %u overflowed, queue type %d", + __func__, queue_id, tail_incr, queue_type); + return -EINVAL; + } + + switch (queue_type) { + case GDMA_QUEUE_SEND: + e.sq.id = queue_id; + e.sq.tail_ptr_incr = tail_incr; + addr += DOORBELL_SHORT_OFFSET_SQ; + break; + + case GDMA_QUEUE_RECEIVE: + e.rq.id = queue_id; + e.rq.tail_ptr_incr = tail_incr; + addr += DOORBELL_SHORT_OFFSET_RQ; + break; + + case GDMA_QUEUE_COMPLETION: + e.cq.id = queue_id; + e.cq.tail_ptr_incr = tail_incr; + e.cq.arm = arm; + addr += DOORBELL_SHORT_OFFSET_CQ; + break; + + default: + DP_LOG(ERR, "Unsupported queue type %d", queue_type); + return -1; + } + + /* Ensure all writes are done before ringing doorbell */ + rte_wmb(); + + DP_LOG(DEBUG, "db_page %p addr %p queue_id %u type %u tail %u arm %u", + db_page, addr, queue_id, queue_type, tail_incr, arm); + + rte_write32(e.as_uint32, addr); + return 0; +} +#else union gdma_doorbell_entry { uint64_t as_uint64; @@ -248,6 +339,7 @@ mana_ring_doorbell(void *db_page, enum gdma_queue_types queue_type, rte_write64(e.as_uint64, addr); return 0; } +#endif /* * Poll completion queue for completions. diff --git a/drivers/net/mana/mana.h b/drivers/net/mana/mana.h index 5801491d75..74e37706be 100644 --- a/drivers/net/mana/mana.h +++ b/drivers/net/mana/mana.h @@ -50,6 +50,21 @@ struct mana_shared_data { #define MAX_TX_WQE_SIZE 512 #define MAX_RX_WQE_SIZE 256 +/* For 32 bit only */ +#ifdef RTE_ARCH_32 +#define GDMA_SHORT_DB_INC_MASK 0xffff +#define GDMA_SHORT_DB_QID_MASK 0xfff + +#define GDMA_SHORT_DB_MAX_WQE (0x10000 / GDMA_WQE_ALIGNMENT_UNIT_SIZE) + +#define TX_WQE_SHORT_DB_THRESHOLD \ + (GDMA_SHORT_DB_MAX_WQE - \ + (MAX_TX_WQE_SIZE / GDMA_WQE_ALIGNMENT_UNIT_SIZE)) +#define RX_WQE_SHORT_DB_THRESHOLD \ + (GDMA_SHORT_DB_MAX_WQE - \ + (MAX_RX_WQE_SIZE / GDMA_WQE_ALIGNMENT_UNIT_SIZE)) +#endif + /* Values from the GDMA specification document, WQE format description */ #define INLINE_OOB_SMALL_SIZE_IN_BYTES 8 #define INLINE_OOB_LARGE_SIZE_IN_BYTES 24 @@ -425,6 +440,11 @@ struct mana_rxq { */ uint32_t desc_ring_head, desc_ring_tail; +#ifdef RTE_ARCH_32 + /* For storing wqe increment count btw each short doorbell ring */ + uint32_t wqe_cnt_to_short_db; +#endif + struct mana_gdma_queue gdma_rq; struct mana_gdma_queue gdma_cq; struct gdma_comp *gdma_comp_buf; @@ -455,8 +475,14 @@ extern int mana_logtype_init; #define PMD_INIT_FUNC_TRACE() PMD_INIT_LOG(DEBUG, " >>") +#ifdef RTE_ARCH_32 +int mana_ring_short_doorbell(void *db_page, enum gdma_queue_types queue_type, + uint32_t queue_id, uint32_t tail_incr, + uint8_t arm); +#else int mana_ring_doorbell(void *db_page, enum gdma_queue_types queue_type, uint32_t queue_id, uint32_t tail, uint8_t arm); +#endif int mana_rq_ring_doorbell(struct mana_rxq *rxq); int gdma_post_work_request(struct mana_gdma_queue *queue, diff --git a/drivers/net/mana/rx.c b/drivers/net/mana/rx.c index 14d9085801..fc1587e206 100644 --- a/drivers/net/mana/rx.c +++ b/drivers/net/mana/rx.c @@ -39,10 +39,18 @@ mana_rq_ring_doorbell(struct mana_rxq *rxq) /* Hardware Spec specifies that software client should set 0 for * wqe_cnt for Receive Queues. */ +#ifdef RTE_ARCH_32 + ret = mana_ring_short_doorbell(db_page, GDMA_QUEUE_RECEIVE, + rxq->gdma_rq.id, + rxq->wqe_cnt_to_short_db * + GDMA_WQE_ALIGNMENT_UNIT_SIZE, + 0); +#else ret = mana_ring_doorbell(db_page, GDMA_QUEUE_RECEIVE, rxq->gdma_rq.id, rxq->gdma_rq.head * GDMA_WQE_ALIGNMENT_UNIT_SIZE, 0); +#endif if (ret) DP_LOG(ERR, "failed to ring RX doorbell ret %d", ret); @@ -97,6 +105,9 @@ mana_alloc_and_post_rx_wqe(struct mana_rxq *rxq) /* update queue for tracking pending packets */ desc->pkt = mbuf; desc->wqe_size_in_bu = wqe_size_in_bu; +#ifdef RTE_ARCH_32 + rxq->wqe_cnt_to_short_db += wqe_size_in_bu; +#endif rxq->desc_ring_head = (rxq->desc_ring_head + 1) % rxq->num_desc; } else { DP_LOG(DEBUG, "failed to post recv ret %d", ret); @@ -115,12 +126,22 @@ mana_alloc_and_post_rx_wqes(struct mana_rxq *rxq) int ret; uint32_t i; +#ifdef RTE_ARCH_32 + rxq->wqe_cnt_to_short_db = 0; +#endif for (i = 0; i < rxq->num_desc; i++) { ret = mana_alloc_and_post_rx_wqe(rxq); if (ret) { DP_LOG(ERR, "failed to post RX ret = %d", ret); return ret; } + +#ifdef RTE_ARCH_32 + if (rxq->wqe_cnt_to_short_db > RX_WQE_SHORT_DB_THRESHOLD) { + mana_rq_ring_doorbell(rxq); + rxq->wqe_cnt_to_short_db = 0; + } +#endif } mana_rq_ring_doorbell(rxq); @@ -397,6 +418,10 @@ mana_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n) uint32_t i; int polled = 0; +#ifdef RTE_ARCH_32 + rxq->wqe_cnt_to_short_db = 0; +#endif + repoll: /* Polling on new completions if we have no backlog */ if (rxq->comp_buf_idx == rxq->comp_buf_len) { @@ -505,6 +530,16 @@ mana_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n) wqe_posted++; if (pkt_received == pkts_n) break; + +#ifdef RTE_ARCH_32 + /* Ring short doorbell if approaching the wqe increment + * limit. + */ + if (rxq->wqe_cnt_to_short_db > RX_WQE_SHORT_DB_THRESHOLD) { + mana_rq_ring_doorbell(rxq); + rxq->wqe_cnt_to_short_db = 0; + } +#endif } rxq->backlog_idx = pkt_idx; @@ -525,6 +560,15 @@ mana_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n) return pkt_received; } +#ifdef RTE_ARCH_32 +static int +mana_arm_cq(struct mana_rxq *rxq __rte_unused, uint8_t arm __rte_unused) +{ + DP_LOG(ERR, "Do not support in 32 bit"); + + return -ENODEV; +} +#else static int mana_arm_cq(struct mana_rxq *rxq, uint8_t arm) { @@ -538,6 +582,7 @@ mana_arm_cq(struct mana_rxq *rxq, uint8_t arm) return mana_ring_doorbell(priv->db_page, GDMA_QUEUE_COMPLETION, rxq->gdma_cq.id, head, arm); } +#endif int mana_rx_intr_enable(struct rte_eth_dev *dev, uint16_t rx_queue_id) diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c index 11ba2ee1ac..1e2508e1f2 100644 --- a/drivers/net/mana/tx.c +++ b/drivers/net/mana/tx.c @@ -176,6 +176,9 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) void *db_page; uint16_t pkt_sent = 0; uint32_t num_comp, i; +#ifdef RTE_ARCH_32 + uint32_t wqe_count = 0; +#endif /* Process send completions from GDMA */ num_comp = gdma_poll_completion_queue(&txq->gdma_cq, @@ -418,6 +421,20 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) DP_LOG(DEBUG, "nb_pkts %u pkt[%d] sent", nb_pkts, pkt_idx); +#ifdef RTE_ARCH_32 + wqe_count += wqe_size_in_bu; + if (wqe_count > TX_WQE_SHORT_DB_THRESHOLD) { + /* wqe_count approaching to short doorbell + * increment limit. Stop processing further + * more packets and just ring short + * doorbell. + */ + DP_LOG(DEBUG, "wqe_count %u reaching limit, " + "pkt_sent %d", + wqe_count, pkt_sent); + break; + } +#endif } else { DP_LOG(DEBUG, "pkt[%d] failed to post send ret %d", pkt_idx, ret); @@ -436,11 +453,19 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) } if (pkt_sent) { +#ifdef RTE_ARCH_32 + ret = mana_ring_short_doorbell(db_page, GDMA_QUEUE_SEND, + txq->gdma_sq.id, + wqe_count * + GDMA_WQE_ALIGNMENT_UNIT_SIZE, + 0); +#else ret = mana_ring_doorbell(db_page, GDMA_QUEUE_SEND, txq->gdma_sq.id, txq->gdma_sq.head * GDMA_WQE_ALIGNMENT_UNIT_SIZE, 0); +#endif if (ret) DP_LOG(ERR, "mana_ring_doorbell failed ret %d", ret); }