From patchwork Tue Nov 16 08:57:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 104378 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B07A7A0C43; Tue, 16 Nov 2021 09:58:11 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 306BF410E7; Tue, 16 Nov 2021 09:58:11 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2074.outbound.protection.outlook.com [40.107.244.74]) by mails.dpdk.org (Postfix) with ESMTP id 43AC540141 for ; Tue, 16 Nov 2021 09:58:10 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e+xwx2FyYNK28FqV/PIi+xvEbt6iXy0Cv26Zws80VtNQhwPHIzpYYWBFxocakAtuzwbURf0W9Yyir9IJ7hzUyTEUqvLRz9JL8/AaiTBRWURQAnpV3WhgCwPCgSdRuBFWd7EUjfE48PVm6iKlQDvkbvWZkiwh+wthLZ839lHkwA0Uh0kvInJY4wdTCgE0H40Mjq5M8PbFnYab/bz8wg0QnV2R2Yn/S7e9/6DbwZl9+JdiaeKZQUO4B8TctUKPtq4G/CK39Vm97bZmE83EbdmaMo572nrmuIAriqwVS1t/aHT4d2xW5vluVd9lS8fwWwnXLVxQ5wd7SH3SIiym+BU7Qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Hji1thGoOKsjD2SDjzXOyk5LfOlfudJCWMQS5+C6tNM=; b=L/a4VXv+kzP1i0ykniAEuxJlhFIk8a+SuREgM0vDidvoEVqK8GN8idY9jXVMTUAnSwQFOER31emcPrCwy7LlkG0zTpdWIKk+fQpCFthC7m+GRRlx9za0zd3SkMnkA/F+FwESTCKnTh5Gpbw2JF/c1HG/VK9HVBEbOO/Fhk4OuUU8WrX2y8kkb35KT4ZFr2eMzyCyvRYMQkWTBRfH4XSf58ceowAZhKQj9DvaqLlzq7Qwk9sqmFgoSKSPisoZRDP58mtkvLdnqBAmHulmzbblC7aLo7fFJhrRje81vFAFPZ/iL93uo0sDbuQKdH0a5BJMafngcHlMadqeXf0+iAoUMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Hji1thGoOKsjD2SDjzXOyk5LfOlfudJCWMQS5+C6tNM=; b=b36swsLL5k5F+rzeJePwcV6+mo35qc1yd3VBN7y9nHKDCbq4FmzEMWFmVbWQs43fFZ5CkzxMwexZ3+E0qvtC+1hN13ssZ9d8Tia5vLjSRmqHdlKgD2j8NrdaqPbqOOpvZCnZacDYyVbGuP5cP9+5OkGbcuIq8HDmr2lYbmDlRl+/c1mgtGk4obcHMboNFxYigCOb6D1U3Me3NhL2QAgDR8yAzIITTDrduJRhGkrrWWQHQ3hdwuanKZcYotbmeO3012lj9GOXhUMfjGjcvUY4wLPzAcJMeszx5z6+9Zo2xqIpn4bnpe7q94mb0dAiVMJPOsiANseQGvMlsunOlhIH+A== Received: from BN9PR03CA0363.namprd03.prod.outlook.com (2603:10b6:408:f7::8) by DM6PR12MB2956.namprd12.prod.outlook.com (2603:10b6:5:182::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4690.26; Tue, 16 Nov 2021 08:58:08 +0000 Received: from BN8NAM11FT051.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f7:cafe::3b) by BN9PR03CA0363.outlook.office365.com (2603:10b6:408:f7::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4690.26 via Frontend Transport; Tue, 16 Nov 2021 08:58:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT051.mail.protection.outlook.com (10.13.177.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4690.15 via Frontend Transport; Tue, 16 Nov 2021 08:58:07 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 16 Nov 2021 08:58:06 +0000 From: Dmitry Kozlyuk To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Subject: [PATCH] common/mlx5: fix MPRQ mempool registration Date: Tue, 16 Nov 2021 10:57:52 +0200 Message-ID: <20211116085752.2419240-1-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dd8c10bc-24be-4819-19a3-08d9a8df35a4 X-MS-TrafficTypeDiagnostic: DM6PR12MB2956: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3383; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nI39K0tQPGIMl1raPTQavs4YlR+Exx8mRK80h34Xj6rGmb8Uln4Hq4PUnakWNU9mjKY9mCEVxmsH23GUF8o5T6WzviMGasgQXEve5oc/Ae/lEWcH4phTGXxRAnJffORvU4T4R0QN/FtWcd7i3s+8eNSI1chB+0giAzYEHZeobDiLRRXGRb2oZ2d68Gr+yne1vS0kJvZTY2xwxbtX34Z2H5WH/J6o+8nlrQ5BayPv2DuJHqZKINcMyyiv3V39aJSy29jeNbfnFdCkEdXrypNOFuOr4M5LLLs9kN+/0Vyu9TOzgkHoWzu5OEban8TlDs8xORHxBzyx+1bxHI275AhuCtfQ8ladO92EvfLNHgQa43Ojx8s5+caxsJusWHO/B4fy1dTyfRSd1fsJ8GmkTzSwXzP9zwBgDXUctlesK+rDHl5qckqjsRgG0WaIyV7Hde9g875FD+HFat/hpzDqw0pr3Tz+0Rui8mvhoaf/rKV8SBCN5ZvFUvsr+VUKGpGu1303sMmLMkGL4Yeqx1aEaQSQDVdlvkYwpN5wavPqU8EyK/N1hfPbD6KWk0RsDRsgDq0C8pqwj0Q+is9jag8lpmpjGgRgJ9e5zJTcpbkpTtn6/krA9WnPLqz7oOSNxPbkTldjSNMTPFRztUmP45zflQc++92a3IL7IrcGgDRagEBdfYi8A9UZybd2LlwxR80VU7nac4aakIJ8iUnAbrX7rQukknsgOtu1nQ1GvN+RLqSs4h4= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(8676002)(47076005)(336012)(8936002)(36860700001)(356005)(36756003)(2906002)(54906003)(36906005)(1076003)(508600001)(316002)(55016002)(82310400003)(5660300002)(6666004)(6286002)(70586007)(86362001)(186003)(6916009)(70206006)(7696005)(107886003)(4326008)(2616005)(16526019)(26005)(7636003)(426003)(83380400001)(290074003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Nov 2021 08:58:07.8977 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dd8c10bc-24be-4819-19a3-08d9a8df35a4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT051.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB2956 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Mempool registration code had a wrong assumption that it is always dealing with packet mempools and always called rte_pktmbuf_priv_flags(), which returned a random value for different types of mempools. In particular, it could consider MPRQ mempools as having externally pinned buffers, which is wrong. Packet mempools cannot be reliably recognized, but it is sufficient to check that the mempool is not a packet one, so it cannot have externally pinned buffers. Compare mempool private data size to that of packet mempools to check. Fixes: 690b2a88c2f7 ("common/mlx5: add mempool registration facilities") Fixes: fec28ca0e3a9 ("net/mlx5: support mempool registration") Signed-off-by: Dmitry Kozlyuk Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_common.c | 3 +-- drivers/common/mlx5/mlx5_common_mr.c | 2 +- drivers/common/mlx5/mlx5_common_mr.h | 10 ++++++++++ drivers/net/mlx5/mlx5_trigger.c | 5 +++-- 4 files changed, 15 insertions(+), 5 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index b9ed5ee676..66c2c08b7d 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -390,8 +390,7 @@ mlx5_dev_mempool_event_cb(enum rte_mempool_event event, struct rte_mempool *mp, void *arg) { struct mlx5_common_device *cdev = arg; - bool extmem = rte_pktmbuf_priv_flags(mp) & - RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF; + bool extmem = mlx5_mempool_is_extmem(mp); switch (event) { case RTE_MEMPOOL_EVENT_READY: diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c index 49feea4474..69fae88fb0 100644 --- a/drivers/common/mlx5/mlx5_common_mr.c +++ b/drivers/common/mlx5/mlx5_common_mr.c @@ -1414,7 +1414,7 @@ mlx5_get_mempool_ranges(struct rte_mempool *mp, struct mlx5_range **out, int ret; /* Collect the pool underlying memory. */ - ret = (rte_pktmbuf_priv_flags(mp) & RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF) ? + ret = mlx5_mempool_is_extmem(mp) ? mlx5_mempool_get_extmem(mp, &chunks, &chunks_n) : mlx5_mempool_get_chunks(mp, &chunks, &chunks_n); if (ret < 0) diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h index dc7ddc3513..442b9d4694 100644 --- a/drivers/common/mlx5/mlx5_common_mr.h +++ b/drivers/common/mlx5/mlx5_common_mr.h @@ -263,4 +263,14 @@ int mlx5_mr_mempool_unregister(struct mlx5_common_device *cdev, struct rte_mempool *mp); +/** Check if @p mp has buffers pinned in external memory. */ +static inline bool +mlx5_mempool_is_extmem(struct rte_mempool *mp) +{ + return (mp->private_data_size == + sizeof(struct rte_pktmbuf_pool_private)) && + (mp->elt_size >= sizeof(struct rte_mbuf)) && + (rte_pktmbuf_priv_flags(mp) & RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF); +} + #endif /* RTE_PMD_MLX5_COMMON_MR_H_ */ diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index 1952d68444..4440a765d9 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -149,13 +149,14 @@ mlx5_rxq_mempool_register(struct mlx5_rxq_ctrl *rxq_ctrl) uint32_t flags; mp = rxq_ctrl->rxq.rxseg[s].mp; - flags = rte_pktmbuf_priv_flags(mp); + flags = mp != rxq_ctrl->rxq.mprq_mp ? + rte_pktmbuf_priv_flags(mp) : 0; ret = mlx5_mr_mempool_register(rxq_ctrl->sh->cdev, mp); if (ret < 0 && rte_errno != EEXIST) return ret; if ((flags & RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF) == 0) rte_mempool_mem_iter(mp, mlx5_rxq_mempool_register_cb, - &rxq_ctrl->rxq); + &rxq_ctrl->rxq); } return 0; }