From patchwork Mon Aug 8 09:42:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 114715 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1DF1EA034C; Mon, 8 Aug 2022 11:42:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3D16342B79; Mon, 8 Aug 2022 11:42:48 +0200 (CEST) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam07on2046.outbound.protection.outlook.com [40.107.212.46]) by mails.dpdk.org (Postfix) with ESMTP id 77AC142B71; Mon, 8 Aug 2022 11:42:46 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ka+UlnAJWFfJjzSn7Y7bf4Kwb0FipSYp7/AQLjwjuP81m7Sn6QGyvwl7CJl/AHZ6zbR4jlhJhKGLIXncLlDFY8Piw4N/gtrxXo3JWLUC6eDErKynbfToEfk69Js7u8VUfF1VU14SZrBTq2hr7kGP8c2SKOTSZDRjUptuhEEjDKElxrMYUqwTP8QTXBUJqgzw8wgS/0mi9I3+9atePxALmO8geQ9yHUOCvkditgP3zKel1WRvQSZEi62KHwEjqe2P+ew1MRME54JKkCvxvGCMup8t72FsU08WcvXzu7xrXnqnLOIoy+6imZAQzs0+5xxgpFH6bU4/bLuzMDZTeJXmkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2Ew8YKNL6er9fJJXtmEIEyNBR39toWXDP/bk6jbT/SY=; b=gkBx/OkTYBQC7qHL//0OMMAc98F04gfU/+S5XDLohnv8H2dyGakOrq59UnWUpaw8NiAXGIdDlbq5jfIfB7UuxWHF0qzqk4a0Vez+x3k0KGfyR2t3utHRnzr21xK1K692tDCFztyw4cN5artuDSrlDEUT/QKn+8wbaFheUHlxDCjSYhcJsbISNvw8JLD+cANprpmPa6eiIpztyNUTi0u8G8WbRgk1H+QC5Y8BUtYrF5qLdDmaBmxt6bnLQK2k+nKsbsvqM7qCjJp769gUCka8CGTraJKT86mZQUEvxmi41czA1RFTryIVLDYg2otgzbrGwTkrnQ3S0IIyyAzU00Ccig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=oktetlabs.ru smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2Ew8YKNL6er9fJJXtmEIEyNBR39toWXDP/bk6jbT/SY=; b=hyCfdYghlMwqjj/vQogR8IWHlTNvm2mPVLRUOCP3Rm9sw+/sbymBNiZXL+Dm7ziWdrGgD1TMAeD+RuN2sa4NCHn7En3J9cf9Z/BWnISg+q5ZLDIzOX3fTVzAj7IoebXgxR8lXWQBIlD3lDfr7g+bOw4rcAxBok98PJUGbjgxTxyNBF6OwxCd2sT/Y4L5IuEb88oY2TfQZJjUbSxu4GdAf9/jEGhb9u+EudMf3QHQTXLClbOPos26DvhzllU+QVcXPpScKBONHyX5JUUEyxbDTWv+e2aXdRaSTaN37nmtm4zca1y2AD2MDCmljaC/d6jsnIZivtDtMTjn+sBORGkZWg== Received: from MW4PR02CA0013.namprd02.prod.outlook.com (2603:10b6:303:16d::13) by CO6PR12MB5409.namprd12.prod.outlook.com (2603:10b6:5:357::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.15; Mon, 8 Aug 2022 09:42:44 +0000 Received: from CO1NAM11FT031.eop-nam11.prod.protection.outlook.com (2603:10b6:303:16d:cafe::71) by MW4PR02CA0013.outlook.office365.com (2603:10b6:303:16d::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.16 via Frontend Transport; Mon, 8 Aug 2022 09:42:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT031.mail.protection.outlook.com (10.13.174.118) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5504.14 via Frontend Transport; Mon, 8 Aug 2022 09:42:42 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 8 Aug 2022 09:42:42 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Mon, 8 Aug 2022 02:42:41 -0700 Received: from nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.26 via Frontend Transport; Mon, 8 Aug 2022 02:42:40 -0700 From: Dmitry Kozlyuk To: CC: Olivier Matz , Andrew Rybchenko , Subject: [PATCH 1/2] mempool: make event callbacks process-private Date: Mon, 8 Aug 2022 12:42:35 +0300 Message-ID: <20220808094236.3395516-2-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808094236.3395516-1-dkozlyuk@nvidia.com> References: <20220808094236.3395516-1-dkozlyuk@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c1bab691-2834-4efd-953a-08da79225776 X-MS-TrafficTypeDiagnostic: CO6PR12MB5409:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: s8LG6dWJwX5zu/FLua6ddfD8x24Nq5YDx6ldkNMy95xtgzGoreDW0Wp6F3YqiN5ik8b7GaOf8EjqH8FqhuNzhj+J4C3Ve5GNTBixM1J3h4ulVdx+fSYAFNcPBsbuAlC/EMliQJgztrYPDRUirw4o0hGoN6t+LIqTk7UwOoX+UvhHmA/Qt4DUoq95APhpvJD07Ose/VLCjSRDhvz9qzoglBiO6XKq48aWKGvwRrQCpFzTesb5FIJ9NySxJdoOnCE5d1lIOkIp9rAU1QY9snmn7YvLIu2I9tw/Wb7b5Tnx3BVvDGz89lDUeHl6p+VyFnPcjjenIsByHCYlU1vcW7c5Prof5kWvU1tH3YxjXKxA4XGh7/AbPxu3eaj2cw2PfosIMhNPMQ/BJq3aVTbj17VMCWYBfRAEvnHz4gC6XETJlvnZKpElQUJ1eApJzM5JP3mBzxFSOZa2azFV+I5oO7A2F6kiuJiwIIQJdM40LHbnKHCztqdr6ClU74p8zkRrmJooxs+J/ecNto9DMHPGTy3on2rIsBHMaK5IaM1zrCPumVH+IGrh6LrREYR9F8DHENclPbuUUEYKmtYGH8WzCV9v5Rla8p0W1aUl1IEeF4A9iHK5VraKE52bH2QleJSzbJhWlU8ndSoegWNJumV80n2Dkpb8Mhm6N8M9hg1uvP7FiShAnA25ZusX1uySvqkGRAMAWu2Ii4sDOtTthQnb5hX0iwmpQgdlZYLLYUZyWnVfZdt+n9jQZQ3QNP5BbWx3AiF0eDVGVsDi7laUWXBELiru/N5hSe7tLx5opkuT4gqyQeNqAVXgotec4w75vDo7wmBcKUyKnGEJcNYWw180jeSjqA== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230016)(4636009)(376002)(346002)(39860400002)(396003)(136003)(40470700004)(36840700001)(46966006)(70206006)(70586007)(36756003)(8676002)(4326008)(1076003)(55016003)(186003)(316002)(86362001)(8936002)(40460700003)(5660300002)(478600001)(54906003)(6916009)(336012)(83380400001)(7696005)(2906002)(356005)(6286002)(82740400003)(26005)(41300700001)(6666004)(81166007)(47076005)(82310400005)(426003)(2616005)(40480700001)(36860700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2022 09:42:42.8334 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c1bab691-2834-4efd-953a-08da79225776 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT031.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5409 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Callbacks for mempool events were registered in a process-shared tailq. This was inherently incorrect because the same function may be loaded to a different address in each process. Make the tailq process-private. Use the EAL tailq lock to reduce the number of different locks this module operates. Fixes: da2b9cb25e5f ("mempool: add event callbacks") Cc: stable@dpdk.org Signed-off-by: Dmitry Kozlyuk Acked-by: Viacheslav Ovsiienko Reviewed-by: Andrew Rybchenko --- lib/mempool/rte_mempool.c | 54 ++++++++++----------------------------- lib/mempool/rte_mempool.h | 2 ++ 2 files changed, 16 insertions(+), 40 deletions(-) diff --git a/lib/mempool/rte_mempool.c b/lib/mempool/rte_mempool.c index de59009baf..1e039f6247 100644 --- a/lib/mempool/rte_mempool.c +++ b/lib/mempool/rte_mempool.c @@ -36,12 +36,10 @@ static struct rte_tailq_elem rte_mempool_tailq = { }; EAL_REGISTER_TAILQ(rte_mempool_tailq) -TAILQ_HEAD(mempool_callback_list, rte_tailq_entry); +TAILQ_HEAD(mempool_callback_tailq, mempool_callback_data); -static struct rte_tailq_elem callback_tailq = { - .name = "RTE_MEMPOOL_CALLBACK", -}; -EAL_REGISTER_TAILQ(callback_tailq) +static struct mempool_callback_tailq callback_tailq = + TAILQ_HEAD_INITIALIZER(callback_tailq); /* Invoke all registered mempool event callbacks. */ static void @@ -1372,6 +1370,7 @@ void rte_mempool_walk(void (*func)(struct rte_mempool *, void *), } struct mempool_callback_data { + TAILQ_ENTRY(mempool_callback_data) callbacks; rte_mempool_event_callback *func; void *user_data; }; @@ -1380,14 +1379,11 @@ static void mempool_event_callback_invoke(enum rte_mempool_event event, struct rte_mempool *mp) { - struct mempool_callback_list *list; - struct rte_tailq_entry *te; + struct mempool_callback_data *cb; void *tmp_te; rte_mcfg_tailq_read_lock(); - list = RTE_TAILQ_CAST(callback_tailq.head, mempool_callback_list); - RTE_TAILQ_FOREACH_SAFE(te, list, next, tmp_te) { - struct mempool_callback_data *cb = te->data; + RTE_TAILQ_FOREACH_SAFE(cb, &callback_tailq, callbacks, tmp_te) { rte_mcfg_tailq_read_unlock(); cb->func(event, mp, cb->user_data); rte_mcfg_tailq_read_lock(); @@ -1399,10 +1395,7 @@ int rte_mempool_event_callback_register(rte_mempool_event_callback *func, void *user_data) { - struct mempool_callback_list *list; - struct rte_tailq_entry *te = NULL; struct mempool_callback_data *cb; - void *tmp_te; int ret; if (func == NULL) { @@ -1411,36 +1404,23 @@ rte_mempool_event_callback_register(rte_mempool_event_callback *func, } rte_mcfg_tailq_write_lock(); - list = RTE_TAILQ_CAST(callback_tailq.head, mempool_callback_list); - RTE_TAILQ_FOREACH_SAFE(te, list, next, tmp_te) { - cb = te->data; + TAILQ_FOREACH(cb, &callback_tailq, callbacks) { if (cb->func == func && cb->user_data == user_data) { ret = -EEXIST; goto exit; } } - te = rte_zmalloc("mempool_cb_tail_entry", sizeof(*te), 0); - if (te == NULL) { - RTE_LOG(ERR, MEMPOOL, - "Cannot allocate event callback tailq entry!\n"); - ret = -ENOMEM; - goto exit; - } - - cb = rte_malloc("mempool_cb_data", sizeof(*cb), 0); + cb = calloc(1, sizeof(*cb)); if (cb == NULL) { - RTE_LOG(ERR, MEMPOOL, - "Cannot allocate event callback!\n"); - rte_free(te); + RTE_LOG(ERR, MEMPOOL, "Cannot allocate event callback!\n"); ret = -ENOMEM; goto exit; } cb->func = func; cb->user_data = user_data; - te->data = cb; - TAILQ_INSERT_TAIL(list, te, next); + TAILQ_INSERT_TAIL(&callback_tailq, cb, callbacks); ret = 0; exit: @@ -1453,27 +1433,21 @@ int rte_mempool_event_callback_unregister(rte_mempool_event_callback *func, void *user_data) { - struct mempool_callback_list *list; - struct rte_tailq_entry *te = NULL; struct mempool_callback_data *cb; int ret = -ENOENT; rte_mcfg_tailq_write_lock(); - list = RTE_TAILQ_CAST(callback_tailq.head, mempool_callback_list); - TAILQ_FOREACH(te, list, next) { - cb = te->data; + TAILQ_FOREACH(cb, &callback_tailq, callbacks) { if (cb->func == func && cb->user_data == user_data) { - TAILQ_REMOVE(list, te, next); + TAILQ_REMOVE(&callback_tailq, cb, callbacks); ret = 0; break; } } rte_mcfg_tailq_write_unlock(); - if (ret == 0) { - rte_free(te); - rte_free(cb); - } + if (ret == 0) + free(cb); rte_errno = -ret; return ret; } diff --git a/lib/mempool/rte_mempool.h b/lib/mempool/rte_mempool.h index 3ada37cb86..bfc1f3c823 100644 --- a/lib/mempool/rte_mempool.h +++ b/lib/mempool/rte_mempool.h @@ -1847,6 +1847,8 @@ typedef void (rte_mempool_event_callback)( * Register a callback function invoked on mempool life cycle event. * The function will be invoked in the process * that performs an action which triggers the callback. + * Registration is process-private, + * i.e. each process must manage callbacks on its own if needed. * * @param func * Callback function. From patchwork Mon Aug 8 09:42:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 114716 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 89CE5A034C; Mon, 8 Aug 2022 11:42:55 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7AF4B42B8B; Mon, 8 Aug 2022 11:42:49 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2068.outbound.protection.outlook.com [40.107.94.68]) by mails.dpdk.org (Postfix) with ESMTP id CDADD42B73; Mon, 8 Aug 2022 11:42:46 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RaVgTQQjeMZjeQhBFYf617mz2w35DcrKPv35Jy1WthCxHFbknja7pPnSGUyHHMs4SQLiufW17xn+7vq1gEiBRJbGAUeNq0jPtxNdFFG1l1WEr2/wf0JFn7wADHBMqhlNG/Bvs3vh0dBPUruQDAY4PT+tOxry0H+6km2nULgbKikenbRDT8Wa/fWSam6rO1hkUkW/6JLYGtx00OsMlTzBwUVDA1yrzqD3lgemqQXXw3pWypY6Cw9WNqjEqfedoilTCpgvr+xDzVbWJ3L4n32JKHz/N5vW9Ih3ydUrRgIGsNfl8W8EE6LiqfMUKsSCNFT4+taHLqvrXnpCoCx1UjLPFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=scMOvLh5JGpIsZrBM+FGJdukVtiSUNV8Wsiu3SUA9XE=; b=MQAkYy/SxGy9Cs1cX6Dot7cmkxdzOvgAzaMHxwgyzjjr6REUYUnTuhYeiHfWNyeRBfzrVKvfatbTwM9NHjEzLjdBXgJr9JNTeSUCHGr7CM5sTZ/Kh1kTF4XL2LX2riXheCAdiodfHsMnQI33yT4C4GPCzMgTZAxxyYtaWwRfEVOJmcBhecSbHvnAXRdAaamzlUM5AmsD4DOXm1yu7w8fa6nI+BLfHsaugH358iIGCLxw7hzC8SfssRaXm1Txw/NLsxLLiau/0PnOUw2TUrWBPAZoxRh4U1ruIZ7UtqWSjJSTbWnrHUn6jBjsjotg4AhsSOXXsSoA2xVTrupDUzgtIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=scMOvLh5JGpIsZrBM+FGJdukVtiSUNV8Wsiu3SUA9XE=; b=mWfVeBhxy/R/aZWTW7LGo3ZUHUFnJUYbPlHOIe3hc9KA5AJA/9NKRe4082USzeoYmEHNf8vWd6UHD5Gz7AkzSD1jFXi/wzFGHElZOMZeNJSJ1akAIJTe6GGh7k5kOGMMqwp3fRGOoLXFWbRl2c6e1e+n7ixoNarvBiQ5/KqUCsyfjibzULtPnzyffOb5PvVedkL+j6gxQm6ly6lC8Qu1T9Hk9Ef0ErxFomqEiagWTj/M34AETAoBq8jclPBR5m6HZhE2p0ROq4t4m5kWl0fuCYqJeGnk9/p/POulpCTESJ8DU2E2z4XSNaJtkzz1n9nFBLFu94QnAFXwRKFu4GLWCw== Received: from MW4PR04CA0052.namprd04.prod.outlook.com (2603:10b6:303:6a::27) by MWHPR12MB1310.namprd12.prod.outlook.com (2603:10b6:300:a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.14; Mon, 8 Aug 2022 09:42:45 +0000 Received: from CO1NAM11FT033.eop-nam11.prod.protection.outlook.com (2603:10b6:303:6a:cafe::98) by MW4PR04CA0052.outlook.office365.com (2603:10b6:303:6a::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5482.15 via Frontend Transport; Mon, 8 Aug 2022 09:42:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT033.mail.protection.outlook.com (10.13.174.247) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5504.14 via Frontend Transport; Mon, 8 Aug 2022 09:42:44 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 8 Aug 2022 09:42:44 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Mon, 8 Aug 2022 02:42:43 -0700 Received: from nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.26 via Frontend Transport; Mon, 8 Aug 2022 02:42:42 -0700 From: Dmitry Kozlyuk To: CC: Matan Azrad , Viacheslav Ovsiienko , Subject: [PATCH 2/2] common/mlx5: fix multi-process mempool registration Date: Mon, 8 Aug 2022 12:42:36 +0300 Message-ID: <20220808094236.3395516-3-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220808094236.3395516-1-dkozlyuk@nvidia.com> References: <20220808094236.3395516-1-dkozlyuk@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 836a2844-47c3-4992-a260-08da7922589d X-MS-TrafficTypeDiagnostic: MWHPR12MB1310:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /GVJHEfdavji+Gh5qKtm3M+sovhG7qMfnVGOHKAT1HbAH0rxwweTzxo6AhQLNSjZ5hAaXqZkVkV3pURrKeTrcykcZIY798qkaIZVWnbBCLUxG47oA16Cwa95qizkNRWDziugD8pK5U+aVG0cJJWmhfnzAr9x74534Ox0Z6gaoyzt9AAUPVd8S5+MTCTjaJ2ulXkTNfDv2OJue5ABPyXGY5MCEUjVlE93O3f8RkbDnvmtO1dj7l0TljWoYH6O8Zwxw2mrH6f2TcCfkTOILw+XR/UHB0xLRgxipd3ESFY5jENsCQJN9FageZMOe4WeyyvpI0vzwkTKh9dWuQGK2aUEg5w2fW6/5lLZZ/K5B6AYL8LFwLDSPvgkpKn6PmqU7eSmq80YQ/QVnAzYCRl22NgSpT7Wr/MqS/QyN4HE/Ooczbej2fgz2pfppSSgUiQx8lSs19nOs6tNU8HuwFbsw+clchD1MOTLwjt4dV5TJnMGndQG3WKLlKhlxPfa7jamjTQmlm5z4Kix+n6F3bMLikvJ6ZPvj5XZjZFzxCKBXAfApyDHy75NE23DXJPfx/2naBsWEhRr07KxBOfg39e3lls7Fm9TYLcPtwQzijV4HZmdt+W+JAkPeRvj7wPh/yzpi4I4t01r0WRvB6PUO/bWQPQ2pdpVA9/Sb7XH1R7j5B+uKqEWooBYKUX+fDFtm63DiZUg+6CRMU4oJNjlGWmNXlJGOy7Ewq2PZLrLtAo7aytBDfdQHoWCfVzoO2TkZdq4ZJA3WTNxhzDmKRqzfHEm3O2l0SuGbAdiB16XMNOUH5Z65bAarMvhRrlBTUzFUY5K1aavitSLGqfxiYfGtymruAht5Q== X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230016)(4636009)(346002)(136003)(376002)(396003)(39860400002)(40470700004)(46966006)(36840700001)(4326008)(70586007)(70206006)(6916009)(54906003)(36860700001)(8936002)(82310400005)(8676002)(186003)(5660300002)(316002)(450100002)(36756003)(478600001)(2616005)(1076003)(2906002)(6286002)(26005)(426003)(47076005)(41300700001)(336012)(6666004)(7696005)(55016003)(40480700001)(356005)(83380400001)(81166007)(82740400003)(40460700003)(86362001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2022 09:42:44.7827 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 836a2844-47c3-4992-a260-08da7922589d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT033.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1310 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The `mp_cb_registered` flag shared between all processes was used to ensure that for any IB device (MLX5 common device) mempool event callback was registered only once and mempools that had been existing before the device start were traversed only once to register them. Since mempool callback registrations have become process-private, callback registration must be done by every process. The flag can no longer reflect the state for any single process. Replace it with a registration counter to track when no more callbacks are registered for the device in any process. It is sufficient to only register pre-existing mempools in the primary process because it is the one that starts the device. Fixes: 690b2a88c2f7 ("common/mlx5: add mempool registration facilities") Cc: stable@dpdk.org Signed-off-by: Dmitry Kozlyuk Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_common.c | 15 +++++++++------ drivers/common/mlx5/mlx5_common_mr.c | 2 +- drivers/common/mlx5/mlx5_common_mr.h | 2 +- 3 files changed, 11 insertions(+), 8 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 89fef2b535..4dcc8cc49c 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -583,18 +583,17 @@ mlx5_dev_mempool_subscribe(struct mlx5_common_device *cdev) if (!cdev->config.mr_mempool_reg_en) return 0; rte_rwlock_write_lock(&cdev->mr_scache.mprwlock); - if (cdev->mr_scache.mp_cb_registered) - goto exit; /* Callback for this device may be already registered. */ ret = rte_mempool_event_callback_register(mlx5_dev_mempool_event_cb, cdev); if (ret != 0 && rte_errno != EEXIST) goto exit; + __atomic_add_fetch(&cdev->mr_scache.mempool_cb_reg_n, 1, + __ATOMIC_ACQUIRE); /* Register mempools only once for this device. */ - if (ret == 0) + if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_mempool_walk(mlx5_dev_mempool_register_cb, cdev); ret = 0; - cdev->mr_scache.mp_cb_registered = 1; exit: rte_rwlock_write_unlock(&cdev->mr_scache.mprwlock); return ret; @@ -603,10 +602,14 @@ mlx5_dev_mempool_subscribe(struct mlx5_common_device *cdev) static void mlx5_dev_mempool_unsubscribe(struct mlx5_common_device *cdev) { + uint32_t mempool_cb_reg_n; int ret; - if (!cdev->mr_scache.mp_cb_registered || - !cdev->config.mr_mempool_reg_en) + if (!cdev->config.mr_mempool_reg_en) + return; + mempool_cb_reg_n = __atomic_sub_fetch(&cdev->mr_scache.mempool_cb_reg_n, + 1, __ATOMIC_RELEASE); + if (mempool_cb_reg_n > 0) return; /* Stop watching for mempool events and unregister all mempools. */ ret = rte_mempool_event_callback_unregister(mlx5_dev_mempool_event_cb, diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c index 8d8bec99a9..1d54102b54 100644 --- a/drivers/common/mlx5/mlx5_common_mr.c +++ b/drivers/common/mlx5/mlx5_common_mr.c @@ -1138,7 +1138,7 @@ mlx5_mr_create_cache(struct mlx5_mr_share_cache *share_cache, int socket) &share_cache->dereg_mr_cb); rte_rwlock_init(&share_cache->rwlock); rte_rwlock_init(&share_cache->mprwlock); - share_cache->mp_cb_registered = 0; + share_cache->mempool_cb_reg_n = 0; /* Initialize B-tree and allocate memory for global MR cache table. */ return mlx5_mr_btree_init(&share_cache->cache, MLX5_MR_BTREE_CACHE_N * 2, socket); diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h index 213f5427cb..a5f2d4fd35 100644 --- a/drivers/common/mlx5/mlx5_common_mr.h +++ b/drivers/common/mlx5/mlx5_common_mr.h @@ -81,7 +81,7 @@ struct mlx5_mr_share_cache { uint32_t dev_gen; /* Generation number to flush local caches. */ rte_rwlock_t rwlock; /* MR cache Lock. */ rte_rwlock_t mprwlock; /* Mempool Registration Lock. */ - uint8_t mp_cb_registered; /* Mempool are Registered. */ + uint32_t mempool_cb_reg_n; /* Mempool event callabck registrants. */ struct mlx5_mr_btree cache; /* Global MR cache table. */ struct mlx5_mr_list mr_list; /* Registered MR list. */ struct mlx5_mr_list mr_free_list; /* Freed MR list. */