Message ID | 20220922080642.315800-1-niklas.soderlund@corigine.com (mailing list archive) |
---|---|
State | Accepted, archived |
Delegated to: | Thomas Monjalon |
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 817DFA0543; Thu, 22 Sep 2022 10:09:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 744D740156; Thu, 22 Sep 2022 10:09:02 +0200 (CEST) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2115.outbound.protection.outlook.com [40.107.102.115]) by mails.dpdk.org (Postfix) with ESMTP id F165F400D7; Thu, 22 Sep 2022 10:09:00 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GPhqTUd1LwGV/hkT4q2pJxB+8xZxT0RuvRCCdN41RJvjiZ6HktJNCb6uXmzBGQUXxFoCAZKtxO5xu+Ait2tPK0LFLeoLtcrlbWNCFQU2jtiFStKvSrBJfUnbKxsxEGZIYDXpRcnxpotcEqBb6sXXehUqlPIozKJu5wX6L8lBc/9FcYc6mBoSeZuB4MxCYiaZNp4bF4tDKcgip2YaTM2YM5oNZOztMCvb5Q8KM/uG6iAX8crhUNOS+66jUsVAbfkigzxONhGqhEnxhpRyXp1Q2RtX1dIYgQh5010KByPB9KKIOXI7b+07R/Eu1oOYhCLoLbJyFzU3KC1rzoJK56Zb3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XlYpErlhF+lRAt9B9GFAYMGLOBcYJVn+ZWS3BG18wrM=; b=Grrhu99kvTVwYNP++RuTpt/tjhlk4DsKJZAxTn35kQVpT/xde2gY4sx+VtfDYuXo1ms4LpT2L4Dvjcy4oFDK1EiuMNbXr+T3sT80a7NPeB3gASW9USkxlFOexhkvKRZgEbty34RwKj8l3A/L/Eyi92OzE56DoDP4nLEgNmQNFWOnRruugZ5QBJ89iLX7tpGH3cBbKM6Z46ULzhop2fuapvnpvqRv3PFAmkIaiHx70AZgm1HsvhUR/iq4fKEjAiLBl7zdnCnShnkzmDyofGpf5W9bp2P3uQxwgBGZMrmmNhLkSVc1IPO0eexLMrwBOP9DGBcuQBS94CtO2ezk9RW1+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XlYpErlhF+lRAt9B9GFAYMGLOBcYJVn+ZWS3BG18wrM=; b=RqLLWpU61LyjlRfxFL8XH3mzyq3YQTnfivq1xWpYe7md3xiK9DWyVZ2e5T+NzCES/FL/NcQPSz78kzBrvVtsYK65KgSmCv7BagDUZMzsqLANuv3FzoAyZ8QQoGu6wvU+Kf4a+E3Uu92y5pzjK6GQ5yjd+k15yHNAltIkP+m7EmE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from DM6PR13MB4431.namprd13.prod.outlook.com (2603:10b6:5:1bb::21) by DS7PR13MB4590.namprd13.prod.outlook.com (2603:10b6:5:297::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.7; Thu, 22 Sep 2022 08:08:58 +0000 Received: from DM6PR13MB4431.namprd13.prod.outlook.com ([fe80::e198:fd7c:5eb4:89c1]) by DM6PR13MB4431.namprd13.prod.outlook.com ([fe80::e198:fd7c:5eb4:89c1%5]) with mapi id 15.20.5654.014; Thu, 22 Sep 2022 08:08:58 +0000 From: =?utf-8?q?Niklas_S=C3=B6derlund?= <niklas.soderlund@corigine.com> To: dev@dpdk.org Cc: oss-drivers@corigine.com, Chaoyong He <chaoyong.he@corigine.com>, Peng Zhang <peng.zhang@corigine.com>, stable@dpdk.org, =?utf-8?q?Niklas_S?= =?utf-8?q?=C3=B6derlund?= <niklas.soderlund@corigine.com> Subject: [PATCH v2] buildtools: ensure the NUMA nodes are counted correct Date: Thu, 22 Sep 2022 10:06:42 +0200 Message-Id: <20220922080642.315800-1-niklas.soderlund@corigine.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <1659426855-11014-1-git-send-email-chaoyong.he@corigine.com> References: <1659426855-11014-1-git-send-email-chaoyong.he@corigine.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: AM6P191CA0014.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8b::27) To DM6PR13MB4431.namprd13.prod.outlook.com (2603:10b6:5:1bb::21) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR13MB4431:EE_|DS7PR13MB4590:EE_ X-MS-Office365-Filtering-Correlation-Id: 46da68b1-4510-4c89-f27f-08da9c71b385 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OBg73Zz4PLyUqofLgmtMV1WYRysRFSXWLVm1mFlppUV9qX0Mt1IWDPaKKOSfx9pWtgE5BhauZEdLdLxtZCw3sqVTl2A7dQIrNCM8A6Iu56hb6sfR+8dXdde6r6Lq8ZyCHn1vUMTyR2I286Kw1NJoFjwOK5epFwi18GwdsLIgBABi6ems9/P0t8N0gI4iDC0fRVtip8H+sXvw1jjXzthYkPKkB13+ogXyuQLh+3AfDDkSijeUaSTF41ZT1mkSXkH/7OFHFVmgBL5tXgSVCB8GAL8hVED0l6dsXJ9cA79csQpTSx4lFV89JI2rG+iTtrcu15uwnA4o2dKFxfnJeVhWZdncd4hfJZiEXh4T1EGtXV5IzaGx0cdv88ucPapZCjZq0G3zDvfn2L0gwO1pJFyy1RPoNQmriR9c75mO2TW8Bwm3o2Zd36LFFL8RqLbL1xGPOWXIUBYFmJ8y0XYCnnEGpuxbZUBkY480ByUHjzPoteMsNdQD4pB2v8hw/tQuSdx6XnDXmcvefTGLqAt3O/SCmlJI0H9JEc2BiiIzv9bRcYgL/9wmuCYgSXbmAjo64u6B1oN61PAgo95rRTX8x5Z7OJ1LQw0LlA9IvlpWEW2L4lyLd2bWLJBEposH6mwovsWcc0BvdkaZn3/Zsz7TEs5qPl0zjZXxt7gdYLBQxYQNBLPFoUduL86xg562GjAPqERkezd2osnn135DgZe/iAkgxs1srTrob/OjVJUfTs/beYGqRMJNbozhdI/m1fUmuFIKeyq3k3IjKIuojkyWXmPGyQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR13MB4431.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(346002)(366004)(39840400004)(376002)(136003)(396003)(451199015)(36756003)(26005)(6506007)(52116002)(6512007)(41300700001)(86362001)(316002)(54906003)(6916009)(4326008)(107886003)(6666004)(6486002)(66946007)(8676002)(66476007)(450100002)(478600001)(66556008)(83380400001)(66574015)(8936002)(2616005)(1076003)(2906002)(186003)(5660300002)(38100700002)(38350700002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?tJ/8MdadDOU5mgu4NLiwhex472z4?= =?utf-8?q?z3ydxFcjIsRzzuGH+YgFhU2qJFEWq9ZXBVIIW6z/LzFtNiEHfqZEp3+qZpg7emsn8?= =?utf-8?q?/FqvhUBn6jp5Vm/ejo5ZJbOqZnQJBYZsJkkOCwnzWs71qq8UTwjvovoQSOeAh9v76?= =?utf-8?q?kZ9ikqPmGegnKPZ4GwUw4zHC+xH15apQXXTw/fDvRHi4tcuiKV/SqQlPZ0ra/FWp+?= =?utf-8?q?6ygwdBJ0KiJ9YUlPTNlpd38rYzR2rwxRJFCsqIBcIKxwA7orKJt36dAKeRwNYKYNO?= =?utf-8?q?HyNKLcXwsrbvsDGRTDk6lG3ywli0h9ryvt515Beya3jbBrqcUvLQ3FWp7pbxhbHUX?= =?utf-8?q?9mj84+h79N9tbOq9vFG3fkyhbrSkwWF7ohbYmjUCQesgow+Hc1GSqGCPyYEoqQ/VD?= =?utf-8?q?pQOo3czT4xIiATWCPsF1rGa5VO9FwaPaB+B7+jsYwVudO4UjRodhS6JFoFSKyd4+A?= =?utf-8?q?Uo4L7L++F3u9usXtkHZrlJHCZyCKveghbN+RwXgEOIqVe2m7+5TxaLr1CdWMynZCT?= =?utf-8?q?w2EcLqoWgs/OTmfaW23dg1quCaF1ZA5um+hW+sm5zKDFNC+qwto3zibsjNIWZinm1?= =?utf-8?q?rontUohcrhpJXf0aPg5jhE8AUjACyy49NMJSlUpowo3yK7lEVpkfAw+5JUfJwHhsp?= =?utf-8?q?mevyE+hiOpTM3QW0kQmY3U096wocuSbquba9jOQLzvaB5BqP5Wr791SXJGG9wTu0+?= =?utf-8?q?9yVFUqIqanyUEfD5m+mPu+V6hps+79lXnQHRpSJZhseLm2gjStbJCQvjIiMlgZYFd?= =?utf-8?q?vPJVDF4wA8RuZkMDDC3KcKVNHupmesi4B71n34IYXzPatTi3bn41hMvDqn0/vREj1?= =?utf-8?q?D4dnhwZWCZIci972YI4gYIKH0j0uAyIYRonhI/E14t4lSye611pqT3Ncu6ZmVdnL0?= =?utf-8?q?kZB5mN42mXIU+VqnOBeYDjZ/1qhFHZH3AC4bapbiP/zr+od5undNwBz2Q4mKDeGcp?= =?utf-8?q?vCy/bjf82gAR+0mwPkSZUQ7zTgZpXfE617P2T+AJ8lvxWqFhbHy5WShedVlPgncMs?= =?utf-8?q?QbB4GGWBM/WV+jx3zVd+hGgXKwRkMmivL+tvzKrtpH7Q3mbL15T79PnRJb4mQGb1D?= =?utf-8?q?EE3kUBJqcsNLOpXdjhlVZ39nvfIQKt/f6eiQd9Cg80594rCfDCy/VgP5pwruMi9SF?= =?utf-8?q?cFxxBfnLNtBiF6hPxiTNWMPktQurhNxHxoWn6UGitH7/f5gTX+1A7y9j1b0I7lLlX?= =?utf-8?q?4JEGemp66urBhHHtHh9k4ns/BZwb4evYv+TStgNiujil2Eqzv/NXqubsdNGnz+qkK?= =?utf-8?q?98KNmeoGPJYf/id3URlr+ma35JuZcdUTCCtRYO204lIi2GmmyA6ooEb2OxQhiLLMK?= =?utf-8?q?rLMUeVAc7WPDvjGQf4XDjotBzrh7deTBYHvjc1rKf5pYl/lvsrRtC12K3aa3EgJSx?= =?utf-8?q?/R8JzhyicTLBEBKrosRfWBBmKVGRuT/+9tgowy4uFV1hV+f2WfBYlmOYn9PVX0T0E?= =?utf-8?q?opBlWzwDixs/qNC6trFpbx4BZ3aVBi9HheckPtM0j9EQ68JELyxamv9m0t2pJMNA1?= =?utf-8?q?R+F2FYpc0w0t/hJPdWEgyGAzOePZGUH5Uw=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 46da68b1-4510-4c89-f27f-08da9c71b385 X-MS-Exchange-CrossTenant-AuthSource: DM6PR13MB4431.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2022 08:08:58.5202 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gGH4PttmrSlQwuoorhha3zWww7oOc9PDiq9uM3mst9+i+0N39E9eBFJm3ojkxIRejSXRKlNebPTsFOc2cNywG+ABxMWGfmisG41n+I4LoRc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR13MB4590 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org |
Series |
[v2] buildtools: ensure the NUMA nodes are counted correct
|
|
Checks
Context | Check | Description |
---|---|---|
ci/checkpatch | success | coding style OK |
ci/Intel-compilation | success | Compilation OK |
ci/iol-mellanox-Performance | success | Performance Testing PASS |
ci/iol-aarch64-unit-testing | success | Testing PASS |
ci/iol-aarch64-compile-testing | success | Testing PASS |
ci/github-robot: build | success | github build: passed |
ci/iol-x86_64-compile-testing | success | Testing PASS |
ci/iol-x86_64-unit-testing | success | Testing PASS |
ci/iol-intel-Performance | success | Performance Testing PASS |
ci/iol-intel-Functional | success | Functional Testing PASS |
ci/intel-Testing | success | Testing PASS |
Commit Message
Niklas Söderlund
Sept. 22, 2022, 8:06 a.m. UTC
From: Peng Zhang <peng.zhang@corigine.com> The method to fetch, sort and read the last entry of a list to figure out the total number of NUMA nodes in the system fails with 10 or more nodes. The reason being the usage of string compare while sorting, hence node 'node10' will be sorted before 'node2'. Solve this by sorting the list based on integer comparison of the numerical part of the node name. Before this change on a system with 16 NUMA nodes, EAL: Detected CPU lcores: 128 EAL: Detected NUMA nodes: 10 EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem EAL: Detected static linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: VFIO support initialized With this change on the same system, EAL: Detected CPU lcores: 128 EAL: Detected NUMA nodes: 16 EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem EAL: Detected static linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: VFIO support initialized Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection") Cc: stable@dpdk.org Signed-off-by: Peng Zhang <peng.zhang@corigine.com> Signed-off-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com> --- buildtools/get-numa-count.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Comments
Hi, Genteel ping on this. On 2022-09-22 10:06:42 +0200, Niklas Söderlund wrote: > From: Peng Zhang <peng.zhang@corigine.com> > > The method to fetch, sort and read the last entry of a list to figure > out the total number of NUMA nodes in the system fails with 10 or more > nodes. The reason being the usage of string compare while sorting, hence > node 'node10' will be sorted before 'node2'. > > Solve this by sorting the list based on integer comparison of the > numerical part of the node name. > > Before this change on a system with 16 NUMA nodes, > > EAL: Detected CPU lcores: 128 > EAL: Detected NUMA nodes: 10 > EAL: Static memory layout is selected, amount of reserved memory can > be adjusted with -m or --socket-mem > EAL: Detected static linkage of DPDK > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: Selected IOVA mode 'VA' > EAL: VFIO support initialized > > With this change on the same system, > > EAL: Detected CPU lcores: 128 > EAL: Detected NUMA nodes: 16 > EAL: Static memory layout is selected, amount of reserved memory can > be adjusted with -m or --socket-mem > EAL: Detected static linkage of DPDK > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: Selected IOVA mode 'VA' > EAL: VFIO support initialized > > Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection") > Cc: stable@dpdk.org > > Signed-off-by: Peng Zhang <peng.zhang@corigine.com> > Signed-off-by: Chaoyong He <chaoyong.he@corigine.com> > Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com> > --- > buildtools/get-numa-count.py | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/buildtools/get-numa-count.py b/buildtools/get-numa-count.py > index 1b7787787f71..2f243886cd21 100644 > --- a/buildtools/get-numa-count.py > +++ b/buildtools/get-numa-count.py > @@ -6,11 +6,12 @@ > import glob > import os > import subprocess > +import re > > if os.name == 'posix': > if os.path.isdir('/sys/devices/system/node'): > numa_nodes = glob.glob('/sys/devices/system/node/node*') > - numa_nodes.sort() > + numa_nodes.sort(key=lambda l: int(re.findall('\d+', l)[0])) > print(int(os.path.basename(numa_nodes[-1])[4:]) + 1) > else: > subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False) > -- > 2.37.3 >
22/09/2022 10:06, Niklas Söderlund: > From: Peng Zhang <peng.zhang@corigine.com> > > The method to fetch, sort and read the last entry of a list to figure > out the total number of NUMA nodes in the system fails with 10 or more > nodes. The reason being the usage of string compare while sorting, hence > node 'node10' will be sorted before 'node2'. > > Solve this by sorting the list based on integer comparison of the > numerical part of the node name. > > Before this change on a system with 16 NUMA nodes, > > EAL: Detected CPU lcores: 128 > EAL: Detected NUMA nodes: 10 > EAL: Static memory layout is selected, amount of reserved memory can > be adjusted with -m or --socket-mem > EAL: Detected static linkage of DPDK > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: Selected IOVA mode 'VA' > EAL: VFIO support initialized > > With this change on the same system, > > EAL: Detected CPU lcores: 128 > EAL: Detected NUMA nodes: 16 > EAL: Static memory layout is selected, amount of reserved memory can > be adjusted with -m or --socket-mem > EAL: Detected static linkage of DPDK > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: Selected IOVA mode 'VA' > EAL: VFIO support initialized > > Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection") > Cc: stable@dpdk.org > > Signed-off-by: Peng Zhang <peng.zhang@corigine.com> > Signed-off-by: Chaoyong He <chaoyong.he@corigine.com> > Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com> Applied, thanks.
diff --git a/buildtools/get-numa-count.py b/buildtools/get-numa-count.py index 1b7787787f71..2f243886cd21 100644 --- a/buildtools/get-numa-count.py +++ b/buildtools/get-numa-count.py @@ -6,11 +6,12 @@ import glob import os import subprocess +import re if os.name == 'posix': if os.path.isdir('/sys/devices/system/node'): numa_nodes = glob.glob('/sys/devices/system/node/node*') - numa_nodes.sort() + numa_nodes.sort(key=lambda l: int(re.findall('\d+', l)[0])) print(int(os.path.basename(numa_nodes[-1])[4:]) + 1) else: subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False)