From patchwork Thu Apr 23 05:11:06 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ramia, Kannan Babu" X-Patchwork-Id: 4447 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 20CC76A94; Thu, 23 Apr 2015 07:12:03 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 7BBD66942 for ; Thu, 23 Apr 2015 07:12:00 +0200 (CEST) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP; 22 Apr 2015 22:12:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.11,628,1422950400"; d="scan'208";a="699588249" Received: from kmsmsx152.gar.corp.intel.com ([172.21.73.87]) by fmsmga001.fm.intel.com with ESMTP; 22 Apr 2015 22:11:58 -0700 Received: from bgsmsx151.gar.corp.intel.com (10.224.48.42) by KMSMSX152.gar.corp.intel.com (172.21.73.87) with Microsoft SMTP Server (TLS) id 14.3.224.2; Thu, 23 Apr 2015 13:11:07 +0800 Received: from bgsmsx102.gar.corp.intel.com ([169.254.2.22]) by BGSMSX151.gar.corp.intel.com ([10.224.48.42]) with mapi id 14.03.0224.002; Thu, 23 Apr 2015 10:41:06 +0530 From: "Ramia, Kannan Babu" To: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH] eal: fix linuxapp numa node detection Thread-Index: AQHQfQDDDjvnZSIT5EmBxov77btVk51aDL+A Date: Thu, 23 Apr 2015 05:11:06 +0000 Message-ID: <682698A055A0F44AA47192B2014114972AE656CC@BGSMSX102.gar.corp.intel.com> References: <1429709515-14361-1-git-send-email-bruce.richardson@intel.com> In-Reply-To: <1429709515-14361-1-git-send-email-bruce.richardson@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.223.10.10] MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] eal: fix linuxapp numa node detection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" BTW, the BIOS fix has been posted for the NUMA node reporting issue in Linux kernel when COD is disabled. Now the /sys/devices/*/*/numa_node will provide you the correct information. Regards Kannan Babu -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson Sent: Wednesday, April 22, 2015 7:02 PM To: dev@dpdk.org Subject: [dpdk-dev] [PATCH] eal: fix linuxapp numa node detection Using the "physical_package_id" as a fallback for determining the numa node of a core tends to be unreliable. Fix this by using a detection routine which reads the numa information from /sys/devices/system/node and just returns a numa node of 0 on failure. Signed-off-by: Bruce Richardson --- lib/librte_eal/linuxapp/eal/eal_lcore.c | 60 ++++++++------------------------- 1 file changed, 14 insertions(+), 46 deletions(-) } -/* Get CPU socket id (NUMA node) by reading directory - * /sys/devices/system/cpu/cpuX looking for symlink "nodeY" - * which gives the NUMA topology information. - * Note: physical package id != NUMA node, but we use it as a - * fallback for kernels which don't create a nodeY link +/* + * Get CPU socket id (NUMA node) for a logical core. + * + * This searches each nodeX directories in /sys for the symlink for the +given + * lcore_id and returns the numa node where the lcore is found. If +lcore is not + * found on any numa node, returns zero. */ unsigned eal_cpu_socket_id(unsigned lcore_id) { - const char node_prefix[] = "node"; - const size_t prefix_len = sizeof(node_prefix) - 1; - char path[PATH_MAX]; - DIR *d = NULL; - unsigned long id = 0; - struct dirent *e; - char *endptr = NULL; - - int len = snprintf(path, sizeof(path), - SYS_CPU_DIR, lcore_id); - if (len <= 0 || (unsigned)len >= sizeof(path)) - goto err; + unsigned socket; - d = opendir(path); - if (!d) - goto err; + for (socket = 0; socket < RTE_MAX_NUMA_NODES; socket++) { + char path[PATH_MAX]; - while ((e = readdir(d)) != NULL) { - if (strncmp(e->d_name, node_prefix, prefix_len) == 0) { - id = strtoul(e->d_name+prefix_len, &endptr, 0); - break; - } - } - if (endptr == NULL || *endptr!='\0' || endptr == e->d_name+prefix_len) { - RTE_LOG(WARNING, EAL, "Cannot read numa node link " - "for lcore %u - using physical package id instead\n", - lcore_id); - - len = snprintf(path, sizeof(path), SYS_CPU_DIR "/%s", - lcore_id, PHYS_PKG_FILE); - if (len <= 0 || (unsigned)len >= sizeof(path)) - goto err; - if (eal_parse_sysfs_value(path, &id) != 0) - goto err; + snprintf(path, sizeof(path),"%s/node%u/cpu%u", NUMA_NODE_PATH, + socket, lcore_id); + if (access(path, F_OK) == 0) + return socket; } - closedir(d); - return (unsigned)id; - -err: - if (d) - closedir(d); - RTE_LOG(ERR, EAL, "Error getting NUMA socket information from %s " - "for lcore %u - assuming NUMA socket 0\n", SYS_CPU_DIR, lcore_id); return 0; } -- 2.1.0 diff --git a/lib/librte_eal/linuxapp/eal/eal_lcore.c b/lib/librte_eal/linuxapp/eal/eal_lcore.c index ef8c433..f25108e 100644 --- a/lib/librte_eal/linuxapp/eal/eal_lcore.c +++ b/lib/librte_eal/linuxapp/eal/eal_lcore.c @@ -49,7 +49,7 @@ #define SYS_CPU_DIR "/sys/devices/system/cpu/cpu%u" #define CORE_ID_FILE "topology/core_id" -#define PHYS_PKG_FILE "topology/physical_package_id" +#define NUMA_NODE_PATH "/sys/devices/system/node" /* Check if a cpu is present by the presence of the cpu information for it */ static int @@ -66,58 +66,26 @@ cpu_detected(unsigned lcore_id) return 1;