[v8,08/10] eal: add minimum viable code for eal on windows
Checks
Commit Message
Add windows specific logic for eal.c, eal_lcore.c,
eal_debug.c and eal_thread.c. Updated header files to
contain suitable function declaractions.
Signed-off-by: Anand Rawat <anand.rawat@intel.com>
Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
Reviewed-by: Jeff Shaw <jeffrey.b.shaw@intel.com>
Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
---
lib/librte_eal/windows/eal/eal.c | 76 +++++++++-
lib/librte_eal/windows/eal/eal_debug.c | 14 +-
lib/librte_eal/windows/eal/eal_lcore.c | 102 +++++++++++---
lib/librte_eal/windows/eal/eal_thread.c | 145 +++++++++++++++++++-
lib/librte_eal/windows/eal/include/regex.h | 2 +
lib/librte_eal/windows/eal/include/rte_os.h | 20 +++
lib/librte_eal/windows/eal/meson.build | 2 +
7 files changed, 331 insertions(+), 30 deletions(-)
Comments
02/04/2019 05:54, Anand Rawat:
> Add windows specific logic for eal.c, eal_lcore.c,
> eal_debug.c and eal_thread.c. Updated header files to
> contain suitable function declaractions.
>
> Signed-off-by: Anand Rawat <anand.rawat@intel.com>
> Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
> Reviewed-by: Jeff Shaw <jeffrey.b.shaw@intel.com>
> Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
> ---
[...]
> + /*
> + * create communication pipes between master thread
> + * and children
> + */
> + if (_pipe(lcore_config[i].pipe_master2slave,
> + sizeof(char), _O_BINARY) < 0)
> + rte_panic("Cannot create pipe\n");
rte_panic should be used only in applications, not in libraries.
The application can do something in case of DPDK failure.
Please remove all calls to rte_panic() during next weeks, thanks.
On 4/2/2019 1:06 PM, Thomas Monjalon wrote:
> 02/04/2019 05:54, Anand Rawat:
>> Add windows specific logic for eal.c, eal_lcore.c,
>> eal_debug.c and eal_thread.c. Updated header files to
>> contain suitable function declaractions.
>>
>> Signed-off-by: Anand Rawat <anand.rawat@intel.com>
>> Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
>> Reviewed-by: Jeff Shaw <jeffrey.b.shaw@intel.com>
>> Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
>> ---
> [...]
>> + /*
>> + * create communication pipes between master thread
>> + * and children
>> + */
>> + if (_pipe(lcore_config[i].pipe_master2slave,
>> + sizeof(char), _O_BINARY) < 0)
>> + rte_panic("Cannot create pipe\n");
> rte_panic should be used only in applications, not in libraries.
> The application can do something in case of DPDK failure.
> Please remove all calls to rte_panic() during next weeks, thanks.
Ok, will try to incorporate in v9. Just curious, is this the major issue?
Calls to rte_panic() can be seen in linux/eal and freebsd/eal.
>
>
02/04/2019 23:21, Pallavi Kadam:
>
> On 4/2/2019 1:06 PM, Thomas Monjalon wrote:
> > 02/04/2019 05:54, Anand Rawat:
> >> Add windows specific logic for eal.c, eal_lcore.c,
> >> eal_debug.c and eal_thread.c. Updated header files to
> >> contain suitable function declaractions.
> >>
> >> Signed-off-by: Anand Rawat <anand.rawat@intel.com>
> >> Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
> >> Reviewed-by: Jeff Shaw <jeffrey.b.shaw@intel.com>
> >> Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
> >> ---
> > [...]
> >> + /*
> >> + * create communication pipes between master thread
> >> + * and children
> >> + */
> >> + if (_pipe(lcore_config[i].pipe_master2slave,
> >> + sizeof(char), _O_BINARY) < 0)
> >> + rte_panic("Cannot create pipe\n");
> > rte_panic should be used only in applications, not in libraries.
> > The application can do something in case of DPDK failure.
> > Please remove all calls to rte_panic() during next weeks, thanks.
>
> Ok, will try to incorporate in v9. Just curious, is this the major issue?
>
> Calls to rte_panic() can be seen in linux/eal and freebsd/eal.
No, there is a misunderstanding.
I am merging v8 and I am asking for a new patch to remove these calls.
By the way, I am fixing patch 3, please check my comment on this patch.
On 4/2/2019 2:38 PM, Thomas Monjalon wrote:
> 02/04/2019 23:21, Pallavi Kadam:
>> On 4/2/2019 1:06 PM, Thomas Monjalon wrote:
>>> 02/04/2019 05:54, Anand Rawat:
>>>> Add windows specific logic for eal.c, eal_lcore.c,
>>>> eal_debug.c and eal_thread.c. Updated header files to
>>>> contain suitable function declaractions.
>>>>
>>>> Signed-off-by: Anand Rawat <anand.rawat@intel.com>
>>>> Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
>>>> Reviewed-by: Jeff Shaw <jeffrey.b.shaw@intel.com>
>>>> Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
>>>> ---
>>> [...]
>>>> + /*
>>>> + * create communication pipes between master thread
>>>> + * and children
>>>> + */
>>>> + if (_pipe(lcore_config[i].pipe_master2slave,
>>>> + sizeof(char), _O_BINARY) < 0)
>>>> + rte_panic("Cannot create pipe\n");
>>> rte_panic should be used only in applications, not in libraries.
>>> The application can do something in case of DPDK failure.
>>> Please remove all calls to rte_panic() during next weeks, thanks.
>> Ok, will try to incorporate in v9. Just curious, is this the major issue?
>>
>> Calls to rte_panic() can be seen in linux/eal and freebsd/eal.
> No, there is a misunderstanding.
> I am merging v8 and I am asking for a new patch to remove these calls.
>
> By the way, I am fixing patch 3, please check my comment on this patch.
Got it. Thanks.
>
>
>
I think the change below is bad because it is exposing an internal
function in a header file which may be included by apps.
I know you are still looking for pthread integration solutions,
so we can discuss how to fix it later.
Note that there is already an EAL function to create normal threads:
rte_ctrl_thread_create()
You may want a similar function for dataplane threads perhaps.
02/04/2019 05:54, Anand Rawat:
> --- a/lib/librte_eal/windows/eal/include/rte_os.h
> +++ b/lib/librte_eal/windows/eal/include/rte_os.h
> @@ -15,7 +15,9 @@
> extern "C" {
> #endif
>
> +#include <Windows.h>
> #include <BaseTsd.h>
> +#include <pthread.h>
>
> #define strerror_r(a, b, c) strerror_s(b, c, a)
>
> @@ -26,6 +28,24 @@ typedef SSIZE_T ssize_t;
>
> #define strtok_r(str, delim, saveptr) strtok_s(str, delim, saveptr)
>
> +/**
> + * Create a thread.
> + * This function is private to EAL.
> + *
> + * @param thread
> + * The location to store the thread id if successful.
> + * @return
> + * 0 for success, -1 if the thread is not created.
> + */
> +int eal_thread_create(pthread_t *thread);
> +
> +/**
> + * Create a map of processors and cores on the system.
> + * This function is private to EAL.
> + *
> + */
> +void eal_create_cpu_map(void);
On 4/2/2019 3:39 PM, Thomas Monjalon wrote:
> I think the change below is bad because it is exposing an internal
> function in a header file which may be included by apps.
> I know you are still looking for pthread integration solutions,
> so we can discuss how to fix it later.
>
> Note that there is already an EAL function to create normal threads:
> rte_ctrl_thread_create()
> You may want a similar function for dataplane threads perhaps.
>
>
Yes, we can work on removing eal_thread_create and substituting it
with rte_ctrl_thread_create. We have been able to macro substitute
quite a lot of pthread lib functionality with Microsoft libc. I can
start looking into porting those functionalities for
rte_ctrl_thread_create() on Windows.
@@ -2,13 +2,83 @@
* Copyright(c) 2019 Intel Corporation
*/
-#include <rte_common.h>
+#include <io.h>
+#include <fcntl.h>
+#include <rte_debug.h>
+#include <rte_eal.h>
+#include <rte_errno.h>
+#include <rte_lcore.h>
+#include <eal_thread.h>
+#include <eal_private.h>
+
+/* Address of global and public configuration */
+static struct rte_config rte_config;
+
+/* internal configuration (per-core) */
+struct lcore_config lcore_config[RTE_MAX_LCORE];
+
+/* Return a pointer to the configuration structure */
+struct rte_config *
+rte_eal_get_configuration(void)
+{
+ return &rte_config;
+}
+
+static int
+sync_func(void *arg __rte_unused)
+{
+ return 0;
+}
+
+static void
+rte_eal_init_alert(const char *msg)
+{
+ fprintf(stderr, "EAL: FATAL: %s\n", msg);
+ RTE_LOG(ERR, EAL, "%s\n", msg);
+}
/* Launch threads, called at application init(). */
int
rte_eal_init(int argc __rte_unused, char **argv __rte_unused)
{
- /* TODO */
- /* This is a stub, not the expected result */
+ int i;
+
+ /* create a map of all processors in the system */
+ eal_create_cpu_map();
+
+ if (rte_eal_cpu_init() < 0) {
+ rte_eal_init_alert("Cannot detect lcores.");
+ rte_errno = ENOTSUP;
+ return -1;
+ }
+
+ eal_thread_init_master(rte_config.master_lcore);
+
+ RTE_LCORE_FOREACH_SLAVE(i) {
+
+ /*
+ * create communication pipes between master thread
+ * and children
+ */
+ if (_pipe(lcore_config[i].pipe_master2slave,
+ sizeof(char), _O_BINARY) < 0)
+ rte_panic("Cannot create pipe\n");
+ if (_pipe(lcore_config[i].pipe_slave2master,
+ sizeof(char), _O_BINARY) < 0)
+ rte_panic("Cannot create pipe\n");
+
+ lcore_config[i].state = WAIT;
+
+ /* create a thread for each lcore */
+ if (eal_thread_create(&lcore_config[i].thread_id) != 0)
+ rte_panic("Cannot create thread\n");
+ }
+
+ /*
+ * Launch a dummy function on all slave lcores, so that master lcore
+ * knows they are all ready when this function returns.
+ */
+ rte_eal_mp_remote_launch(sync_func, NULL, SKIP_MASTER);
+ rte_eal_mp_wait_lcore();
return 0;
}
@@ -2,14 +2,18 @@
* Copyright(c) 2019 Intel Corporation
*/
-#include <rte_common.h>
+#include <stdarg.h>
+#include <rte_log.h>
/* call abort(), it will generate a coredump if enabled */
void
-__rte_panic(const char *funcname __rte_unused,
- const char *format __rte_unused, ...)
+__rte_panic(const char *funcname, const char *format, ...)
{
- /* TODO */
- /* This is a stub, not the expected result */
+ va_list ap;
+
+ rte_log(RTE_LOG_CRIT, RTE_LOGTYPE_EAL, "PANIC in %s():\n", funcname);
+ va_start(ap, format);
+ rte_vlog(RTE_LOG_CRIT, RTE_LOGTYPE_EAL, format, ap);
+ va_end(ap);
abort();
}
@@ -2,31 +2,99 @@
* Copyright(c) 2019 Intel Corporation
*/
+#include <stdint.h>
+
#include <rte_common.h>
- /* Get the cpu core id value */
-unsigned int
-eal_cpu_core_id(unsigned int lcore_id)
+/* global data structure that contains the CPU map */
+static struct _wcpu_map {
+ unsigned int total_procs;
+ unsigned int proc_sockets;
+ unsigned int proc_cores;
+ unsigned int reserved;
+ struct _win_lcore_map {
+ uint8_t socket_id;
+ uint8_t core_id;
+ } wlcore_map[RTE_MAX_LCORE];
+} wcpu_map = { 0 };
+
+/*
+ * Create a map of all processors and associated cores on the system
+ */
+void
+eal_create_cpu_map()
{
- /* TODO */
- /* This is a stub, not the expected result */
- return lcore_id;
+ wcpu_map.total_procs =
+ GetActiveProcessorCount(ALL_PROCESSOR_GROUPS);
+
+ LOGICAL_PROCESSOR_RELATIONSHIP lprocRel;
+ DWORD lprocInfoSize = 0;
+ BOOL ht_enabled = FALSE;
+
+ /* First get the processor package information */
+ lprocRel = RelationProcessorPackage;
+ /* Determine the size of buffer we need (pass NULL) */
+ GetLogicalProcessorInformationEx(lprocRel, NULL, &lprocInfoSize);
+ wcpu_map.proc_sockets = lprocInfoSize / 48;
+
+ lprocInfoSize = 0;
+ /* Next get the processor core information */
+ lprocRel = RelationProcessorCore;
+ GetLogicalProcessorInformationEx(lprocRel, NULL, &lprocInfoSize);
+ wcpu_map.proc_cores = lprocInfoSize / 48;
+
+ if (wcpu_map.total_procs > wcpu_map.proc_cores)
+ ht_enabled = TRUE;
+
+ /* Distribute the socket and core ids appropriately
+ * across the logical cores. For now, split the cores
+ * equally across the sockets.
+ */
+ unsigned int lcore = 0;
+ for (unsigned int socket = 0; socket <
+ wcpu_map.proc_sockets; ++socket) {
+ for (unsigned int core = 0;
+ core < (wcpu_map.proc_cores / wcpu_map.proc_sockets);
+ ++core) {
+ wcpu_map.wlcore_map[lcore]
+ .socket_id = socket;
+ wcpu_map.wlcore_map[lcore]
+ .core_id = core;
+ lcore++;
+ if (ht_enabled) {
+ wcpu_map.wlcore_map[lcore]
+ .socket_id = socket;
+ wcpu_map.wlcore_map[lcore]
+ .core_id = core;
+ lcore++;
+ }
+ }
+ }
}
-/* Check if a cpu is present by the presence of the cpu information for it */
+/*
+ * Check if a cpu is present by the presence of the cpu information for it
+ */
int
-eal_cpu_detected(unsigned int lcore_id __rte_unused)
+eal_cpu_detected(unsigned int lcore_id)
+{
+ return (lcore_id < wcpu_map.total_procs);
+}
+
+/*
+ * Get CPU socket id for a logical core
+ */
+unsigned
+eal_cpu_socket_id(unsigned int lcore_id)
{
- /* TODO */
- /* This is a stub, not the expected result */
- return 1;
+ return wcpu_map.wlcore_map[lcore_id].socket_id;
}
-/* Get CPU socket id (NUMA node) for a logical core */
-unsigned int
-eal_cpu_socket_id(unsigned int cpu_id __rte_unused)
+/*
+ * Get CPU socket id (NUMA node) for a logical core
+ */
+unsigned
+eal_cpu_core_id(unsigned int lcore_id)
{
- /* TODO */
- /* This is a stub, not the expected result */
- return 0;
+ return wcpu_map.wlcore_map[lcore_id].core_id;
}
@@ -2,17 +2,152 @@
* Copyright(c) 2019 Intel Corporation
*/
-#include <windows.h>
+#include <io.h>
+#include <rte_atomic.h>
+#include <rte_debug.h>
+#include <rte_launch.h>
+#include <rte_lcore.h>
+#include <rte_per_lcore.h>
#include <rte_common.h>
+#include <eal_thread.h>
-typedef uintptr_t eal_thread_t;
+
+RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY;
+
+/*
+ * Send a message to a slave lcore identified by slave_id to call a
+ * function f with argument arg. Once the execution is done, the
+ * remote lcore switch in FINISHED state.
+ */
+int
+rte_eal_remote_launch(lcore_function_t *f, void *arg, unsigned int slave_id)
+{
+ int n;
+ char c = 0;
+ int m2s = lcore_config[slave_id].pipe_master2slave[1];
+ int s2m = lcore_config[slave_id].pipe_slave2master[0];
+
+ if (lcore_config[slave_id].state != WAIT)
+ return -EBUSY;
+
+ lcore_config[slave_id].f = f;
+ lcore_config[slave_id].arg = arg;
+
+ /* send message */
+ n = 0;
+ while (n == 0 || (n < 0 && errno == EINTR))
+ n = _write(m2s, &c, 1);
+ if (n < 0)
+ rte_panic("cannot write on configuration pipe\n");
+
+ /* wait ack */
+ do {
+ n = _read(s2m, &c, 1);
+ } while (n < 0 && errno == EINTR);
+
+ if (n <= 0)
+ rte_panic("cannot read on configuration pipe\n");
+
+ return 0;
+}
+
+void
+eal_thread_init_master(unsigned int lcore_id)
+{
+ /* set the lcore ID in per-lcore memory area */
+ RTE_PER_LCORE(_lcore_id) = lcore_id;
+}
+
+static inline pthread_t
+eal_thread_self(void)
+{
+ return GetCurrentThreadId();
+}
+
+/* main loop of threads */
+void *
+eal_thread_loop(void *arg __rte_unused)
+{
+ char c;
+ int n, ret;
+ unsigned int lcore_id;
+ pthread_t thread_id;
+ int m2s, s2m;
+ char cpuset[RTE_CPU_AFFINITY_STR_LEN];
+
+ thread_id = eal_thread_self();
+
+ /* retrieve our lcore_id from the configuration structure */
+ RTE_LCORE_FOREACH_SLAVE(lcore_id) {
+ if (thread_id == lcore_config[lcore_id].thread_id)
+ break;
+ }
+ if (lcore_id == RTE_MAX_LCORE)
+ rte_panic("cannot retrieve lcore id\n");
+
+ m2s = lcore_config[lcore_id].pipe_master2slave[0];
+ s2m = lcore_config[lcore_id].pipe_slave2master[1];
+
+ /* set the lcore ID in per-lcore memory area */
+ RTE_PER_LCORE(_lcore_id) = lcore_id;
+
+ RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s])\n",
+ lcore_id, (uintptr_t)thread_id, cpuset);
+
+ /* read on our pipe to get commands */
+ while (1) {
+ void *fct_arg;
+
+ /* wait command */
+ do {
+ n = _read(m2s, &c, 1);
+ } while (n < 0 && errno == EINTR);
+
+ if (n <= 0)
+ rte_panic("cannot read on configuration pipe\n");
+
+ lcore_config[lcore_id].state = RUNNING;
+
+ /* send ack */
+ n = 0;
+ while (n == 0 || (n < 0 && errno == EINTR))
+ n = _write(s2m, &c, 1);
+ if (n < 0)
+ rte_panic("cannot write on configuration pipe\n");
+
+ if (lcore_config[lcore_id].f == NULL)
+ rte_panic("NULL function pointer\n");
+
+ /* call the function and store the return value */
+ fct_arg = lcore_config[lcore_id].arg;
+ ret = lcore_config[lcore_id].f(fct_arg);
+ lcore_config[lcore_id].ret = ret;
+ rte_wmb();
+
+ /* when a service core returns, it should go directly to WAIT
+ * state, because the application will not lcore_wait() for it.
+ */
+ if (lcore_config[lcore_id].core_role == ROLE_SERVICE)
+ lcore_config[lcore_id].state = WAIT;
+ else
+ lcore_config[lcore_id].state = FINISHED;
+ }
+}
/* function to create threads */
int
-eal_thread_create(eal_thread_t *thread __rte_unused)
+eal_thread_create(pthread_t *thread)
{
- /* TODO */
- /* This is a stub, not the expected result */
+ HANDLE th;
+
+ th = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)eal_thread_loop,
+ NULL, 0, (LPDWORD)thread);
+ if (!th)
+ return -1;
+
+ SetPriorityClass(GetCurrentProcess(), REALTIME_PRIORITY_CLASS);
+ SetThreadPriority(th, THREAD_PRIORITY_TIME_CRITICAL);
+
return 0;
}
@@ -17,6 +17,8 @@ extern "C" {
#define REG_NOMATCH 1
#define REG_ESPACE 12
+#include <rte_common.h>
+
/* defining regex_t for Windows */
typedef void *regex_t;
/* defining regmatch_t for Windows */
@@ -15,7 +15,9 @@
extern "C" {
#endif
+#include <Windows.h>
#include <BaseTsd.h>
+#include <pthread.h>
#define strerror_r(a, b, c) strerror_s(b, c, a)
@@ -26,6 +28,24 @@ typedef SSIZE_T ssize_t;
#define strtok_r(str, delim, saveptr) strtok_s(str, delim, saveptr)
+/**
+ * Create a thread.
+ * This function is private to EAL.
+ *
+ * @param thread
+ * The location to store the thread id if successful.
+ * @return
+ * 0 for success, -1 if the thread is not created.
+ */
+int eal_thread_create(pthread_t *thread);
+
+/**
+ * Create a map of processors and cores on the system.
+ * This function is private to EAL.
+ *
+ */
+void eal_create_cpu_map(void);
+
#ifdef __cplusplus
}
#endif
@@ -1,6 +1,8 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2019 Intel Corporation
+eal_inc += include_directories('include')
+
env_objs = []
env_headers = []
env_sources = files('eal.c',