[v6] eal: add generic thread-local-storage functions
Checks
Commit Message
Add support for tls functionality in EAL.
The following functions are added:
rte_thread_tls_create_key - function to create a tls data key.
rte_thread_tls_delete_key - function to delete a tls data key.
rte_thread_tls_set_value - function to set value bound to the tls key
rte_thread_tls_get_value - function to get value bound to the tls key
tls key will be defined by the new type rte_tls_key
Windows implementation is under librte_eal/windows and
implemented using WIN32 API for Windows only.
common implementation is under librte_eal/common and
implemented using pthread for UNIX and Windows compilation
using extenral pthread libraries, when supported.
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
v3: switch from pthread shim to generic eal implementation [DmitryK]
v4: modify file names, function names, move unix code to common
for future external pthreads support [DmitryK]
v5: rename var used for extenal pthreads, add description in
meson_options.txt. [DmitryK]
v6: remove external_pthread support as it collide with pthread
shim implementation [DmitryK]
---
lib/librte_eal/common/meson.build | 1 +
lib/librte_eal/common/rte_thread.c | 86 ++++++++++++++++++++++++++++++++++++
lib/librte_eal/include/meson.build | 1 +
lib/librte_eal/include/rte_thread.h | 88 +++++++++++++++++++++++++++++++++++++
lib/librte_eal/rte_eal_exports.def | 5 +++
lib/librte_eal/version.map | 6 +++
lib/librte_eal/windows/meson.build | 1 +
lib/librte_eal/windows/rte_thread.c | 82 ++++++++++++++++++++++++++++++++++
8 files changed, 270 insertions(+)
create mode 100644 lib/librte_eal/common/rte_thread.c
create mode 100644 lib/librte_eal/include/rte_thread.h
create mode 100644 lib/librte_eal/windows/rte_thread.c
Comments
On Wed, 30 Dec 2020 13:12:44 +0200, Tal Shnaiderman wrote:
> Add support for tls functionality in EAL.
>
> The following functions are added:
> rte_thread_tls_create_key - function to create a tls data key.
> rte_thread_tls_delete_key - function to delete a tls data key.
> rte_thread_tls_set_value - function to set value bound to the tls key
> rte_thread_tls_get_value - function to get value bound to the tls key
>
> tls key will be defined by the new type rte_tls_key
>
> Windows implementation is under librte_eal/windows and
> implemented using WIN32 API for Windows only.
>
> common implementation is under librte_eal/common and
> implemented using pthread for UNIX and Windows compilation
> using extenral pthread libraries, when supported.
>
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> ---
> v3: switch from pthread shim to generic eal implementation [DmitryK]
> v4: modify file names, function names, move unix code to common
> for future external pthreads support [DmitryK]
> v5: rename var used for extenal pthreads, add description in
> meson_options.txt. [DmitryK]
> v6: remove external_pthread support as it collide with pthread
> shim implementation [DmitryK]
> ---
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
30/12/2020 12:12, Tal Shnaiderman:
> Add support for tls functionality in EAL.
You should write TLS uppercase here and below.
There is already a TLS capability in rte_per_lcore.h,
using __thread keyword.
Note: I think the historical wording "lcore" is confusing.
We talk about lcore, assuming the thread is bound to its lcore.
Please explain in the patch why the more complex TLS functions
are required, i.e. can __thread variable be used in Windows?
limitation, etc.
> The following functions are added:
> rte_thread_tls_create_key - function to create a tls data key.
> rte_thread_tls_delete_key - function to delete a tls data key.
> rte_thread_tls_set_value - function to set value bound to the tls key
> rte_thread_tls_get_value - function to get value bound to the tls key
We can assume the TLS key logic is known,
but this patch is the right place to explain it.
> tls key will be defined by the new type rte_tls_key
>
> Windows implementation is under librte_eal/windows and
> implemented using WIN32 API for Windows only.
>
> common implementation is under librte_eal/common and
> implemented using pthread for UNIX and Windows compilation
> using extenral pthread libraries, when supported.
typo: external
[...]
> v3: switch from pthread shim to generic eal implementation [DmitryK]
> v4: modify file names, function names, move unix code to common
> for future external pthreads support [DmitryK]
For now, there is no external pthread library.
It did not have been discussed generally in the techboard.
As it is using the Unix pthread API, it sounds more logical
to have this implementation in the Unix sub-directory I think.
> --- a/lib/librte_eal/common/meson.build
> +++ b/lib/librte_eal/common/meson.build
> + 'rte_thread.c',
[...]
> --- a/lib/librte_eal/include/meson.build
> +++ b/lib/librte_eal/include/meson.build
> + 'rte_thread.h',
There are existing functions in rte_lcore.h and rte_per_lcore.h
which are managing threads.
I think a prior patch should move such functions in rte_thread files.
At least we can start moving the affinity functions,
which are not using pthread types as parameters.
> --- /dev/null
> +++ b/lib/librte_eal/include/rte_thread.h
> @@ -0,0 +1,88 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2020 Mellanox Technologies, Ltd
> + */
> +
> +#include <rte_compat.h>
Why do you need rte_compat?
> +/**
> + * Function to create a TLS data key visible to all threads in the process
> + * function need to be called once to create a key usable by all threads.
> + * rte_tls_key is an opaque pointer used to store the allocated key.
> + * and optional destructor can be set to be called when a thread expires.
You may explain that the key is later used to get/set a value.
> + *
> + * @param key
> + * Pointer to store the allocated rte_tls_key
> + * @param destructor
> + * The function to be called when the thread expires.
> + * Not supported on Windows OS.
What happens when specifying a destructor on Windows?
> + *
> + * @return
> + * On success, zero.
> + * On failure, a negative number.
> + */
> +__rte_experimental
> +int
> +rte_thread_tls_create_key(rte_tls_key *key, void (*destructor)(void *));
For .h files, we have the return type on the same line (I know it is stupid).
I would suggest moving the verb "create" at the end of the function names.
[...]
> +rte_thread_tls_delete_key(rte_tls_key key);
Could be rte_thread_tls_key_delete?
[...]
> +rte_thread_tls_set_value(rte_tls_key key, const void *value);
rte_thread_tls_value_set?
> +rte_thread_tls_get_value(rte_tls_key key);
rte_thread_tls_value_get?
@@ -77,6 +77,7 @@ sources += files(
'rte_random.c',
'rte_reciprocal.c',
'rte_service.c',
+ 'rte_thread.c',
)
if is_linux
new file mode 100644
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <pthread.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_common.h>
+#include <rte_errno.h>
+#include <rte_log.h>
+#include <rte_thread.h>
+
+struct eal_tls_key {
+ pthread_key_t thread_index;
+};
+
+int
+rte_thread_tls_create_key(rte_tls_key *key, void (*destructor)(void *))
+{
+ int err;
+
+ *key = malloc(sizeof(struct eal_tls_key));
+ if ((*key) == NULL) {
+ RTE_LOG(DEBUG, EAL, "Cannot allocate tls key.");
+ return -1;
+ }
+ err = pthread_key_create(&((*key)->thread_index), destructor);
+ if (err) {
+ RTE_LOG(DEBUG, EAL, "pthread_key_create failed: %s\n",
+ strerror(err));
+ free(*key);
+ return -1;
+ }
+ return 0;
+}
+
+int
+rte_thread_tls_delete_key(rte_tls_key key)
+{
+ int err;
+
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+ return -1;
+ }
+ err = pthread_key_delete(key->thread_index);
+ if (err) {
+ RTE_LOG(DEBUG, EAL, "pthread_key_delete failed: %s\n",
+ strerror(err));
+ free(key);
+ return -1;
+ }
+ free(key);
+ return 0;
+}
+
+int
+rte_thread_tls_set_value(rte_tls_key key, const void *value)
+{
+ int err;
+
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+ return -1;
+ }
+ err = pthread_setspecific(key->thread_index, value);
+ if (err) {
+ RTE_LOG(DEBUG, EAL, "pthread_setspecific failed: %s\n",
+ strerror(err));
+ return -1;
+ }
+ return 0;
+}
+
+void *
+rte_thread_tls_get_value(rte_tls_key key)
+{
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ return pthread_getspecific(key->thread_index);
+}
@@ -40,6 +40,7 @@ headers += files(
'rte_service_component.h',
'rte_string_fns.h',
'rte_tailq.h',
+ 'rte_thread.h',
'rte_time.h',
'rte_trace.h',
'rte_trace_point.h',
new file mode 100644
@@ -0,0 +1,88 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2020 Mellanox Technologies, Ltd
+ */
+
+#include <rte_compat.h>
+
+#ifndef _RTE_THREAD_H_
+#define _RTE_THREAD_H_
+
+/**
+ * @file
+ *
+ * Threading functions
+ *
+ * Simple threads functionality supplied by EAL.
+ */
+
+/**
+ * Opaque pointer for TLS key.
+ */
+typedef struct eal_tls_key *rte_tls_key;
+
+/**
+ * Function to create a TLS data key visible to all threads in the process
+ * function need to be called once to create a key usable by all threads.
+ * rte_tls_key is an opaque pointer used to store the allocated key.
+ * and optional destructor can be set to be called when a thread expires.
+ *
+ * @param key
+ * Pointer to store the allocated rte_tls_key
+ * @param destructor
+ * The function to be called when the thread expires.
+ * Not supported on Windows OS.
+ *
+ * @return
+ * On success, zero.
+ * On failure, a negative number.
+ */
+__rte_experimental
+int
+rte_thread_tls_create_key(rte_tls_key *key, void (*destructor)(void *));
+
+/**
+ * Function to delete a TLS data key visible to all threads in the process
+ * rte_tls_key is the opaque pointer allocated by rte_thread_tls_create_key.
+ *
+ * @param key
+ * The rte_tls_key will contain the allocated key
+ *
+ * @return
+ * On success, zero.
+ * On failure, a negative number.
+ */
+__rte_experimental
+int
+rte_thread_tls_delete_key(rte_tls_key key);
+
+/**
+ * Function to set value bound to the tls key on behalf of the calling thread
+ *
+ * @param key
+ * The rte_tls_key key allocated by rte_thread_tls_create_key.
+ * @param value
+ * The value bound to the rte_tls_key key for the calling thread.
+ *
+ * @return
+ * On success, zero.
+ * On failure, a negative number.
+ */
+__rte_experimental
+int
+rte_thread_tls_set_value(rte_tls_key key, const void *value);
+
+/**
+ * Function to get value bound to the tls key on behalf of the calling thread
+ *
+ * @param key
+ * The rte_tls_key key allocated by rte_thread_tls_create_key.
+ *
+ * @return
+ * On success, value data pointer (can also be NULL).
+ * On failure, NULL and an error number is set in rte_errno.
+ */
+__rte_experimental
+void *
+rte_thread_tls_get_value(rte_tls_key key);
+
+#endif /* _RTE_THREAD_H_ */
@@ -308,6 +308,11 @@ EXPORTS
rte_vect_get_max_simd_bitwidth
rte_vect_set_max_simd_bitwidth
+ rte_thread_tls_create_key
+ rte_thread_tls_delete_key
+ rte_thread_tls_set_value
+ rte_thread_tls_get_value
+
rte_mem_lock
rte_mem_map
rte_mem_page_size
@@ -403,6 +403,12 @@ EXPERIMENTAL {
rte_service_lcore_may_be_active;
rte_vect_get_max_simd_bitwidth;
rte_vect_set_max_simd_bitwidth;
+
+ # added in 21.02
+ rte_thread_tls_create_key;
+ rte_thread_tls_delete_key;
+ rte_thread_tls_set_value;
+ rte_thread_tls_get_value;
};
INTERNAL {
@@ -19,6 +19,7 @@ sources += files(
'eal_timer.c',
'fnmatch.c',
'getopt.c',
+ 'rte_thread.c',
)
dpdk_conf.set10('RTE_EAL_NUMA_AWARE_HUGEPAGES', true)
new file mode 100644
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+#include <rte_common.h>
+#include <rte_errno.h>
+#include <rte_thread.h>
+#include <rte_windows.h>
+
+struct eal_tls_key {
+ DWORD thread_index;
+};
+
+int
+rte_thread_tls_create_key(rte_tls_key *key,
+ __rte_unused void (*destructor)(void *))
+{
+ *key = malloc(sizeof(struct eal_tls_key));
+ if ((*key) == NULL) {
+ RTE_LOG(DEBUG, EAL, "Cannot allocate tls key.");
+ return -1;
+ }
+ (*key)->thread_index = TlsAlloc();
+ if ((*key)->thread_index == TLS_OUT_OF_INDEXES) {
+ RTE_LOG_WIN32_ERR("TlsAlloc()");
+ free(*key);
+ return -1;
+ }
+ return 0;
+}
+
+int
+rte_thread_tls_delete_key(rte_tls_key key)
+{
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+ return -1;
+ }
+ if (!TlsFree(key->thread_index)) {
+ RTE_LOG_WIN32_ERR("TlsFree()");
+ free(key);
+ return -1;
+ }
+ free(key);
+ return 0;
+}
+
+int
+rte_thread_tls_set_value(rte_tls_key key, const void *value)
+{
+ char *p;
+
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+ return -1;
+ }
+ /* discard const qualifier */
+ p = (char *) (uintptr_t) value;
+ if (!TlsSetValue(key->thread_index, p)) {
+ RTE_LOG_WIN32_ERR("TlsSetValue()");
+ return -1;
+ }
+ return 0;
+}
+
+void *
+rte_thread_tls_get_value(rte_tls_key key)
+{
+ void *output;
+
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ output = TlsGetValue(key->thread_index);
+ if (GetLastError() != ERROR_SUCCESS) {
+ RTE_LOG_WIN32_ERR("TlsGetValue()");
+ rte_errno = ENOEXEC;
+ return NULL;
+ }
+ return output;
+}