[v9,2/2] eal: add generic thread-local-storage functions
Checks
Commit Message
Add support for TLS functionality in EAL.
The following functions are added:
rte_thread_tls_key_create - create a TLS data key.
rte_thread_tls_key_delete - delete a TLS data key.
rte_thread_tls_value_set - set value bound to the TLS key
rte_thread_tls_value_get - get value bound to the TLS key
TLS key is defined by the new type rte_tls_key.
The API allocates the thread local storage (TLS) key.
Any thread of the process can subsequently use this key
to store and retrieve values that are local to the thread.
Those functions are added in addition to TLS capability
in rte_per_lcore.h to allow abstraction of the pthread
layer for all operating systems.
Windows implementation is under librte_eal/windows and
implemented using WIN32 API for Windows only.
Unix implementation is under librte_eal/unix and
implemented using pthread for UNIX compilation.
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
v3: switch from pthread shim to generic eal implementation [DmitryK]
v4: modify file names, function names, move unix code to common
for future external pthreads support [DmitryK]
v5: rename var used for external pthreads, add description in
meson_options.txt. [DmitryK]
v6: remove external_pthread support as it collide with pthread
shim implementation [DmitryK]
v7: move unix code to unix instead of common, rename functions
add more documentation [Thomas]
---
lib/librte_eal/include/rte_thread.h | 65 ++++++++++++++++++++++++++++
lib/librte_eal/rte_eal_exports.def | 5 +++
lib/librte_eal/unix/meson.build | 1 +
lib/librte_eal/unix/rte_thread.c | 86 +++++++++++++++++++++++++++++++++++++
lib/librte_eal/version.map | 6 +++
lib/librte_eal/windows/meson.build | 1 +
lib/librte_eal/windows/rte_thread.c | 83 +++++++++++++++++++++++++++++++++++
7 files changed, 247 insertions(+)
create mode 100644 lib/librte_eal/unix/rte_thread.c
create mode 100644 lib/librte_eal/windows/rte_thread.c
Comments
On Wed, 6 Jan 2021 22:35:53 +0200, Tal Shnaiderman wrote:
[...]
> +int
> +rte_thread_tls_key_create(rte_tls_key *key, void (*destructor)(void *))
> +{
> + int err;
> +
> + *key = malloc(sizeof(**key));
> + if ((*key) == NULL) {
> + RTE_LOG(DEBUG, EAL, "Cannot allocate TLS key.");
Missing "\n", same for Windows part.
Aside from this nit, for series,
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
On 06-Jan-21 8:35 PM, Tal Shnaiderman wrote:
> Add support for TLS functionality in EAL.
>
> The following functions are added:
> rte_thread_tls_key_create - create a TLS data key.
> rte_thread_tls_key_delete - delete a TLS data key.
> rte_thread_tls_value_set - set value bound to the TLS key
> rte_thread_tls_value_get - get value bound to the TLS key
>
> TLS key is defined by the new type rte_tls_key.
>
> The API allocates the thread local storage (TLS) key.
> Any thread of the process can subsequently use this key
> to store and retrieve values that are local to the thread.
>
> Those functions are added in addition to TLS capability
> in rte_per_lcore.h to allow abstraction of the pthread
> layer for all operating systems.
>
> Windows implementation is under librte_eal/windows and
> implemented using WIN32 API for Windows only.
>
> Unix implementation is under librte_eal/unix and
> implemented using pthread for UNIX compilation.
>
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> ---
I'm a bit late to the party as the patch has been merged already, but
perhaps as a further enhancement, you could add rte_errno setting for
errors? You seem to have it only for get() API but not for others, and
you pass pthread_setspecific()'s return value unmodified, even though it
might return an error. Presumably, these error codes would be different
on Unix and Windows, so return values would effectively be potentially
different on different OS's, so perhaps it's better to return -1 with
rte_errno to indicate which specific error was seen.
Hi Tal,
On 10/02/2021 13:33, Burakov, Anatoly wrote:
> On 06-Jan-21 8:35 PM, Tal Shnaiderman wrote:
>> Add support for TLS functionality in EAL.
>>
>> The following functions are added:
>> rte_thread_tls_key_create - create a TLS data key.
>> rte_thread_tls_key_delete - delete a TLS data key.
>> rte_thread_tls_value_set - set value bound to the TLS key
>> rte_thread_tls_value_get - get value bound to the TLS key
>>
>> TLS key is defined by the new type rte_tls_key.
I would suggest changing rte_tls_key to rte_thread_tls_key so that it is
consistent with the API and not confused with transport layer security.
>>
>> The API allocates the thread local storage (TLS) key.
>> Any thread of the process can subsequently use this key
>> to store and retrieve values that are local to the thread.
>>
>> Those functions are added in addition to TLS capability
>> in rte_per_lcore.h to allow abstraction of the pthread
>> layer for all operating systems.
>>
>> Windows implementation is under librte_eal/windows and
>> implemented using WIN32 API for Windows only.
>>
>> Unix implementation is under librte_eal/unix and
>> implemented using pthread for UNIX compilation.
>>
>> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
>> ---
>
> Subject: Re: [dpdk-dev] [PATCH v9 2/2] eal: add generic thread-local-storage
> functions
>
> External email: Use caution opening links or attachments
>
>
> On 06-Jan-21 8:35 PM, Tal Shnaiderman wrote:
> > Add support for TLS functionality in EAL.
> >
> > The following functions are added:
> > rte_thread_tls_key_create - create a TLS data key.
> > rte_thread_tls_key_delete - delete a TLS data key.
> > rte_thread_tls_value_set - set value bound to the TLS key
> > rte_thread_tls_value_get - get value bound to the TLS key
> >
> > TLS key is defined by the new type rte_tls_key.
> >
> > The API allocates the thread local storage (TLS) key.
> > Any thread of the process can subsequently use this key to store and
> > retrieve values that are local to the thread.
> >
> > Those functions are added in addition to TLS capability in
> > rte_per_lcore.h to allow abstraction of the pthread layer for all
> > operating systems.
> >
> > Windows implementation is under librte_eal/windows and implemented
> > using WIN32 API for Windows only.
> >
> > Unix implementation is under librte_eal/unix and implemented using
> > pthread for UNIX compilation.
> >
> > Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> > ---
>
> I'm a bit late to the party as the patch has been merged already, but perhaps
> as a further enhancement, you could add rte_errno setting for errors? You
> seem to have it only for get() API but not for others, and you pass
> pthread_setspecific()'s return value unmodified, even though it might return
> an error. Presumably, these error codes would be different on Unix and
> Windows, so return values would effectively be potentially different on
> different OS's, so perhaps it's better to return -1 with rte_errno to indicate
> which specific error was seen.
>
Thanks Anatoly, I'll add the error codes to rte_errno in 21.05.
> --
> Thanks,
> Anatoly
> Subject: Re: [dpdk-dev] [PATCH v9 2/2] eal: add generic thread-local-storage
> functions
>
> External email: Use caution opening links or attachments
>
>
> Hi Tal,
>
> On 10/02/2021 13:33, Burakov, Anatoly wrote:
> > On 06-Jan-21 8:35 PM, Tal Shnaiderman wrote:
> >> Add support for TLS functionality in EAL.
> >>
> >> The following functions are added:
> >> rte_thread_tls_key_create - create a TLS data key.
> >> rte_thread_tls_key_delete - delete a TLS data key.
> >> rte_thread_tls_value_set - set value bound to the TLS key
> >> rte_thread_tls_value_get - get value bound to the TLS key
> >>
> >> TLS key is defined by the new type rte_tls_key.
>
> I would suggest changing rte_tls_key to rte_thread_tls_key so that it is
> consistent with the API and not confused with transport layer security.
>
Thanks Vladimir, will modify for 21.05.
> >>
> >> The API allocates the thread local storage (TLS) key.
> >> Any thread of the process can subsequently use this key to store and
> >> retrieve values that are local to the thread.
> >>
> >> Those functions are added in addition to TLS capability in
> >> rte_per_lcore.h to allow abstraction of the pthread layer for all
> >> operating systems.
> >>
> >> Windows implementation is under librte_eal/windows and implemented
> >> using WIN32 API for Windows only.
> >>
> >> Unix implementation is under librte_eal/unix and implemented using
> >> pthread for UNIX compilation.
> >>
> >> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> >> ---
> >
>
> --
> Regards,
> Vladimir
@@ -19,6 +19,11 @@
extern "C" {
#endif
+/**
+ * TLS key type, an opaque pointer.
+ */
+typedef struct eal_tls_key *rte_tls_key;
+
/**
* Set core affinity of the current thread.
* Support both EAL and non-EAL thread and update TLS.
@@ -40,6 +45,66 @@ int rte_thread_set_affinity(rte_cpuset_t *cpusetp);
*/
void rte_thread_get_affinity(rte_cpuset_t *cpusetp);
+/**
+ * Create a TLS data key visible to all threads in the process.
+ * the created key is later used to get/set a value.
+ * and optional destructor can be set to be called when a thread exits.
+ *
+ * @param key
+ * Pointer to store the allocated rte_tls_key
+ * @param destructor
+ * The function to be called when the thread exits.
+ * Ignored on Windows OS.
+ *
+ * @return
+ * On success, zero.
+ * On failure, a negative number.
+ */
+
+__rte_experimental
+int rte_thread_tls_key_create(rte_tls_key *key, void (*destructor)(void *));
+
+/**
+ * Delete a TLS data key visible to all threads in the process.
+ *
+ * @param key
+ * The rte_tls_key allocated by rte_thread_tls_key_create().
+ *
+ * @return
+ * On success, zero.
+ * On failure, a negative number.
+ */
+__rte_experimental
+int rte_thread_tls_key_delete(rte_tls_key key);
+
+/**
+ * Set value bound to the TLS key on behalf of the calling thread.
+ *
+ * @param key
+ * The rte_tls_key key allocated by rte_thread_tls_key_create.
+ * @param value
+ * The value bound to the rte_tls_key key for the calling thread.
+ *
+ * @return
+ * On success, zero.
+ * On failure, a negative number.
+ */
+__rte_experimental
+int rte_thread_tls_value_set(rte_tls_key key, const void *value);
+
+/**
+ * Get value bound to the TLS key on behalf of the calling thread.
+ *
+ * @param key
+ * The rte_tls_key key allocated by rte_thread_tls_key_create.
+ *
+ * @return
+ * On success, value data pointer (can also be NULL).
+ * On failure, NULL and an error number is set in rte_errno.
+ */
+__rte_experimental
+void *rte_thread_tls_value_get(rte_tls_key key);
+
#ifdef __cplusplus
}
#endif
@@ -306,6 +306,11 @@ EXPORTS
rte_vect_get_max_simd_bitwidth
rte_vect_set_max_simd_bitwidth
+ rte_thread_tls_key_create
+ rte_thread_tls_key_delete
+ rte_thread_tls_value_set
+ rte_thread_tls_value_get
+
rte_mem_lock
rte_mem_map
rte_mem_page_size
@@ -5,4 +5,5 @@ sources += files(
'eal_file.c',
'eal_unix_memory.c',
'eal_unix_timer.c',
+ 'rte_thread.c',
)
new file mode 100644
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2021 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <pthread.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_common.h>
+#include <rte_errno.h>
+#include <rte_log.h>
+#include <rte_thread.h>
+
+struct eal_tls_key {
+ pthread_key_t thread_index;
+};
+
+int
+rte_thread_tls_key_create(rte_tls_key *key, void (*destructor)(void *))
+{
+ int err;
+
+ *key = malloc(sizeof(**key));
+ if ((*key) == NULL) {
+ RTE_LOG(DEBUG, EAL, "Cannot allocate TLS key.");
+ return -1;
+ }
+ err = pthread_key_create(&((*key)->thread_index), destructor);
+ if (err) {
+ RTE_LOG(DEBUG, EAL, "pthread_key_create failed: %s\n",
+ strerror(err));
+ free(*key);
+ return -1;
+ }
+ return 0;
+}
+
+int
+rte_thread_tls_key_delete(rte_tls_key key)
+{
+ int err;
+
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "Invalid TLS key.\n");
+ return -1;
+ }
+ err = pthread_key_delete(key->thread_index);
+ if (err) {
+ RTE_LOG(DEBUG, EAL, "pthread_key_delete failed: %s\n",
+ strerror(err));
+ free(key);
+ return -1;
+ }
+ free(key);
+ return 0;
+}
+
+int
+rte_thread_tls_value_set(rte_tls_key key, const void *value)
+{
+ int err;
+
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "Invalid TLS key.\n");
+ return -1;
+ }
+ err = pthread_setspecific(key->thread_index, value);
+ if (err) {
+ RTE_LOG(DEBUG, EAL, "pthread_setspecific failed: %s\n",
+ strerror(err));
+ return -1;
+ }
+ return 0;
+}
+
+void *
+rte_thread_tls_value_get(rte_tls_key key)
+{
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "Invalid TLS key.\n");
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ return pthread_getspecific(key->thread_index);
+}
@@ -403,6 +403,12 @@ EXPERIMENTAL {
rte_service_lcore_may_be_active;
rte_vect_get_max_simd_bitwidth;
rte_vect_set_max_simd_bitwidth;
+
+ # added in 21.02
+ rte_thread_tls_key_create;
+ rte_thread_tls_key_delete;
+ rte_thread_tls_value_set;
+ rte_thread_tls_value_get;
};
INTERNAL {
@@ -19,6 +19,7 @@ sources += files(
'eal_timer.c',
'fnmatch.c',
'getopt.c',
+ 'rte_thread.c',
)
dpdk_conf.set10('RTE_EAL_NUMA_AWARE_HUGEPAGES', true)
new file mode 100644
@@ -0,0 +1,83 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2021 Mellanox Technologies, Ltd
+ */
+
+#include <rte_common.h>
+#include <rte_errno.h>
+#include <rte_thread.h>
+#include <rte_windows.h>
+
+struct eal_tls_key {
+ DWORD thread_index;
+};
+
+int
+rte_thread_tls_key_create(rte_tls_key *key,
+ __rte_unused void (*destructor)(void *))
+{
+ *key = malloc(sizeof(**key));
+ if ((*key) == NULL) {
+ RTE_LOG(DEBUG, EAL, "Cannot allocate TLS key.");
+ return -1;
+ }
+ (*key)->thread_index = TlsAlloc();
+ if ((*key)->thread_index == TLS_OUT_OF_INDEXES) {
+ RTE_LOG_WIN32_ERR("TlsAlloc()");
+ free(*key);
+ return -1;
+ }
+ return 0;
+}
+
+int
+rte_thread_tls_key_delete(rte_tls_key key)
+{
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "Invalid TLS key.\n");
+ return -1;
+ }
+ if (!TlsFree(key->thread_index)) {
+ RTE_LOG_WIN32_ERR("TlsFree()");
+ free(key);
+ return -1;
+ }
+ free(key);
+ return 0;
+}
+
+int
+rte_thread_tls_value_set(rte_tls_key key, const void *value)
+{
+ char *p;
+
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "Invalid TLS key.\n");
+ return -1;
+ }
+ /* discard const qualifier */
+ p = (char *) (uintptr_t) value;
+ if (!TlsSetValue(key->thread_index, p)) {
+ RTE_LOG_WIN32_ERR("TlsSetValue()");
+ return -1;
+ }
+ return 0;
+}
+
+void *
+rte_thread_tls_value_get(rte_tls_key key)
+{
+ void *output;
+
+ if (!key) {
+ RTE_LOG(DEBUG, EAL, "Invalid TLS key.\n");
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ output = TlsGetValue(key->thread_index);
+ if (GetLastError() != ERROR_SUCCESS) {
+ RTE_LOG_WIN32_ERR("TlsGetValue()");
+ rte_errno = ENOEXEC;
+ return NULL;
+ }
+ return output;
+}