[v5,1/3] eal: add rte control thread create API
Checks
Commit Message
Add rte_control_thread_create API as a replacement for
rte_ctrl_thread_create to allow deprecation of the use of platform
specific types in DPDK public API.
Duplicate the rte_ctrl_thread_create test adapted to use
rte_control_thread create to keep both APIs under test until
rte_ctrl_thread_create is removed.
Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Reviewed-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
---
app/test/test_lcores.c | 41 ++++++++++++++++++
lib/eal/common/eal_common_thread.c | 85 ++++++++++++++++++++++++++++++++++----
lib/eal/include/rte_thread.h | 33 +++++++++++++++
lib/eal/version.map | 1 +
4 files changed, 152 insertions(+), 8 deletions(-)
Comments
On Tue, Jan 31, 2023 at 9:30 PM Tyler Retzlaff
<roretzla@linux.microsoft.com> wrote:
>
> Add rte_control_thread_create API as a replacement for
> rte_ctrl_thread_create to allow deprecation of the use of platform
> specific types in DPDK public API.
>
> Duplicate the rte_ctrl_thread_create test adapted to use
> rte_control_thread create to keep both APIs under test until
> rte_ctrl_thread_create is removed.
>
> Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> Reviewed-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> ---
> app/test/test_lcores.c | 41 ++++++++++++++++++
> lib/eal/common/eal_common_thread.c | 85 ++++++++++++++++++++++++++++++++++----
> lib/eal/include/rte_thread.h | 33 +++++++++++++++
> lib/eal/version.map | 1 +
> 4 files changed, 152 insertions(+), 8 deletions(-)
>
> diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c
> index 5b43aa5..9766f78 100644
> --- a/app/test/test_lcores.c
> +++ b/app/test/test_lcores.c
> @@ -353,6 +353,18 @@ static void *ctrl_thread_loop(void *arg)
> return NULL;
> }
>
> +static uint32_t control_thread_loop(void *arg)
> +{
> + struct thread_context *t = arg;
> +
> + printf("Control thread running successfully\n");
> +
> + /* Set the thread state to DONE */
> + t->state = Thread_DONE;
> +
> + return 0;
> +}
> +
> static int
> test_ctrl_thread(void)
> {
> @@ -380,6 +392,32 @@ static void *ctrl_thread_loop(void *arg)
> }
>
> static int
> +test_control_thread(void)
> +{
> + struct thread_context ctrl_thread_context;
> + struct thread_context *t;
> +
> + /* Create one control thread */
> + t = &ctrl_thread_context;
> + t->state = Thread_INIT;
> + if (rte_control_thread_create(&t->id, "test_control_threads",
> + NULL, control_thread_loop, t) != 0)
> + return -1;
> +
> + /* Wait till the control thread exits.
> + * This also acts as the barrier such that the memory operations
> + * in control thread are visible to this thread.
> + */
> + rte_thread_join(t->id, NULL);
> +
> + /* Check if the control thread set the correct state */
> + if (t->state != Thread_DONE)
> + return -1;
> +
> + return 0;
> +}
> +
> +static int
> test_lcores(void)
> {
> unsigned int eal_threads_count = 0;
> @@ -409,6 +447,9 @@ static void *ctrl_thread_loop(void *arg)
> if (test_ctrl_thread() < 0)
> return TEST_FAILED;
>
> + if (test_control_thread() < 0)
> + return TEST_FAILED;
> +
> return TEST_SUCCESS;
> }
>
Afair, the "legacy" API test being in test_lcores.c is mainly a side
effect of the API being defined in rte_lcore.h.
The new API is genuinely located in rte_thread.h and there is no
consideration over lcores: control thread are just "specialised"
rte_thread objects.
I'd rather see this test in app/test/test_threads.c with other
rte_thread API tests.
I think something like below would be enough, wdyt?
If you are fine with it and there is no other comment on this patch, I
plan to do this change before applying.
diff --git a/app/test/test_threads.c b/app/test/test_threads.c
index e0f18e4329..cc0bb69190 100644
--- a/app/test/test_threads.c
+++ b/app/test/test_threads.c
@@ -232,6 +232,31 @@ test_thread_attributes_priority(void)
return 0;
}
+static int
+test_control_thread_create_join(void)
+{
+ rte_thread_t thread_id;
+ rte_thread_t thread_main_id;
+
+ thread_id_ready = 0;
+ RTE_TEST_ASSERT(rte_control_thread_create(&thread_id,
"test_control_threads", NULL,
+ thread_main, &thread_main_id) == 0,
+ "Failed to create thread.");
+
+ while (__atomic_load_n(&thread_id_ready, __ATOMIC_ACQUIRE) == 0)
+ ;
+
+ RTE_TEST_ASSERT(rte_thread_equal(thread_id, thread_main_id) != 0,
+ "Unexpected thread id.");
+
+ __atomic_store_n(&thread_id_ready, 2, __ATOMIC_RELEASE);
+
+ RTE_TEST_ASSERT(rte_thread_join(thread_id, NULL) == 0,
+ "Failed to join thread.");
+
+ return 0;
+}
+
static struct unit_test_suite threads_test_suite = {
.suite_name = "threads autotest",
.setup = NULL,
@@ -243,6 +268,7 @@ static struct unit_test_suite threads_test_suite = {
TEST_CASE(test_thread_priority),
TEST_CASE(test_thread_attributes_affinity),
TEST_CASE(test_thread_attributes_priority),
+ TEST_CASE(test_control_thread_create_join),
TEST_CASES_END()
}
};
On Tue, Feb 07, 2023 at 02:12:18PM +0100, David Marchand wrote:
> On Tue, Jan 31, 2023 at 9:30 PM Tyler Retzlaff
> <roretzla@linux.microsoft.com> wrote:
> >
> > Add rte_control_thread_create API as a replacement for
> > rte_ctrl_thread_create to allow deprecation of the use of platform
> > specific types in DPDK public API.
> >
> > Duplicate the rte_ctrl_thread_create test adapted to use
> > rte_control_thread create to keep both APIs under test until
> > rte_ctrl_thread_create is removed.
> >
> > Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
> > Acked-by: Morten Brørup <mb@smartsharesystems.com>
> > Reviewed-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> > ---
> > app/test/test_lcores.c | 41 ++++++++++++++++++
> > lib/eal/common/eal_common_thread.c | 85 ++++++++++++++++++++++++++++++++++----
> > lib/eal/include/rte_thread.h | 33 +++++++++++++++
> > lib/eal/version.map | 1 +
> > 4 files changed, 152 insertions(+), 8 deletions(-)
> >
> > diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c
> > index 5b43aa5..9766f78 100644
> > --- a/app/test/test_lcores.c
> > +++ b/app/test/test_lcores.c
> > @@ -353,6 +353,18 @@ static void *ctrl_thread_loop(void *arg)
> > return NULL;
> > }
> >
> > +static uint32_t control_thread_loop(void *arg)
> > +{
> > + struct thread_context *t = arg;
> > +
> > + printf("Control thread running successfully\n");
> > +
> > + /* Set the thread state to DONE */
> > + t->state = Thread_DONE;
> > +
> > + return 0;
> > +}
> > +
> > static int
> > test_ctrl_thread(void)
> > {
> > @@ -380,6 +392,32 @@ static void *ctrl_thread_loop(void *arg)
> > }
> >
> > static int
> > +test_control_thread(void)
> > +{
> > + struct thread_context ctrl_thread_context;
> > + struct thread_context *t;
> > +
> > + /* Create one control thread */
> > + t = &ctrl_thread_context;
> > + t->state = Thread_INIT;
> > + if (rte_control_thread_create(&t->id, "test_control_threads",
> > + NULL, control_thread_loop, t) != 0)
> > + return -1;
> > +
> > + /* Wait till the control thread exits.
> > + * This also acts as the barrier such that the memory operations
> > + * in control thread are visible to this thread.
> > + */
> > + rte_thread_join(t->id, NULL);
> > +
> > + /* Check if the control thread set the correct state */
> > + if (t->state != Thread_DONE)
> > + return -1;
> > +
> > + return 0;
> > +}
> > +
> > +static int
> > test_lcores(void)
> > {
> > unsigned int eal_threads_count = 0;
> > @@ -409,6 +447,9 @@ static void *ctrl_thread_loop(void *arg)
> > if (test_ctrl_thread() < 0)
> > return TEST_FAILED;
> >
> > + if (test_control_thread() < 0)
> > + return TEST_FAILED;
> > +
> > return TEST_SUCCESS;
> > }
> >
>
> Afair, the "legacy" API test being in test_lcores.c is mainly a side
> effect of the API being defined in rte_lcore.h.
>
> The new API is genuinely located in rte_thread.h and there is no
> consideration over lcores: control thread are just "specialised"
> rte_thread objects.
> I'd rather see this test in app/test/test_threads.c with other
> rte_thread API tests.
no problem, i wondered if i should retain the original location or not.
i'll move it to test_threads.c
i kind of wonder if the function should be named rte_thread_create_ctrl
or something now so that all the functions from rte_thread have a
consistent naming prefix?
let me know if this is desired. if it is i'll submit a new version
otherwise you can apply the series as is with your suggested test moved
to test_threads.c below.
>
> I think something like below would be enough, wdyt?
> If you are fine with it and there is no other comment on this patch, I
> plan to do this change before applying.
no problem with this, sounds good.
>
>
> diff --git a/app/test/test_threads.c b/app/test/test_threads.c
> index e0f18e4329..cc0bb69190 100644
> --- a/app/test/test_threads.c
> +++ b/app/test/test_threads.c
> @@ -232,6 +232,31 @@ test_thread_attributes_priority(void)
> return 0;
> }
>
> +static int
> +test_control_thread_create_join(void)
> +{
> + rte_thread_t thread_id;
> + rte_thread_t thread_main_id;
> +
> + thread_id_ready = 0;
> + RTE_TEST_ASSERT(rte_control_thread_create(&thread_id,
> "test_control_threads", NULL,
> + thread_main, &thread_main_id) == 0,
> + "Failed to create thread.");
> +
> + while (__atomic_load_n(&thread_id_ready, __ATOMIC_ACQUIRE) == 0)
> + ;
> +
> + RTE_TEST_ASSERT(rte_thread_equal(thread_id, thread_main_id) != 0,
> + "Unexpected thread id.");
> +
> + __atomic_store_n(&thread_id_ready, 2, __ATOMIC_RELEASE);
> +
> + RTE_TEST_ASSERT(rte_thread_join(thread_id, NULL) == 0,
> + "Failed to join thread.");
> +
> + return 0;
> +}
> +
> static struct unit_test_suite threads_test_suite = {
> .suite_name = "threads autotest",
> .setup = NULL,
> @@ -243,6 +268,7 @@ static struct unit_test_suite threads_test_suite = {
> TEST_CASE(test_thread_priority),
> TEST_CASE(test_thread_attributes_affinity),
> TEST_CASE(test_thread_attributes_priority),
> + TEST_CASE(test_control_thread_create_join),
> TEST_CASES_END()
> }
> };
>
>
> --
> David Marchand
@@ -353,6 +353,18 @@ static void *ctrl_thread_loop(void *arg)
return NULL;
}
+static uint32_t control_thread_loop(void *arg)
+{
+ struct thread_context *t = arg;
+
+ printf("Control thread running successfully\n");
+
+ /* Set the thread state to DONE */
+ t->state = Thread_DONE;
+
+ return 0;
+}
+
static int
test_ctrl_thread(void)
{
@@ -380,6 +392,32 @@ static void *ctrl_thread_loop(void *arg)
}
static int
+test_control_thread(void)
+{
+ struct thread_context ctrl_thread_context;
+ struct thread_context *t;
+
+ /* Create one control thread */
+ t = &ctrl_thread_context;
+ t->state = Thread_INIT;
+ if (rte_control_thread_create(&t->id, "test_control_threads",
+ NULL, control_thread_loop, t) != 0)
+ return -1;
+
+ /* Wait till the control thread exits.
+ * This also acts as the barrier such that the memory operations
+ * in control thread are visible to this thread.
+ */
+ rte_thread_join(t->id, NULL);
+
+ /* Check if the control thread set the correct state */
+ if (t->state != Thread_DONE)
+ return -1;
+
+ return 0;
+}
+
+static int
test_lcores(void)
{
unsigned int eal_threads_count = 0;
@@ -409,6 +447,9 @@ static void *ctrl_thread_loop(void *arg)
if (test_ctrl_thread() < 0)
return TEST_FAILED;
+ if (test_control_thread() < 0)
+ return TEST_FAILED;
+
return TEST_SUCCESS;
}
@@ -232,7 +232,10 @@ enum __rte_ctrl_thread_status {
};
struct rte_thread_ctrl_params {
- void *(*start_routine)(void *);
+ union {
+ void *(*ctrl_start_routine)(void *arg);
+ rte_thread_func control_start_routine;
+ } u;
void *arg;
int ret;
/* Control thread status.
@@ -241,27 +244,47 @@ struct rte_thread_ctrl_params {
enum __rte_ctrl_thread_status ctrl_thread_status;
};
-static void *ctrl_thread_init(void *arg)
+static int ctrl_thread_init(void *arg)
{
struct internal_config *internal_conf =
eal_get_internal_configuration();
rte_cpuset_t *cpuset = &internal_conf->ctrl_cpuset;
struct rte_thread_ctrl_params *params = arg;
- void *(*start_routine)(void *) = params->start_routine;
- void *routine_arg = params->arg;
__rte_thread_init(rte_lcore_id(), cpuset);
params->ret = rte_thread_set_affinity_by_id(rte_thread_self(), cpuset);
if (params->ret != 0) {
__atomic_store_n(¶ms->ctrl_thread_status,
CTRL_THREAD_ERROR, __ATOMIC_RELEASE);
- return NULL;
+ return params->ret;
}
__atomic_store_n(¶ms->ctrl_thread_status,
CTRL_THREAD_RUNNING, __ATOMIC_RELEASE);
- return start_routine(routine_arg);
+ return 0;
+}
+
+static void *ctrl_thread_start(void *arg)
+{
+ struct rte_thread_ctrl_params *params = arg;
+ void *(*start_routine)(void *) = params->u.ctrl_start_routine;
+
+ if (ctrl_thread_init(arg) != 0)
+ return NULL;
+
+ return start_routine(params->arg);
+}
+
+static uint32_t control_thread_start(void *arg)
+{
+ struct rte_thread_ctrl_params *params = arg;
+ rte_thread_func start_routine = params->u.control_start_routine;
+
+ if (ctrl_thread_init(arg) != 0)
+ return params->ret;
+
+ return start_routine(params->arg);
}
int
@@ -277,12 +300,12 @@ static void *ctrl_thread_init(void *arg)
if (!params)
return -ENOMEM;
- params->start_routine = start_routine;
+ params->u.ctrl_start_routine = start_routine;
params->arg = arg;
params->ret = 0;
params->ctrl_thread_status = CTRL_THREAD_LAUNCHING;
- ret = pthread_create(thread, attr, ctrl_thread_init, (void *)params);
+ ret = pthread_create(thread, attr, ctrl_thread_start, (void *)params);
if (ret != 0) {
free(params);
return -ret;
@@ -315,6 +338,52 @@ static void *ctrl_thread_init(void *arg)
}
int
+rte_control_thread_create(rte_thread_t *thread, const char *name,
+ const rte_thread_attr_t *attr,
+ rte_thread_func start_routine, void *arg)
+{
+ struct rte_thread_ctrl_params *params;
+ enum __rte_ctrl_thread_status ctrl_thread_status;
+ int ret;
+
+ params = malloc(sizeof(*params));
+ if (params == NULL)
+ return -ENOMEM;
+
+ params->u.control_start_routine = start_routine;
+ params->arg = arg;
+ params->ret = 0;
+ params->ctrl_thread_status = CTRL_THREAD_LAUNCHING;
+
+ ret = rte_thread_create(thread, attr, control_thread_start, params);
+ if (ret != 0) {
+ free(params);
+ return -ret;
+ }
+
+ if (name != NULL)
+ rte_thread_set_name(*thread, name);
+
+ /* Wait for the control thread to initialize successfully */
+ while ((ctrl_thread_status =
+ __atomic_load_n(¶ms->ctrl_thread_status,
+ __ATOMIC_ACQUIRE)) == CTRL_THREAD_LAUNCHING) {
+ rte_delay_us_sleep(1);
+ }
+
+ /* Check if the control thread encountered an error */
+ if (ctrl_thread_status == CTRL_THREAD_ERROR) {
+ /* ctrl thread is exiting */
+ rte_thread_join(*thread, NULL);
+ }
+
+ ret = params->ret;
+ free(params);
+
+ return ret;
+}
+
+int
rte_thread_register(void)
{
unsigned int lcore_id;
@@ -98,6 +98,39 @@ int rte_thread_create(rte_thread_t *thread_id,
* @warning
* @b EXPERIMENTAL: this API may change without prior notice.
*
+ * Create a control thread.
+ *
+ * Creates a control thread with the given name and attributes. The
+ * affinity of the new thread is based on the CPU affinity retrieved
+ * at the time rte_eal_init() was called, the EAL threads are then
+ * excluded. If setting the name of the thread fails, the error is
+ * ignored and a debug message is logged.
+ *
+ * @param thread
+ * Filled with the thread id of the new created thread.
+ * @param name
+ * The name of the control thread
+ * (max RTE_MAX_THREAD_NAME_LEN characters including '\0').
+ * @param thread_attr
+ * Attributes for the new thread.
+ * @param thread_func
+ * Function to be executed by the new thread.
+ * @param arg
+ * Argument passed to start_routine.
+ * @return
+ * On success, returns 0; on error, it returns a negative value
+ * corresponding to the error number.
+ */
+__rte_experimental
+int
+rte_control_thread_create(rte_thread_t *thread, const char *name,
+ const rte_thread_attr_t *thread_attr,
+ rte_thread_func thread_func, void *arg);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
* Waits for the thread identified by 'thread_id' to terminate
*
* @param thread_id
@@ -441,6 +441,7 @@ EXPERIMENTAL {
rte_thread_join;
# added in 23.03
+ rte_control_thread_create;
rte_thread_set_name;
};