[v3,1/3] lib: introduce dispatcher library
Checks
Commit Message
The purpose of the dispatcher library is to help reduce coupling in an
Eventdev-based DPDK application.
In addition, the dispatcher also provides a convenient and flexible
way for the application to use service cores for application-level
processing.
Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
Reviewed-by: Heng Wang <heng.wang@ericsson.com>
--
PATCH v3:
o To underline its optional character and since it does not provide
hardware abstraction, the event dispatcher is now a separate
library.
o Change name from rte_event_dispatcher -> rte_dispatcher, to make it
shorter and to avoid the rte_event_* namespace.
PATCH v2:
o Add dequeue batch count statistic.
o Add statistics reset function to API.
o Clarify MT safety guarantees (or lack thereof) in the API documentation.
o Change loop variable type in evd_lcore_get_handler_by_id() to uint16_t,
to be consistent with similar loops elsewhere in the dispatcher.
o Fix variable names in finalizer unregister function.
PATCH:
o Change prefix from RED to EVD, to avoid confusion with random
early detection.
RFC v4:
o Move handlers to per-lcore data structures.
o Introduce mechanism which rearranges handlers so that often-used
handlers tend to be tried first.
o Terminate dispatch loop in case all events are delivered.
o To avoid the dispatcher's service function hogging the CPU, process
only one batch per call.
o Have service function return -EAGAIN if no work is performed.
o Events delivered in the process function is no longer marked 'const',
since modifying them may be useful for the application and cause
no difficulties for the dispatcher.
o Various minor API documentation improvements.
RFC v3:
o Add stats_get() function to the version.map file.
---
MAINTAINERS | 3 +
lib/dispatcher/meson.build | 17 +
lib/dispatcher/rte_dispatcher.c | 791 ++++++++++++++++++++++++++++++++
lib/dispatcher/rte_dispatcher.h | 480 +++++++++++++++++++
lib/dispatcher/version.map | 20 +
lib/meson.build | 2 +
6 files changed, 1313 insertions(+)
create mode 100644 lib/dispatcher/meson.build
create mode 100644 lib/dispatcher/rte_dispatcher.c
create mode 100644 lib/dispatcher/rte_dispatcher.h
create mode 100644 lib/dispatcher/version.map
Comments
> -----Original Message-----
> From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> Sent: Monday, September 4, 2023 6:33 PM
> To: dev@dpdk.org
> Cc: Jerin Jacob <jerinj@marvell.com>; techboard@dpdk.org; Van Haaren,
> Harry <harry.van.haaren@intel.com>; hofors@lysator.liu.se; Nilsson, Peter
> <peter.j.nilsson@ericsson.com>; Heng Wang <heng.wang@ericsson.com>;
> Naga Harish K, S V <s.v.naga.harish.k@intel.com>; Pavan Nikhilesh
> <pbhagavatula@marvell.com>; Gujjar, Abhinandan S
> <abhinandan.gujjar@intel.com>; Carrillo, Erik G <Erik.G.Carrillo@intel.com>;
> Shijith Thotton <sthotton@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
> Liang Ma <liangma@liangbit.com>; Mccarthy, Peter
> <Peter.Mccarthy@intel.com>; Yan, Zhirun <Zhirun.Yan@intel.com>;
> mattias.ronnblom <mattias.ronnblom@ericsson.com>
> Subject: [PATCH v3 1/3] lib: introduce dispatcher library
>
> The purpose of the dispatcher library is to help reduce coupling in an
> Eventdev-based DPDK application.
>
> In addition, the dispatcher also provides a convenient and flexible way for the
> application to use service cores for application-level processing.
>
> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
> Reviewed-by: Heng Wang <heng.wang@ericsson.com>
>
> --
>
> PATCH v3:
> o To underline its optional character and since it does not provide
> hardware abstraction, the event dispatcher is now a separate
> library.
> o Change name from rte_event_dispatcher -> rte_dispatcher, to make it
> shorter and to avoid the rte_event_* namespace.
>
Rte_dispatcher is basically dispatching events but it feels like the name does not convey that.
Also, it is like any other adapter service that can reside within the eventdev directory.
I can see some discussion in previous threads related to the placement of the dispatcher library.
It is an optional eventdev application service, not enforcing this programming model to the application.
The documentation may need to be updated and mention that this is optional.
If any hardware comes up with the dispatcher feature, then this library may need to be moved inside eventdev library later.
So, It makes sense to keep this optional service in the eventdev folder as an optional feature.
> PATCH v2:
> o Add dequeue batch count statistic.
> o Add statistics reset function to API.
> o Clarify MT safety guarantees (or lack thereof) in the API documentation.
> o Change loop variable type in evd_lcore_get_handler_by_id() to uint16_t,
> to be consistent with similar loops elsewhere in the dispatcher.
> o Fix variable names in finalizer unregister function.
>
> PATCH:
> o Change prefix from RED to EVD, to avoid confusion with random
> early detection.
>
> RFC v4:
> o Move handlers to per-lcore data structures.
> o Introduce mechanism which rearranges handlers so that often-used
> handlers tend to be tried first.
> o Terminate dispatch loop in case all events are delivered.
> o To avoid the dispatcher's service function hogging the CPU, process
> only one batch per call.
> o Have service function return -EAGAIN if no work is performed.
> o Events delivered in the process function is no longer marked 'const',
> since modifying them may be useful for the application and cause
> no difficulties for the dispatcher.
> o Various minor API documentation improvements.
>
> RFC v3:
> o Add stats_get() function to the version.map file.
> ---
> MAINTAINERS | 3 +
> lib/dispatcher/meson.build | 17 +
> lib/dispatcher/rte_dispatcher.c | 791
> ++++++++++++++++++++++++++++++++ lib/dispatcher/rte_dispatcher.h |
> 480 +++++++++++++++++++
> lib/dispatcher/version.map | 20 +
> lib/meson.build | 2 +
> 6 files changed, 1313 insertions(+)
> create mode 100644 lib/dispatcher/meson.build create mode 100644
> lib/dispatcher/rte_dispatcher.c create mode 100644
> lib/dispatcher/rte_dispatcher.h create mode 100644
> lib/dispatcher/version.map
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a926155f26..6704cd5b2c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1726,6 +1726,9 @@ M: Nithin Dabilpuram
> <ndabilpuram@marvell.com>
> M: Pavan Nikhilesh <pbhagavatula@marvell.com>
> F: lib/node/
>
> +Dispatcher - EXPERIMENTAL
> +M: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> +F: lib/dispatcher/
>
> Test Applications
> -----------------
> diff --git a/lib/dispatcher/meson.build b/lib/dispatcher/meson.build new file
> mode 100644 index 0000000000..c6054a3a5d
> --- /dev/null
> +++ b/lib/dispatcher/meson.build
> @@ -0,0 +1,17 @@
> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 Ericsson AB
> +
> +if is_windows
> + build = false
> + reason = 'not supported on Windows'
> + subdir_done()
> +endif
> +
> +sources = files(
> + 'rte_dispatcher.c',
> +)
> +headers = files(
> + 'rte_dispatcher.h',
> +)
> +
> +deps += ['eventdev']
> diff --git a/lib/dispatcher/rte_dispatcher.c b/lib/dispatcher/rte_dispatcher.c
> new file mode 100644 index 0000000000..3319fe09f2
> --- /dev/null
> +++ b/lib/dispatcher/rte_dispatcher.c
> @@ -0,0 +1,791 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2023 Ericsson AB
> + */
> +
> +#include <stdbool.h>
> +#include <stdint.h>
> +
> +#include <rte_branch_prediction.h>
> +#include <rte_common.h>
> +#include <rte_lcore.h>
> +#include <rte_random.h>
> +#include <rte_service_component.h>
> +
> +#include "eventdev_pmd.h"
> +
> +#include <rte_dispatcher.h>
> +
> +#define EVD_MAX_PORTS_PER_LCORE 4
> +#define EVD_MAX_HANDLERS 32
> +#define EVD_MAX_FINALIZERS 16
> +#define EVD_AVG_PRIO_INTERVAL 2000
> +
> +struct rte_dispatcher_lcore_port {
> + uint8_t port_id;
> + uint16_t batch_size;
> + uint64_t timeout;
> +};
> +
> +struct rte_dispatcher_handler {
> + int id;
> + rte_dispatcher_match_t match_fun;
> + void *match_data;
> + rte_dispatcher_process_t process_fun;
> + void *process_data;
> +};
> +
> +struct rte_dispatcher_finalizer {
> + int id;
> + rte_dispatcher_finalize_t finalize_fun;
> + void *finalize_data;
> +};
> +
> +struct rte_dispatcher_lcore {
> + uint8_t num_ports;
> + uint16_t num_handlers;
> + int32_t prio_count;
> + struct rte_dispatcher_lcore_port
> ports[EVD_MAX_PORTS_PER_LCORE];
> + struct rte_dispatcher_handler handlers[EVD_MAX_HANDLERS];
> + struct rte_dispatcher_stats stats;
> +} __rte_cache_aligned;
> +
> +struct rte_dispatcher {
> + uint8_t id;
> + uint8_t event_dev_id;
> + int socket_id;
> + uint32_t service_id;
> + struct rte_dispatcher_lcore lcores[RTE_MAX_LCORE];
> + uint16_t num_finalizers;
> + struct rte_dispatcher_finalizer finalizers[EVD_MAX_FINALIZERS]; };
> +
> +static struct rte_dispatcher *dispatchers[UINT8_MAX];
> +
> +static bool
> +evd_has_dispatcher(uint8_t id)
> +{
> + return dispatchers[id] != NULL;
> +}
> +
> +static struct rte_dispatcher *
> +evd_get_dispatcher(uint8_t id)
> +{
> + return dispatchers[id];
> +}
> +
> +static void
> +evd_set_dispatcher(uint8_t id, struct rte_dispatcher *dispatcher) {
> + dispatchers[id] = dispatcher;
> +}
> +
> +#define EVD_VALID_ID_OR_RET_EINVAL(id)
> \
> + do { \
> + if (unlikely(!evd_has_dispatcher(id))) { \
> + RTE_EDEV_LOG_ERR("Invalid dispatcher id %d\n", id);
> \
> + return -EINVAL; \
> + } \
> + } while (0)
> +
> +static int
> +evd_lookup_handler_idx(struct rte_dispatcher_lcore *lcore,
> + const struct rte_event *event) {
> + uint16_t i;
> +
> + for (i = 0; i < lcore->num_handlers; i++) {
> + struct rte_dispatcher_handler *handler =
> + &lcore->handlers[i];
> +
> + if (handler->match_fun(event, handler->match_data))
> + return i;
> + }
> +
> + return -1;
> +}
> +
> +static void
> +evd_prioritize_handler(struct rte_dispatcher_lcore *lcore,
> + int handler_idx)
> +{
> + struct rte_dispatcher_handler tmp;
> +
> + if (handler_idx == 0)
> + return;
> +
> + /* Let the lucky handler "bubble" up the list */
> +
> + tmp = lcore->handlers[handler_idx - 1];
> +
> + lcore->handlers[handler_idx - 1] = lcore->handlers[handler_idx];
> +
> + lcore->handlers[handler_idx] = tmp;
> +}
> +
> +static inline void
> +evd_consider_prioritize_handler(struct rte_dispatcher_lcore *lcore,
> + int handler_idx, uint16_t handler_events) {
> + lcore->prio_count -= handler_events;
> +
> + if (unlikely(lcore->prio_count <= 0)) {
> + evd_prioritize_handler(lcore, handler_idx);
> +
> + /*
> + * Randomize the interval in the unlikely case
> + * the traffic follow some very strict pattern.
> + */
> + lcore->prio_count =
> + rte_rand_max(EVD_AVG_PRIO_INTERVAL) +
> + EVD_AVG_PRIO_INTERVAL / 2;
> + }
> +}
> +
> +static inline void
> +evd_dispatch_events(struct rte_dispatcher *dispatcher,
> + struct rte_dispatcher_lcore *lcore,
> + struct rte_dispatcher_lcore_port *port,
> + struct rte_event *events, uint16_t num_events) {
> + int i;
> + struct rte_event bursts[EVD_MAX_HANDLERS][num_events];
> + uint16_t burst_lens[EVD_MAX_HANDLERS] = { 0 };
> + uint16_t drop_count = 0;
> + uint16_t dispatch_count;
> + uint16_t dispatched = 0;
> +
> + for (i = 0; i < num_events; i++) {
> + struct rte_event *event = &events[i];
> + int handler_idx;
> +
> + handler_idx = evd_lookup_handler_idx(lcore, event);
> +
> + if (unlikely(handler_idx < 0)) {
> + drop_count++;
> + continue;
> + }
> +
> + bursts[handler_idx][burst_lens[handler_idx]] = *event;
> + burst_lens[handler_idx]++;
> + }
> +
> + dispatch_count = num_events - drop_count;
> +
> + for (i = 0; i < lcore->num_handlers &&
> + dispatched < dispatch_count; i++) {
> + struct rte_dispatcher_handler *handler =
> + &lcore->handlers[i];
> + uint16_t len = burst_lens[i];
> +
> + if (len == 0)
> + continue;
> +
> + handler->process_fun(dispatcher->event_dev_id, port-
> >port_id,
> + bursts[i], len, handler->process_data);
> +
> + dispatched += len;
> +
> + /*
> + * Safe, since any reshuffling will only involve
> + * already-processed handlers.
> + */
> + evd_consider_prioritize_handler(lcore, i, len);
> + }
> +
> + lcore->stats.ev_batch_count++;
> + lcore->stats.ev_dispatch_count += dispatch_count;
> + lcore->stats.ev_drop_count += drop_count;
> +
> + for (i = 0; i < dispatcher->num_finalizers; i++) {
> + struct rte_dispatcher_finalizer *finalizer =
> + &dispatcher->finalizers[i];
> +
> + finalizer->finalize_fun(dispatcher->event_dev_id,
> + port->port_id,
> + finalizer->finalize_data);
> + }
> +}
> +
> +static __rte_always_inline uint16_t
> +evd_port_dequeue(struct rte_dispatcher *dispatcher,
> + struct rte_dispatcher_lcore *lcore,
> + struct rte_dispatcher_lcore_port *port) {
> + uint16_t batch_size = port->batch_size;
> + struct rte_event events[batch_size];
> + uint16_t n;
> +
> + n = rte_event_dequeue_burst(dispatcher->event_dev_id, port-
> >port_id,
> + events, batch_size, port->timeout);
> +
> + if (likely(n > 0))
> + evd_dispatch_events(dispatcher, lcore, port, events, n);
> +
> + lcore->stats.poll_count++;
> +
> + return n;
> +}
> +
> +static __rte_always_inline uint16_t
> +evd_lcore_process(struct rte_dispatcher *dispatcher,
> + struct rte_dispatcher_lcore *lcore) {
> + uint16_t i;
> + uint16_t event_count = 0;
> +
> + for (i = 0; i < lcore->num_ports; i++) {
> + struct rte_dispatcher_lcore_port *port =
> + &lcore->ports[i];
> +
> + event_count += evd_port_dequeue(dispatcher, lcore, port);
> + }
> +
> + return event_count;
> +}
> +
> +static int32_t
> +evd_process(void *userdata)
> +{
> + struct rte_dispatcher *dispatcher = userdata;
> + unsigned int lcore_id = rte_lcore_id();
> + struct rte_dispatcher_lcore *lcore =
> + &dispatcher->lcores[lcore_id];
> + uint64_t event_count;
> +
> + event_count = evd_lcore_process(dispatcher, lcore);
> +
> + if (unlikely(event_count == 0))
> + return -EAGAIN;
> +
> + return 0;
> +}
> +
> +static int
> +evd_service_register(struct rte_dispatcher *dispatcher) {
> + struct rte_service_spec service = {
> + .callback = evd_process,
> + .callback_userdata = dispatcher,
> + .capabilities = RTE_SERVICE_CAP_MT_SAFE,
> + .socket_id = dispatcher->socket_id
> + };
> + int rc;
> +
> + snprintf(service.name, RTE_SERVICE_NAME_MAX - 1, "evd_%d",
> + dispatcher->id);
> +
> + rc = rte_service_component_register(&service,
> +&dispatcher->service_id);
> +
> + if (rc)
> + RTE_EDEV_LOG_ERR("Registration of dispatcher service "
> + "%s failed with error code %d\n",
> + service.name, rc);
> +
> + return rc;
> +}
> +
> +static int
> +evd_service_unregister(struct rte_dispatcher *dispatcher) {
> + int rc;
> +
> + rc = rte_service_component_unregister(dispatcher->service_id);
> +
> + if (rc)
> + RTE_EDEV_LOG_ERR("Unregistration of dispatcher service "
> + "failed with error code %d\n", rc);
> +
> + return rc;
> +}
> +
> +int
> +rte_dispatcher_create(uint8_t id, uint8_t event_dev_id) {
> + int socket_id;
> + struct rte_dispatcher *dispatcher;
> + int rc;
> +
> + if (evd_has_dispatcher(id)) {
> + RTE_EDEV_LOG_ERR("Dispatcher with id %d already exists\n",
> + id);
> + return -EEXIST;
> + }
> +
> + socket_id = rte_event_dev_socket_id(event_dev_id);
> +
> + dispatcher =
> + rte_malloc_socket("dispatcher", sizeof(struct rte_dispatcher),
> + RTE_CACHE_LINE_SIZE, socket_id);
> +
> + if (dispatcher == NULL) {
> + RTE_EDEV_LOG_ERR("Unable to allocate memory for
> dispatcher\n");
> + return -ENOMEM;
> + }
> +
> + *dispatcher = (struct rte_dispatcher) {
> + .id = id,
> + .event_dev_id = event_dev_id,
> + .socket_id = socket_id
> + };
> +
> + rc = evd_service_register(dispatcher);
> +
> + if (rc < 0) {
> + rte_free(dispatcher);
> + return rc;
> + }
> +
> + evd_set_dispatcher(id, dispatcher);
> +
> + return 0;
> +}
> +
> +int
> +rte_dispatcher_free(uint8_t id)
> +{
> + struct rte_dispatcher *dispatcher;
> + int rc;
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
> + rc = evd_service_unregister(dispatcher);
> +
> + if (rc)
> + return rc;
> +
> + evd_set_dispatcher(id, NULL);
> +
> + rte_free(dispatcher);
> +
> + return 0;
> +}
> +
> +int
> +rte_dispatcher_service_id_get(uint8_t id, uint32_t *service_id) {
> + struct rte_dispatcher *dispatcher;
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
Service_id pointer needs to be validated for NULL before accessing
> + *service_id = dispatcher->service_id;
> +
> + return 0;
> +}
> +
> +static int
> +lcore_port_index(struct rte_dispatcher_lcore *lcore,
> + uint8_t event_port_id)
> +{
> + uint16_t i;
> +
> + for (i = 0; i < lcore->num_ports; i++) {
> + struct rte_dispatcher_lcore_port *port =
> + &lcore->ports[i];
> +
> + if (port->port_id == event_port_id)
> + return i;
> + }
> +
> + return -1;
> +}
> +
> +int
> +rte_dispatcher_bind_port_to_lcore(uint8_t id, uint8_t event_port_id,
> + uint16_t batch_size, uint64_t
> timeout,
> + unsigned int lcore_id)
> +{
> + struct rte_dispatcher *dispatcher;
> + struct rte_dispatcher_lcore *lcore;
> + struct rte_dispatcher_lcore_port *port;
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
> + lcore = &dispatcher->lcores[lcore_id];
> +
> + if (lcore->num_ports == EVD_MAX_PORTS_PER_LCORE)
> + return -ENOMEM;
> +
> + if (lcore_port_index(lcore, event_port_id) >= 0)
> + return -EEXIST;
> +
> + port = &lcore->ports[lcore->num_ports];
> +
> + *port = (struct rte_dispatcher_lcore_port) {
> + .port_id = event_port_id,
> + .batch_size = batch_size,
> + .timeout = timeout
> + };
> +
> + lcore->num_ports++;
> +
> + return 0;
> +}
> +
> +int
> +rte_dispatcher_unbind_port_from_lcore(uint8_t id, uint8_t event_port_id,
> + unsigned int lcore_id)
> +{
> + struct rte_dispatcher *dispatcher;
> + struct rte_dispatcher_lcore *lcore;
> + int port_idx;
> + struct rte_dispatcher_lcore_port *port;
> + struct rte_dispatcher_lcore_port *last;
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
> + lcore = &dispatcher->lcores[lcore_id];
> +
> + port_idx = lcore_port_index(lcore, event_port_id);
> +
> + if (port_idx < 0)
> + return -ENOENT;
> +
> + port = &lcore->ports[port_idx];
> + last = &lcore->ports[lcore->num_ports - 1];
> +
> + if (port != last)
> + *port = *last;
> +
> + lcore->num_ports--;
> +
> + return 0;
> +}
> +
> +static struct rte_dispatcher_handler*
> +evd_lcore_get_handler_by_id(struct rte_dispatcher_lcore *lcore,
> + int handler_id)
> +{
> + uint16_t i;
> +
> + for (i = 0; i < lcore->num_handlers; i++) {
> + struct rte_dispatcher_handler *handler =
> + &lcore->handlers[i];
> +
> + if (handler->id == handler_id)
> + return handler;
> + }
> +
> + return NULL;
> +}
> +
> +static int
> +evd_alloc_handler_id(struct rte_dispatcher *dispatcher) {
> + int handler_id = 0;
> + struct rte_dispatcher_lcore *reference_lcore =
> + &dispatcher->lcores[0];
> +
> + if (reference_lcore->num_handlers == EVD_MAX_HANDLERS)
> + return -1;
> +
> + while (evd_lcore_get_handler_by_id(reference_lcore, handler_id) !=
> NULL)
> + handler_id++;
> +
> + return handler_id;
> +}
> +
> +static void
> +evd_lcore_install_handler(struct rte_dispatcher_lcore *lcore,
> + const struct rte_dispatcher_handler *handler) {
> + int handler_idx = lcore->num_handlers;
> +
> + lcore->handlers[handler_idx] = *handler;
> + lcore->num_handlers++;
> +}
> +
> +static void
> +evd_install_handler(struct rte_dispatcher *dispatcher,
> + const struct rte_dispatcher_handler *handler) {
> + int i;
> +
> + for (i = 0; i < RTE_MAX_LCORE; i++) {
> + struct rte_dispatcher_lcore *lcore =
> + &dispatcher->lcores[i];
> + evd_lcore_install_handler(lcore, handler);
> + }
> +}
> +
> +int
> +rte_dispatcher_register(uint8_t id,
> + rte_dispatcher_match_t match_fun,
> + void *match_data,
> + rte_dispatcher_process_t process_fun,
> + void *process_data)
> +{
> + struct rte_dispatcher *dispatcher;
> + struct rte_dispatcher_handler handler = {
> + .match_fun = match_fun,
> + .match_data = match_data,
We can have a default function which uses queue_id as matching data.
This reduces the application load to provide two callbacks, one for matching and one for processing the event.
Application can pass NULL parameter for "match_fun", in that case default function pointer can be used here.
> + .process_fun = process_fun,
> + .process_data = process_data
> + };
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
> + handler.id = evd_alloc_handler_id(dispatcher);
> +
> + if (handler.id < 0)
> + return -ENOMEM;
> +
> + evd_install_handler(dispatcher, &handler);
> +
> + return handler.id;
> +}
> +
> +static int
> +evd_lcore_uninstall_handler(struct rte_dispatcher_lcore *lcore,
> + int handler_id)
> +{
> + struct rte_dispatcher_handler *unreg_handler;
> + int handler_idx;
> + uint16_t last_idx;
> +
> + unreg_handler = evd_lcore_get_handler_by_id(lcore, handler_id);
> +
> + if (unreg_handler == NULL)
> + return -EINVAL;
> +
Shouldn't the logic be " handler_idx = unreg_handler - &lcore->handlers[0];"
Because, unreg_handler will be a higher or equal address to the handler base address (&lcore->handlers[0])
> + handler_idx = &lcore->handlers[0] - unreg_handler;
> +
> + last_idx = lcore->num_handlers - 1;
> +
> + if (handler_idx != last_idx) {
> + /* move all handlers to maintain handler order */
> + int n = last_idx - handler_idx;
> + memmove(unreg_handler, unreg_handler + 1,
> + sizeof(struct rte_dispatcher_handler) * n);
> + }
> +
> + lcore->num_handlers--;
> +
> + return 0;
> +}
> +
> +static int
> +evd_uninstall_handler(struct rte_dispatcher *dispatcher,
> + int handler_id)
> +{
> + unsigned int lcore_id;
> +
> + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
> + struct rte_dispatcher_lcore *lcore =
> + &dispatcher->lcores[lcore_id];
> + int rc;
> +
> + rc = evd_lcore_uninstall_handler(lcore, handler_id);
> +
> + if (rc < 0)
> + return rc;
> + }
> +
> + return 0;
> +}
> +
> +int
> +rte_dispatcher_unregister(uint8_t id, int handler_id) {
> + struct rte_dispatcher *dispatcher;
> + int rc;
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
> + rc = evd_uninstall_handler(dispatcher, handler_id);
> +
> + return rc;
> +}
> +
> +static struct rte_dispatcher_finalizer* evd_get_finalizer_by_id(struct
> +rte_dispatcher *dispatcher,
> + int handler_id)
> +{
> + int i;
> +
> + for (i = 0; i < dispatcher->num_finalizers; i++) {
> + struct rte_dispatcher_finalizer *finalizer =
> + &dispatcher->finalizers[i];
> +
> + if (finalizer->id == handler_id)
> + return finalizer;
> + }
> +
> + return NULL;
> +}
> +
> +static int
> +evd_alloc_finalizer_id(struct rte_dispatcher *dispatcher) {
> + int finalizer_id = 0;
> +
> + while (evd_get_finalizer_by_id(dispatcher, finalizer_id) != NULL)
> + finalizer_id++;
> +
> + return finalizer_id;
> +}
> +
> +static struct rte_dispatcher_finalizer * evd_alloc_finalizer(struct
> +rte_dispatcher *dispatcher) {
> + int finalizer_idx;
> + struct rte_dispatcher_finalizer *finalizer;
> +
> + if (dispatcher->num_finalizers == EVD_MAX_FINALIZERS)
> + return NULL;
> +
> + finalizer_idx = dispatcher->num_finalizers;
> + finalizer = &dispatcher->finalizers[finalizer_idx];
> +
> + finalizer->id = evd_alloc_finalizer_id(dispatcher);
> +
> + dispatcher->num_finalizers++;
> +
> + return finalizer;
> +}
> +
> +int
> +rte_dispatcher_finalize_register(uint8_t id,
> + rte_dispatcher_finalize_t finalize_fun,
> + void *finalize_data)
> +{
> + struct rte_dispatcher *dispatcher;
> + struct rte_dispatcher_finalizer *finalizer;
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
> + finalizer = evd_alloc_finalizer(dispatcher);
> +
> + if (finalizer == NULL)
> + return -ENOMEM;
> +
> + finalizer->finalize_fun = finalize_fun;
> + finalizer->finalize_data = finalize_data;
> +
> + return finalizer->id;
> +}
> +
> +int
> +rte_dispatcher_finalize_unregister(uint8_t id, int handler_id) {
> + struct rte_dispatcher *dispatcher;
> + struct rte_dispatcher_finalizer *unreg_finalizer;
> + int finalizer_idx;
> + uint16_t last_idx;
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
> + unreg_finalizer = evd_get_finalizer_by_id(dispatcher, handler_id);
> +
> + if (unreg_finalizer == NULL)
> + return -EINVAL;
> +
Same as above comment in rte_dispatcher_unregister, base address needs to be subtracted from unreg_finalizer
> + finalizer_idx = &dispatcher->finalizers[0] - unreg_finalizer;
> +
> + last_idx = dispatcher->num_finalizers - 1;
> +
> + if (finalizer_idx != last_idx) {
> + /* move all finalizers to maintain order */
> + int n = last_idx - finalizer_idx;
> + memmove(unreg_finalizer, unreg_finalizer + 1,
> + sizeof(struct rte_dispatcher_finalizer) * n);
> + }
> +
> + dispatcher->num_finalizers--;
> +
> + return 0;
> +}
> +
> +static int
> +evd_set_service_runstate(uint8_t id, int state) {
> + struct rte_dispatcher *dispatcher;
> + int rc;
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
> + rc = rte_service_component_runstate_set(dispatcher->service_id,
> + state);
> +
> + if (rc != 0) {
> + RTE_EDEV_LOG_ERR("Unexpected error %d occurred while
> setting "
> + "service component run state to %d\n", rc,
> + state);
> + RTE_ASSERT(0);
> + }
> +
> + return 0;
> +}
> +
> +int
> +rte_dispatcher_start(uint8_t id)
> +{
> + return evd_set_service_runstate(id, 1); }
> +
> +int
> +rte_dispatcher_stop(uint8_t id)
> +{
> + return evd_set_service_runstate(id, 0); }
> +
> +static void
> +evd_aggregate_stats(struct rte_dispatcher_stats *result,
> + const struct rte_dispatcher_stats *part) {
> + result->poll_count += part->poll_count;
> + result->ev_batch_count += part->ev_batch_count;
> + result->ev_dispatch_count += part->ev_dispatch_count;
> + result->ev_drop_count += part->ev_drop_count; }
> +
> +int
> +rte_dispatcher_stats_get(uint8_t id,
> + struct rte_dispatcher_stats *stats) {
> + struct rte_dispatcher *dispatcher;
> + unsigned int lcore_id;
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
Stats pointer needs to be validated for NULL before accessing
> + *stats = (struct rte_dispatcher_stats) {};
> +
> + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
> + struct rte_dispatcher_lcore *lcore =
> + &dispatcher->lcores[lcore_id];
> +
> + evd_aggregate_stats(stats, &lcore->stats);
> + }
> +
> + return 0;
> +}
> +
> +int
> +rte_dispatcher_stats_reset(uint8_t id)
> +{
> + struct rte_dispatcher *dispatcher;
> + unsigned int lcore_id;
> +
> + EVD_VALID_ID_OR_RET_EINVAL(id);
> + dispatcher = evd_get_dispatcher(id);
> +
> +
> + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
> + struct rte_dispatcher_lcore *lcore =
> + &dispatcher->lcores[lcore_id];
> +
> + lcore->stats = (struct rte_dispatcher_stats) {};
> + }
> +
> + return 0;
> +
> +}
> diff --git a/lib/dispatcher/rte_dispatcher.h b/lib/dispatcher/rte_dispatcher.h
> new file mode 100644 index 0000000000..6712687a08
> --- /dev/null
> +++ b/lib/dispatcher/rte_dispatcher.h
> @@ -0,0 +1,480 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2023 Ericsson AB
> + */
> +
> +#ifndef __RTE_DISPATCHER_H__
> +#define __RTE_DISPATCHER_H__
> +
> +/**
> + * @file
> + *
> + * RTE Dispatcher
> + *
> + * The purpose of the dispatcher is to help decouple different parts
> + * of an application (e.g., modules), sharing the same underlying
> + * event device.
> + */
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +#include <rte_eventdev.h>
> +
> +/**
> + * Function prototype for match callbacks.
> + *
> + * Match callbacks are used by an application to decide how the
> + * dispatcher distributes events to different parts of the
> + * application.
> + *
> + * The application is not expected to process the event at the point
> + * of the match call. Such matters should be deferred to the process
> + * callback invocation.
> + *
> + * The match callback may be used as an opportunity to prefetch data.
> + *
> + * @param event
> + * Pointer to event
> + *
> + * @param cb_data
> + * The pointer supplied by the application in
> + * rte_dispatcher_register().
> + *
> + * @return
> + * Returns true in case this events should be delivered (via
> + * the process callback), and false otherwise.
> + */
> +typedef bool
> +(*rte_dispatcher_match_t)(const struct rte_event *event, void
> +*cb_data);
> +
> +/**
> + * Function prototype for process callbacks.
> + *
> + * The process callbacks are used by the dispatcher to deliver
> + * events for processing.
> + *
> + * @param event_dev_id
> + * The originating event device id.
> + *
> + * @param event_port_id
> + * The originating event port.
> + *
> + * @param events
> + * Pointer to an array of events.
> + *
> + * @param num
> + * The number of events in the @p events array.
> + *
> + * @param cb_data
> + * The pointer supplied by the application in
> + * rte_dispatcher_register().
> + */
> +
> +typedef void
> +(*rte_dispatcher_process_t)(uint8_t event_dev_id, uint8_t event_port_id,
> + struct rte_event *events, uint16_t num,
> + void *cb_data);
> +
> +/**
> + * Function prototype for finalize callbacks.
> + *
> + * The finalize callbacks are used by the dispatcher to notify the
> + * application it has delivered all events from a particular batch
> + * dequeued from the event device.
> + *
> + * @param event_dev_id
> + * The originating event device id.
> + *
> + * @param event_port_id
> + * The originating event port.
> + *
> + * @param cb_data
> + * The pointer supplied by the application in
> + * rte_dispatcher_finalize_register().
> + */
> +
> +typedef void
> +(*rte_dispatcher_finalize_t)(uint8_t event_dev_id, uint8_t event_port_id,
> + void *cb_data);
> +
> +/**
> + * Dispatcher statistics
> + */
> +struct rte_dispatcher_stats {
> + uint64_t poll_count;
> + /**< Number of event dequeue calls made toward the event device. */
> + uint64_t ev_batch_count;
> + /**< Number of non-empty event batches dequeued from event
> device.*/
> + uint64_t ev_dispatch_count;
> + /**< Number of events dispatched to a handler.*/
> + uint64_t ev_drop_count;
> + /**< Number of events dropped because no handler was found. */ };
> +
> +/**
> + * Create a dispatcher with the specified id.
> + *
> + * @param id
> + * An application-specified, unique (across all dispatcher
> + * instances) identifier.
> + *
> + * @param event_dev_id
> + * The identifier of the event device from which this dispatcher
> + * will dequeue events.
> + *
> + * @return
> + * - 0: Success
> + * - <0: Error code on failure
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_create(uint8_t id, uint8_t event_dev_id);
> +
> +/**
> + * Free a dispatcher.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @return
> + * - 0: Success
> + * - <0: Error code on failure
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_free(uint8_t id);
> +
> +/**
> + * Retrieve the service identifier of a dispatcher.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @param [out] service_id
> + * A pointer to a caller-supplied buffer where the dispatcher's
> + * service id will be stored.
> + *
> + * @return
> + * - 0: Success
> + * - <0: Error code on failure.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_service_id_get(uint8_t id, uint32_t *service_id);
> +
> +/**
> + * Binds an event device port to a specific lcore on the specified
> + * dispatcher.
> + *
> + * This function configures the event port id to be used by the event
> + * dispatcher service, if run on the specified lcore.
> + *
> + * Multiple event device ports may be bound to the same lcore. A
> + * particular port must not be bound to more than one lcore.
> + *
> + * If the dispatcher service is mapped (with
> +rte_service_map_lcore_set())
> + * to a lcore to which no ports are bound, the service function will be
> +a
> + * no-operation.
> + *
> + * This function may be called by any thread (including unregistered
> + * non-EAL threads), but not while the dispatcher is running on lcore
> + * specified by @c lcore_id.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @param event_port_id
> + * The event device port identifier.
> + *
> + * @param batch_size
> + * The batch size to use in rte_event_dequeue_burst(), for the
> + * configured event device port and lcore.
> + *
> + * @param timeout
> + * The timeout parameter to use in rte_event_dequeue_burst(), for the
> + * configured event device port and lcore.
> + *
> + * @param lcore_id
> + * The lcore by which this event port will be used.
> + *
> + * @return
> + * - 0: Success
> + * - -ENOMEM: Unable to allocate sufficient resources.
> + * - -EEXISTS: Event port is already configured.
> + * - -EINVAL: Invalid arguments.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_bind_port_to_lcore(uint8_t id, uint8_t event_port_id,
> + uint16_t batch_size, uint64_t
> timeout,
> + unsigned int lcore_id);
> +
> +/**
> + * Unbind an event device port from a specific lcore.
> + *
> + * This function may be called by any thread (including unregistered
> + * non-EAL threads), but not while the dispatcher is running on
> + * lcore specified by @c lcore_id.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @param event_port_id
> + * The event device port identifier.
> + *
> + * @param lcore_id
> + * The lcore which was using this event port.
> + *
> + * @return
> + * - 0: Success
> + * - -EINVAL: Invalid @c id.
> + * - -ENOENT: Event port id not bound to this @c lcore_id.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_unbind_port_from_lcore(uint8_t id, uint8_t event_port_id,
> + unsigned int lcore_id);
> +
> +/**
> + * Register an event handler.
> + *
> + * The match callback function is used to select if a particular event
> + * should be delivered, using the corresponding process callback
> + * function.
> + *
> + * The reason for having two distinct steps is to allow the dispatcher
> + * to deliver all events as a batch. This in turn will cause
> + * processing of a particular kind of events to happen in a
> + * back-to-back manner, improving cache locality.
> + *
> + * The list of handler callback functions is shared among all lcores,
> + * but will only be executed on lcores which has an eventdev port
> + * bound to them, and which are running the dispatcher service.
> + *
> + * An event is delivered to at most one handler. Events where no
> + * handler is found are dropped.
> + *
> + * The application must not depend on the order of which the match
> + * functions are invoked.
> + *
> + * Ordering of events is not guaranteed to be maintained between
> + * different deliver callbacks. For example, suppose there are two
> + * callbacks registered, matching different subsets of events arriving
> + * on an atomic queue. A batch of events [ev0, ev1, ev2] are dequeued
> + * on a particular port, all pertaining to the same flow. The match
> + * callback for registration A returns true for ev0 and ev2, and the
> + * matching function for registration B for ev1. In that scenario, the
> + * dispatcher may choose to deliver first [ev0, ev2] using A's deliver
> + * function, and then [ev1] to B - or vice versa.
> + *
> + * rte_dispatcher_register() may be called by any thread
> + * (including unregistered non-EAL threads), but not while the event
> + * dispatcher is running on any service lcore.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @param match_fun
> + * The match callback function.
> + *
> + * @param match_cb_data
> + * A pointer to some application-specific opaque data (or NULL),
> + * which is supplied back to the application when match_fun is
> + * called.
> + *
> + * @param process_fun
> + * The process callback function.
> + *
> + * @param process_cb_data
> + * A pointer to some application-specific opaque data (or NULL),
> + * which is supplied back to the application when process_fun is
> + * called.
> + *
> + * @return
> + * - >= 0: The identifier for this registration.
> + * - -ENOMEM: Unable to allocate sufficient resources.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_register(uint8_t id,
> + rte_dispatcher_match_t match_fun,
> + void *match_cb_data,
> + rte_dispatcher_process_t process_fun,
> + void *process_cb_data);
> +
> +/**
> + * Unregister an event handler.
> + *
> + * This function may be called by any thread (including unregistered
> + * non-EAL threads), but not while the dispatcher is running on
> + * any service lcore.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @param handler_id
> + * The handler registration id returned by the original
> + * rte_dispatcher_register() call.
> + *
> + * @return
> + * - 0: Success
> + * - -EINVAL: The @c id and/or the @c handler_id parameter was invalid.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_unregister(uint8_t id, int handler_id);
> +
> +/**
> + * Register a finalize callback function.
> + *
> + * An application may optionally install one or more finalize
> + * callbacks.
> + *
> + * All finalize callbacks are invoked by the dispatcher when a
> + * complete batch of events (retrieve using rte_event_dequeue_burst())
> + * have been delivered to the application (or have been dropped).
> + *
> + * The finalize callback is not tied to any particular handler.
> + *
> + * The finalize callback provides an opportunity for the application
> + * to do per-batch processing. One case where this may be useful is if
> + * an event output buffer is used, and is shared among several
> + * handlers. In such a case, proper output buffer flushing may be
> + * assured using a finalize callback.
> + *
> + * rte_dispatcher_finalize_register() may be called by any thread
> + * (including unregistered non-EAL threads), but not while the
> + * dispatcher is running on any service lcore.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @param finalize_fun
> + * The function called after completing the processing of a
> + * dequeue batch.
> + *
> + * @param finalize_data
> + * A pointer to some application-specific opaque data (or NULL),
> + * which is supplied back to the application when @c finalize_fun is
> + * called.
> + *
> + * @return
> + * - >= 0: The identifier for this registration.
> + * - -ENOMEM: Unable to allocate sufficient resources.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_finalize_register(uint8_t id,
> + rte_dispatcher_finalize_t finalize_fun,
> + void *finalize_data);
> +
> +/**
> + * Unregister a finalize callback.
> + *
> + * This function may be called by any thread (including unregistered
> + * non-EAL threads), but not while the dispatcher is running on
> + * any service lcore.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @param reg_id
> + * The finalize registration id returned by the original
> + * rte_dispatcher_finalize_register() call.
> + *
> + * @return
> + * - 0: Success
> + * - -EINVAL: The @c id and/or the @c reg_id parameter was invalid.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_finalize_unregister(uint8_t id, int reg_id);
> +
> +/**
> + * Start a dispatcher instance.
> + *
> + * Enables the dispatcher service.
> + *
> + * The underlying event device must have been started prior to calling
> + * rte_dispatcher_start().
> + *
> + * For the dispatcher to actually perform work (i.e., dispatch
> + * events), its service must have been mapped to one or more service
> + * lcores, and its service run state set to '1'. A dispatcher's
> + * service is retrieved using rte_dispatcher_service_id_get().
> + *
> + * Each service lcore to which the dispatcher is mapped should
> + * have at least one event port configured. Such configuration is
> + * performed by calling rte_dispatcher_bind_port_to_lcore(), prior to
> + * starting the dispatcher.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @return
> + * - 0: Success
> + * - -EINVAL: Invalid @c id.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_start(uint8_t id);
> +
> +/**
> + * Stop a running dispatcher instance.
> + *
> + * Disables the dispatcher service.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @return
> + * - 0: Success
> + * - -EINVAL: Invalid @c id.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_stop(uint8_t id);
> +
> +/**
> + * Retrieve statistics for a dispatcher instance.
> + *
> + * This function is MT safe and may be called by any thread
> + * (including unregistered non-EAL threads).
> + *
> + * @param id
> + * The dispatcher identifier.
> + * @param[out] stats
> + * A pointer to a structure to fill with statistics.
> + * @return
> + * - 0: Success
> + * - -EINVAL: The @c id parameter was invalid.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_stats_get(uint8_t id,
> + struct rte_dispatcher_stats *stats);
> +
> +/**
> + * Reset statistics for a dispatcher instance.
> + *
> + * This function may be called by any thread (including unregistered
> + * non-EAL threads), but may not produce the correct result if the
> + * dispatcher is running on any service lcore.
> + *
> + * @param id
> + * The dispatcher identifier.
> + *
> + * @return
> + * - 0: Success
> + * - -EINVAL: The @c id parameter was invalid.
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_stats_reset(uint8_t id);
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +#endif /* __RTE_DISPATCHER__ */
> diff --git a/lib/dispatcher/version.map b/lib/dispatcher/version.map new file
> mode 100644 index 0000000000..8f9ad96522
> --- /dev/null
> +++ b/lib/dispatcher/version.map
> @@ -0,0 +1,20 @@
> +EXPERIMENTAL {
> + global:
> +
> + # added in 23.11
> + rte_dispatcher_create;
> + rte_dispatcher_free;
> + rte_dispatcher_service_id_get;
> + rte_dispatcher_bind_port_to_lcore;
> + rte_dispatcher_unbind_port_from_lcore;
> + rte_dispatcher_register;
> + rte_dispatcher_unregister;
> + rte_dispatcher_finalize_register;
> + rte_dispatcher_finalize_unregister;
> + rte_dispatcher_start;
> + rte_dispatcher_stop;
> + rte_dispatcher_stats_get;
> + rte_dispatcher_stats_reset;
> +
> + local: *;
> +};
> diff --git a/lib/meson.build b/lib/meson.build index
> 099b0ed18a..3093b338d2 100644
> --- a/lib/meson.build
> +++ b/lib/meson.build
> @@ -35,6 +35,7 @@ libraries = [
> 'distributor',
> 'efd',
> 'eventdev',
> + 'dispatcher', # dispatcher depends on eventdev
> 'gpudev',
> 'gro',
> 'gso',
> @@ -81,6 +82,7 @@ optional_libs = [
> 'cfgfile',
> 'compressdev',
> 'cryptodev',
> + 'dispatcher',
> 'distributor',
> 'dmadev',
> 'efd',
> --
> 2.34.1
On 2023-09-17 18:46, Naga Harish K, S V wrote:
>
>
>> -----Original Message-----
>> From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>> Sent: Monday, September 4, 2023 6:33 PM
>> To: dev@dpdk.org
>> Cc: Jerin Jacob <jerinj@marvell.com>; techboard@dpdk.org; Van Haaren,
>> Harry <harry.van.haaren@intel.com>; hofors@lysator.liu.se; Nilsson, Peter
>> <peter.j.nilsson@ericsson.com>; Heng Wang <heng.wang@ericsson.com>;
>> Naga Harish K, S V <s.v.naga.harish.k@intel.com>; Pavan Nikhilesh
>> <pbhagavatula@marvell.com>; Gujjar, Abhinandan S
>> <abhinandan.gujjar@intel.com>; Carrillo, Erik G <Erik.G.Carrillo@intel.com>;
>> Shijith Thotton <sthotton@marvell.com>; Hemant Agrawal
>> <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
>> Liang Ma <liangma@liangbit.com>; Mccarthy, Peter
>> <Peter.Mccarthy@intel.com>; Yan, Zhirun <Zhirun.Yan@intel.com>;
>> mattias.ronnblom <mattias.ronnblom@ericsson.com>
>> Subject: [PATCH v3 1/3] lib: introduce dispatcher library
>>
>> The purpose of the dispatcher library is to help reduce coupling in an
>> Eventdev-based DPDK application.
>>
>> In addition, the dispatcher also provides a convenient and flexible way for the
>> application to use service cores for application-level processing.
>>
>> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>> Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
>> Reviewed-by: Heng Wang <heng.wang@ericsson.com>
>>
>> --
>>
>> PATCH v3:
>> o To underline its optional character and since it does not provide
>> hardware abstraction, the event dispatcher is now a separate
>> library.
>> o Change name from rte_event_dispatcher -> rte_dispatcher, to make it
>> shorter and to avoid the rte_event_* namespace.
>>
>
> Rte_dispatcher is basically dispatching events but it feels like the name does not convey that.
> Also, it is like any other adapter service that can reside within the eventdev directory.
>
> I can see some discussion in previous threads related to the placement of the dispatcher library.
>
> It is an optional eventdev application service, not enforcing this programming model to the application.
> The documentation may need to be updated and mention that this is optional.
>
> If any hardware comes up with the dispatcher feature, then this library may need to be moved inside eventdev library later.
>
It seems to me that the deciding factor for what functionality goes into
a DPDK library or not is not such much dependent on if it's implemented
in hardware, in software, or some combination thereof. The important
thing is that the library is be able to present a coherent API to the
application (or other libraries).
That said, as I've mentioned before, I have no strong opionion on this
subject.
> So, It makes sense to keep this optional service in the eventdev folder as an optional feature.
>
>> PATCH v2:
>> o Add dequeue batch count statistic.
>> o Add statistics reset function to API.
>> o Clarify MT safety guarantees (or lack thereof) in the API documentation.
>> o Change loop variable type in evd_lcore_get_handler_by_id() to uint16_t,
>> to be consistent with similar loops elsewhere in the dispatcher.
>> o Fix variable names in finalizer unregister function.
>>
>> PATCH:
>> o Change prefix from RED to EVD, to avoid confusion with random
>> early detection.
>>
>> RFC v4:
>> o Move handlers to per-lcore data structures.
>> o Introduce mechanism which rearranges handlers so that often-used
>> handlers tend to be tried first.
>> o Terminate dispatch loop in case all events are delivered.
>> o To avoid the dispatcher's service function hogging the CPU, process
>> only one batch per call.
>> o Have service function return -EAGAIN if no work is performed.
>> o Events delivered in the process function is no longer marked 'const',
>> since modifying them may be useful for the application and cause
>> no difficulties for the dispatcher.
>> o Various minor API documentation improvements.
>>
>> RFC v3:
>> o Add stats_get() function to the version.map file.
>> ---
>> MAINTAINERS | 3 +
>> lib/dispatcher/meson.build | 17 +
>> lib/dispatcher/rte_dispatcher.c | 791
>> ++++++++++++++++++++++++++++++++ lib/dispatcher/rte_dispatcher.h |
>> 480 +++++++++++++++++++
>> lib/dispatcher/version.map | 20 +
>> lib/meson.build | 2 +
>> 6 files changed, 1313 insertions(+)
>> create mode 100644 lib/dispatcher/meson.build create mode 100644
>> lib/dispatcher/rte_dispatcher.c create mode 100644
>> lib/dispatcher/rte_dispatcher.h create mode 100644
>> lib/dispatcher/version.map
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index a926155f26..6704cd5b2c 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -1726,6 +1726,9 @@ M: Nithin Dabilpuram
>> <ndabilpuram@marvell.com>
>> M: Pavan Nikhilesh <pbhagavatula@marvell.com>
>> F: lib/node/
>>
>> +Dispatcher - EXPERIMENTAL
>> +M: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>> +F: lib/dispatcher/
>>
>> Test Applications
>> -----------------
>> diff --git a/lib/dispatcher/meson.build b/lib/dispatcher/meson.build new file
>> mode 100644 index 0000000000..c6054a3a5d
>> --- /dev/null
>> +++ b/lib/dispatcher/meson.build
>> @@ -0,0 +1,17 @@
>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 Ericsson AB
>> +
>> +if is_windows
>> + build = false
>> + reason = 'not supported on Windows'
>> + subdir_done()
>> +endif
>> +
>> +sources = files(
>> + 'rte_dispatcher.c',
>> +)
>> +headers = files(
>> + 'rte_dispatcher.h',
>> +)
>> +
>> +deps += ['eventdev']
>> diff --git a/lib/dispatcher/rte_dispatcher.c b/lib/dispatcher/rte_dispatcher.c
>> new file mode 100644 index 0000000000..3319fe09f2
>> --- /dev/null
>> +++ b/lib/dispatcher/rte_dispatcher.c
>> @@ -0,0 +1,791 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2023 Ericsson AB
>> + */
>> +
>> +#include <stdbool.h>
>> +#include <stdint.h>
>> +
>> +#include <rte_branch_prediction.h>
>> +#include <rte_common.h>
>> +#include <rte_lcore.h>
>> +#include <rte_random.h>
>> +#include <rte_service_component.h>
>> +
>> +#include "eventdev_pmd.h"
>> +
>> +#include <rte_dispatcher.h>
>> +
>> +#define EVD_MAX_PORTS_PER_LCORE 4
>> +#define EVD_MAX_HANDLERS 32
>> +#define EVD_MAX_FINALIZERS 16
>> +#define EVD_AVG_PRIO_INTERVAL 2000
>> +
>> +struct rte_dispatcher_lcore_port {
>> + uint8_t port_id;
>> + uint16_t batch_size;
>> + uint64_t timeout;
>> +};
>> +
>> +struct rte_dispatcher_handler {
>> + int id;
>> + rte_dispatcher_match_t match_fun;
>> + void *match_data;
>> + rte_dispatcher_process_t process_fun;
>> + void *process_data;
>> +};
>> +
>> +struct rte_dispatcher_finalizer {
>> + int id;
>> + rte_dispatcher_finalize_t finalize_fun;
>> + void *finalize_data;
>> +};
>> +
>> +struct rte_dispatcher_lcore {
>> + uint8_t num_ports;
>> + uint16_t num_handlers;
>> + int32_t prio_count;
>> + struct rte_dispatcher_lcore_port
>> ports[EVD_MAX_PORTS_PER_LCORE];
>> + struct rte_dispatcher_handler handlers[EVD_MAX_HANDLERS];
>> + struct rte_dispatcher_stats stats;
>> +} __rte_cache_aligned;
>> +
>> +struct rte_dispatcher {
>> + uint8_t id;
>> + uint8_t event_dev_id;
>> + int socket_id;
>> + uint32_t service_id;
>> + struct rte_dispatcher_lcore lcores[RTE_MAX_LCORE];
>> + uint16_t num_finalizers;
>> + struct rte_dispatcher_finalizer finalizers[EVD_MAX_FINALIZERS]; };
>> +
>> +static struct rte_dispatcher *dispatchers[UINT8_MAX];
>> +
>> +static bool
>> +evd_has_dispatcher(uint8_t id)
>> +{
>> + return dispatchers[id] != NULL;
>> +}
>> +
>> +static struct rte_dispatcher *
>> +evd_get_dispatcher(uint8_t id)
>> +{
>> + return dispatchers[id];
>> +}
>> +
>> +static void
>> +evd_set_dispatcher(uint8_t id, struct rte_dispatcher *dispatcher) {
>> + dispatchers[id] = dispatcher;
>> +}
>> +
>> +#define EVD_VALID_ID_OR_RET_EINVAL(id)
>> \
>> + do { \
>> + if (unlikely(!evd_has_dispatcher(id))) { \
>> + RTE_EDEV_LOG_ERR("Invalid dispatcher id %d\n", id);
>> \
>> + return -EINVAL; \
>> + } \
>> + } while (0)
>> +
>> +static int
>> +evd_lookup_handler_idx(struct rte_dispatcher_lcore *lcore,
>> + const struct rte_event *event) {
>> + uint16_t i;
>> +
>> + for (i = 0; i < lcore->num_handlers; i++) {
>> + struct rte_dispatcher_handler *handler =
>> + &lcore->handlers[i];
>> +
>> + if (handler->match_fun(event, handler->match_data))
>> + return i;
>> + }
>> +
>> + return -1;
>> +}
>> +
>> +static void
>> +evd_prioritize_handler(struct rte_dispatcher_lcore *lcore,
>> + int handler_idx)
>> +{
>> + struct rte_dispatcher_handler tmp;
>> +
>> + if (handler_idx == 0)
>> + return;
>> +
>> + /* Let the lucky handler "bubble" up the list */
>> +
>> + tmp = lcore->handlers[handler_idx - 1];
>> +
>> + lcore->handlers[handler_idx - 1] = lcore->handlers[handler_idx];
>> +
>> + lcore->handlers[handler_idx] = tmp;
>> +}
>> +
>> +static inline void
>> +evd_consider_prioritize_handler(struct rte_dispatcher_lcore *lcore,
>> + int handler_idx, uint16_t handler_events) {
>> + lcore->prio_count -= handler_events;
>> +
>> + if (unlikely(lcore->prio_count <= 0)) {
>> + evd_prioritize_handler(lcore, handler_idx);
>> +
>> + /*
>> + * Randomize the interval in the unlikely case
>> + * the traffic follow some very strict pattern.
>> + */
>> + lcore->prio_count =
>> + rte_rand_max(EVD_AVG_PRIO_INTERVAL) +
>> + EVD_AVG_PRIO_INTERVAL / 2;
>> + }
>> +}
>> +
>> +static inline void
>> +evd_dispatch_events(struct rte_dispatcher *dispatcher,
>> + struct rte_dispatcher_lcore *lcore,
>> + struct rte_dispatcher_lcore_port *port,
>> + struct rte_event *events, uint16_t num_events) {
>> + int i;
>> + struct rte_event bursts[EVD_MAX_HANDLERS][num_events];
>> + uint16_t burst_lens[EVD_MAX_HANDLERS] = { 0 };
>> + uint16_t drop_count = 0;
>> + uint16_t dispatch_count;
>> + uint16_t dispatched = 0;
>> +
>> + for (i = 0; i < num_events; i++) {
>> + struct rte_event *event = &events[i];
>> + int handler_idx;
>> +
>> + handler_idx = evd_lookup_handler_idx(lcore, event);
>> +
>> + if (unlikely(handler_idx < 0)) {
>> + drop_count++;
>> + continue;
>> + }
>> +
>> + bursts[handler_idx][burst_lens[handler_idx]] = *event;
>> + burst_lens[handler_idx]++;
>> + }
>> +
>> + dispatch_count = num_events - drop_count;
>> +
>> + for (i = 0; i < lcore->num_handlers &&
>> + dispatched < dispatch_count; i++) {
>> + struct rte_dispatcher_handler *handler =
>> + &lcore->handlers[i];
>> + uint16_t len = burst_lens[i];
>> +
>> + if (len == 0)
>> + continue;
>> +
>> + handler->process_fun(dispatcher->event_dev_id, port-
>>> port_id,
>> + bursts[i], len, handler->process_data);
>> +
>> + dispatched += len;
>> +
>> + /*
>> + * Safe, since any reshuffling will only involve
>> + * already-processed handlers.
>> + */
>> + evd_consider_prioritize_handler(lcore, i, len);
>> + }
>> +
>> + lcore->stats.ev_batch_count++;
>> + lcore->stats.ev_dispatch_count += dispatch_count;
>> + lcore->stats.ev_drop_count += drop_count;
>> +
>> + for (i = 0; i < dispatcher->num_finalizers; i++) {
>> + struct rte_dispatcher_finalizer *finalizer =
>> + &dispatcher->finalizers[i];
>> +
>> + finalizer->finalize_fun(dispatcher->event_dev_id,
>> + port->port_id,
>> + finalizer->finalize_data);
>> + }
>> +}
>> +
>> +static __rte_always_inline uint16_t
>> +evd_port_dequeue(struct rte_dispatcher *dispatcher,
>> + struct rte_dispatcher_lcore *lcore,
>> + struct rte_dispatcher_lcore_port *port) {
>> + uint16_t batch_size = port->batch_size;
>> + struct rte_event events[batch_size];
>> + uint16_t n;
>> +
>> + n = rte_event_dequeue_burst(dispatcher->event_dev_id, port-
>>> port_id,
>> + events, batch_size, port->timeout);
>> +
>> + if (likely(n > 0))
>> + evd_dispatch_events(dispatcher, lcore, port, events, n);
>> +
>> + lcore->stats.poll_count++;
>> +
>> + return n;
>> +}
>> +
>> +static __rte_always_inline uint16_t
>> +evd_lcore_process(struct rte_dispatcher *dispatcher,
>> + struct rte_dispatcher_lcore *lcore) {
>> + uint16_t i;
>> + uint16_t event_count = 0;
>> +
>> + for (i = 0; i < lcore->num_ports; i++) {
>> + struct rte_dispatcher_lcore_port *port =
>> + &lcore->ports[i];
>> +
>> + event_count += evd_port_dequeue(dispatcher, lcore, port);
>> + }
>> +
>> + return event_count;
>> +}
>> +
>> +static int32_t
>> +evd_process(void *userdata)
>> +{
>> + struct rte_dispatcher *dispatcher = userdata;
>> + unsigned int lcore_id = rte_lcore_id();
>> + struct rte_dispatcher_lcore *lcore =
>> + &dispatcher->lcores[lcore_id];
>> + uint64_t event_count;
>> +
>> + event_count = evd_lcore_process(dispatcher, lcore);
>> +
>> + if (unlikely(event_count == 0))
>> + return -EAGAIN;
>> +
>> + return 0;
>> +}
>> +
>> +static int
>> +evd_service_register(struct rte_dispatcher *dispatcher) {
>> + struct rte_service_spec service = {
>> + .callback = evd_process,
>> + .callback_userdata = dispatcher,
>> + .capabilities = RTE_SERVICE_CAP_MT_SAFE,
>> + .socket_id = dispatcher->socket_id
>> + };
>> + int rc;
>> +
>> + snprintf(service.name, RTE_SERVICE_NAME_MAX - 1, "evd_%d",
>> + dispatcher->id);
>> +
>> + rc = rte_service_component_register(&service,
>> +&dispatcher->service_id);
>> +
>> + if (rc)
>> + RTE_EDEV_LOG_ERR("Registration of dispatcher service "
>> + "%s failed with error code %d\n",
>> + service.name, rc);
>> +
>> + return rc;
>> +}
>> +
>> +static int
>> +evd_service_unregister(struct rte_dispatcher *dispatcher) {
>> + int rc;
>> +
>> + rc = rte_service_component_unregister(dispatcher->service_id);
>> +
>> + if (rc)
>> + RTE_EDEV_LOG_ERR("Unregistration of dispatcher service "
>> + "failed with error code %d\n", rc);
>> +
>> + return rc;
>> +}
>> +
>> +int
>> +rte_dispatcher_create(uint8_t id, uint8_t event_dev_id) {
>> + int socket_id;
>> + struct rte_dispatcher *dispatcher;
>> + int rc;
>> +
>> + if (evd_has_dispatcher(id)) {
>> + RTE_EDEV_LOG_ERR("Dispatcher with id %d already exists\n",
>> + id);
>> + return -EEXIST;
>> + }
>> +
>> + socket_id = rte_event_dev_socket_id(event_dev_id);
>> +
>> + dispatcher =
>> + rte_malloc_socket("dispatcher", sizeof(struct rte_dispatcher),
>> + RTE_CACHE_LINE_SIZE, socket_id);
>> +
>> + if (dispatcher == NULL) {
>> + RTE_EDEV_LOG_ERR("Unable to allocate memory for
>> dispatcher\n");
>> + return -ENOMEM;
>> + }
>> +
>> + *dispatcher = (struct rte_dispatcher) {
>> + .id = id,
>> + .event_dev_id = event_dev_id,
>> + .socket_id = socket_id
>> + };
>> +
>> + rc = evd_service_register(dispatcher);
>> +
>> + if (rc < 0) {
>> + rte_free(dispatcher);
>> + return rc;
>> + }
>> +
>> + evd_set_dispatcher(id, dispatcher);
>> +
>> + return 0;
>> +}
>> +
>> +int
>> +rte_dispatcher_free(uint8_t id)
>> +{
>> + struct rte_dispatcher *dispatcher;
>> + int rc;
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>> + rc = evd_service_unregister(dispatcher);
>> +
>> + if (rc)
>> + return rc;
>> +
>> + evd_set_dispatcher(id, NULL);
>> +
>> + rte_free(dispatcher);
>> +
>> + return 0;
>> +}
>> +
>> +int
>> +rte_dispatcher_service_id_get(uint8_t id, uint32_t *service_id) {
>> + struct rte_dispatcher *dispatcher;
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>
> Service_id pointer needs to be validated for NULL before accessing
>
>
Noted.
Returning error codes on API violations is bad practice (such should be
met with assertions), but it is DPDK standard practice and how Eventdev
does things as well, so I'll change it.
It would also be consistent with allowing invalid 'id' parameter values.
>> + *service_id = dispatcher->service_id;
>> +
>> + return 0;
>> +}
>> +
>> +static int
>> +lcore_port_index(struct rte_dispatcher_lcore *lcore,
>> + uint8_t event_port_id)
>> +{
>> + uint16_t i;
>> +
>> + for (i = 0; i < lcore->num_ports; i++) {
>> + struct rte_dispatcher_lcore_port *port =
>> + &lcore->ports[i];
>> +
>> + if (port->port_id == event_port_id)
>> + return i;
>> + }
>> +
>> + return -1;
>> +}
>> +
>> +int
>> +rte_dispatcher_bind_port_to_lcore(uint8_t id, uint8_t event_port_id,
>> + uint16_t batch_size, uint64_t
>> timeout,
>> + unsigned int lcore_id)
>> +{
>> + struct rte_dispatcher *dispatcher;
>> + struct rte_dispatcher_lcore *lcore;
>> + struct rte_dispatcher_lcore_port *port;
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>> + lcore = &dispatcher->lcores[lcore_id];
>> +
>> + if (lcore->num_ports == EVD_MAX_PORTS_PER_LCORE)
>> + return -ENOMEM;
>> +
>> + if (lcore_port_index(lcore, event_port_id) >= 0)
>> + return -EEXIST;
>> +
>> + port = &lcore->ports[lcore->num_ports];
>> +
>> + *port = (struct rte_dispatcher_lcore_port) {
>> + .port_id = event_port_id,
>> + .batch_size = batch_size,
>> + .timeout = timeout
>> + };
>> +
>> + lcore->num_ports++;
>> +
>> + return 0;
>> +}
>> +
>> +int
>> +rte_dispatcher_unbind_port_from_lcore(uint8_t id, uint8_t event_port_id,
>> + unsigned int lcore_id)
>> +{
>> + struct rte_dispatcher *dispatcher;
>> + struct rte_dispatcher_lcore *lcore;
>> + int port_idx;
>> + struct rte_dispatcher_lcore_port *port;
>> + struct rte_dispatcher_lcore_port *last;
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>> + lcore = &dispatcher->lcores[lcore_id];
>> +
>> + port_idx = lcore_port_index(lcore, event_port_id);
>> +
>> + if (port_idx < 0)
>> + return -ENOENT;
>> +
>> + port = &lcore->ports[port_idx];
>> + last = &lcore->ports[lcore->num_ports - 1];
>> +
>> + if (port != last)
>> + *port = *last;
>> +
>> + lcore->num_ports--;
>> +
>> + return 0;
>> +}
>> +
>> +static struct rte_dispatcher_handler*
>> +evd_lcore_get_handler_by_id(struct rte_dispatcher_lcore *lcore,
>> + int handler_id)
>> +{
>> + uint16_t i;
>> +
>> + for (i = 0; i < lcore->num_handlers; i++) {
>> + struct rte_dispatcher_handler *handler =
>> + &lcore->handlers[i];
>> +
>> + if (handler->id == handler_id)
>> + return handler;
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +static int
>> +evd_alloc_handler_id(struct rte_dispatcher *dispatcher) {
>> + int handler_id = 0;
>> + struct rte_dispatcher_lcore *reference_lcore =
>> + &dispatcher->lcores[0];
>> +
>> + if (reference_lcore->num_handlers == EVD_MAX_HANDLERS)
>> + return -1;
>> +
>> + while (evd_lcore_get_handler_by_id(reference_lcore, handler_id) !=
>> NULL)
>> + handler_id++;
>> +
>> + return handler_id;
>> +}
>> +
>> +static void
>> +evd_lcore_install_handler(struct rte_dispatcher_lcore *lcore,
>> + const struct rte_dispatcher_handler *handler) {
>> + int handler_idx = lcore->num_handlers;
>> +
>> + lcore->handlers[handler_idx] = *handler;
>> + lcore->num_handlers++;
>> +}
>> +
>> +static void
>> +evd_install_handler(struct rte_dispatcher *dispatcher,
>> + const struct rte_dispatcher_handler *handler) {
>> + int i;
>> +
>> + for (i = 0; i < RTE_MAX_LCORE; i++) {
>> + struct rte_dispatcher_lcore *lcore =
>> + &dispatcher->lcores[i];
>> + evd_lcore_install_handler(lcore, handler);
>> + }
>> +}
>> +
>> +int
>> +rte_dispatcher_register(uint8_t id,
>> + rte_dispatcher_match_t match_fun,
>> + void *match_data,
>> + rte_dispatcher_process_t process_fun,
>> + void *process_data)
>> +{
>> + struct rte_dispatcher *dispatcher;
>> + struct rte_dispatcher_handler handler = {
>> + .match_fun = match_fun,
>> + .match_data = match_data,
>
> We can have a default function which uses queue_id as matching data.
> This reduces the application load to provide two callbacks, one for matching and one for processing the event.
> Application can pass NULL parameter for "match_fun", in that case default function pointer can be used here.
>
But which queue id would this default function pointer match?
I think you need more API calls to allow for something like this. I've
discussed this kind of API in some previous messager on this list, if I
recall correctly.
>
>> + .process_fun = process_fun,
>> + .process_data = process_data
>> + };
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>> + handler.id = evd_alloc_handler_id(dispatcher);
>> +
>> + if (handler.id < 0)
>> + return -ENOMEM;
>> +
>> + evd_install_handler(dispatcher, &handler);
>> +
>> + return handler.id;
>> +}
>> +
>> +static int
>> +evd_lcore_uninstall_handler(struct rte_dispatcher_lcore *lcore,
>> + int handler_id)
>> +{
>> + struct rte_dispatcher_handler *unreg_handler;
>> + int handler_idx;
>> + uint16_t last_idx;
>> +
>> + unreg_handler = evd_lcore_get_handler_by_id(lcore, handler_id);
>> +
>> + if (unreg_handler == NULL)
>> + return -EINVAL;
>> +
>
> Shouldn't the logic be " handler_idx = unreg_handler - &lcore->handlers[0];"
> Because, unreg_handler will be a higher or equal address to the handler base address (&lcore->handlers[0])
>
True. Will fix.
>
>> + handler_idx = &lcore->handlers[0] - unreg_handler;
>> +
>> + last_idx = lcore->num_handlers - 1;
>> +
>> + if (handler_idx != last_idx) {
>> + /* move all handlers to maintain handler order */
>> + int n = last_idx - handler_idx;
>> + memmove(unreg_handler, unreg_handler + 1,
>> + sizeof(struct rte_dispatcher_handler) * n);
>> + }
>> +
>> + lcore->num_handlers--;
>> +
>> + return 0;
>> +}
>> +
>> +static int
>> +evd_uninstall_handler(struct rte_dispatcher *dispatcher,
>> + int handler_id)
>> +{
>> + unsigned int lcore_id;
>> +
>> + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
>> + struct rte_dispatcher_lcore *lcore =
>> + &dispatcher->lcores[lcore_id];
>> + int rc;
>> +
>> + rc = evd_lcore_uninstall_handler(lcore, handler_id);
>> +
>> + if (rc < 0)
>> + return rc;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +int
>> +rte_dispatcher_unregister(uint8_t id, int handler_id) {
>> + struct rte_dispatcher *dispatcher;
>> + int rc;
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>> + rc = evd_uninstall_handler(dispatcher, handler_id);
>> +
>> + return rc;
>> +}
>> +
>> +static struct rte_dispatcher_finalizer* evd_get_finalizer_by_id(struct
>> +rte_dispatcher *dispatcher,
>> + int handler_id)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < dispatcher->num_finalizers; i++) {
>> + struct rte_dispatcher_finalizer *finalizer =
>> + &dispatcher->finalizers[i];
>> +
>> + if (finalizer->id == handler_id)
>> + return finalizer;
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +static int
>> +evd_alloc_finalizer_id(struct rte_dispatcher *dispatcher) {
>> + int finalizer_id = 0;
>> +
>> + while (evd_get_finalizer_by_id(dispatcher, finalizer_id) != NULL)
>> + finalizer_id++;
>> +
>> + return finalizer_id;
>> +}
>> +
>> +static struct rte_dispatcher_finalizer * evd_alloc_finalizer(struct
>> +rte_dispatcher *dispatcher) {
>> + int finalizer_idx;
>> + struct rte_dispatcher_finalizer *finalizer;
>> +
>> + if (dispatcher->num_finalizers == EVD_MAX_FINALIZERS)
>> + return NULL;
>> +
>> + finalizer_idx = dispatcher->num_finalizers;
>> + finalizer = &dispatcher->finalizers[finalizer_idx];
>> +
>> + finalizer->id = evd_alloc_finalizer_id(dispatcher);
>> +
>> + dispatcher->num_finalizers++;
>> +
>> + return finalizer;
>> +}
>> +
>> +int
>> +rte_dispatcher_finalize_register(uint8_t id,
>> + rte_dispatcher_finalize_t finalize_fun,
>> + void *finalize_data)
>> +{
>> + struct rte_dispatcher *dispatcher;
>> + struct rte_dispatcher_finalizer *finalizer;
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>> + finalizer = evd_alloc_finalizer(dispatcher);
>> +
>> + if (finalizer == NULL)
>> + return -ENOMEM;
>> +
>> + finalizer->finalize_fun = finalize_fun;
>> + finalizer->finalize_data = finalize_data;
>> +
>> + return finalizer->id;
>> +}
>> +
>> +int
>> +rte_dispatcher_finalize_unregister(uint8_t id, int handler_id) {
>> + struct rte_dispatcher *dispatcher;
>> + struct rte_dispatcher_finalizer *unreg_finalizer;
>> + int finalizer_idx;
>> + uint16_t last_idx;
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>> + unreg_finalizer = evd_get_finalizer_by_id(dispatcher, handler_id);
>> +
>> + if (unreg_finalizer == NULL)
>> + return -EINVAL;
>> +
>
> Same as above comment in rte_dispatcher_unregister, base address needs to be subtracted from unreg_finalizer
>
Yes.
>
>> + finalizer_idx = &dispatcher->finalizers[0] - unreg_finalizer;
>> +
>> + last_idx = dispatcher->num_finalizers - 1;
>> +
>> + if (finalizer_idx != last_idx) {
>> + /* move all finalizers to maintain order */
>> + int n = last_idx - finalizer_idx;
>> + memmove(unreg_finalizer, unreg_finalizer + 1,
>> + sizeof(struct rte_dispatcher_finalizer) * n);
>> + }
>> +
>> + dispatcher->num_finalizers--;
>> +
>> + return 0;
>> +}
>> +
>> +static int
>> +evd_set_service_runstate(uint8_t id, int state) {
>> + struct rte_dispatcher *dispatcher;
>> + int rc;
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>> + rc = rte_service_component_runstate_set(dispatcher->service_id,
>> + state);
>> +
>> + if (rc != 0) {
>> + RTE_EDEV_LOG_ERR("Unexpected error %d occurred while
>> setting "
>> + "service component run state to %d\n", rc,
>> + state);
>> + RTE_ASSERT(0);
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +int
>> +rte_dispatcher_start(uint8_t id)
>> +{
>> + return evd_set_service_runstate(id, 1); }
>> +
>> +int
>> +rte_dispatcher_stop(uint8_t id)
>> +{
>> + return evd_set_service_runstate(id, 0); }
>> +
>> +static void
>> +evd_aggregate_stats(struct rte_dispatcher_stats *result,
>> + const struct rte_dispatcher_stats *part) {
>> + result->poll_count += part->poll_count;
>> + result->ev_batch_count += part->ev_batch_count;
>> + result->ev_dispatch_count += part->ev_dispatch_count;
>> + result->ev_drop_count += part->ev_drop_count; }
>> +
>> +int
>> +rte_dispatcher_stats_get(uint8_t id,
>> + struct rte_dispatcher_stats *stats) {
>> + struct rte_dispatcher *dispatcher;
>> + unsigned int lcore_id;
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>
> Stats pointer needs to be validated for NULL before accessing
>
>
Yes.
Thanks a lot for your review comments.
>> + *stats = (struct rte_dispatcher_stats) {};
>> +
>> + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
>> + struct rte_dispatcher_lcore *lcore =
>> + &dispatcher->lcores[lcore_id];
>> +
>> + evd_aggregate_stats(stats, &lcore->stats);
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +int
>> +rte_dispatcher_stats_reset(uint8_t id)
>> +{
>> + struct rte_dispatcher *dispatcher;
>> + unsigned int lcore_id;
>> +
>> + EVD_VALID_ID_OR_RET_EINVAL(id);
>> + dispatcher = evd_get_dispatcher(id);
>> +
>> +
>> + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
>> + struct rte_dispatcher_lcore *lcore =
>> + &dispatcher->lcores[lcore_id];
>> +
>> + lcore->stats = (struct rte_dispatcher_stats) {};
>> + }
>> +
>> + return 0;
>> +
>> +}
>> diff --git a/lib/dispatcher/rte_dispatcher.h b/lib/dispatcher/rte_dispatcher.h
>> new file mode 100644 index 0000000000..6712687a08
>> --- /dev/null
>> +++ b/lib/dispatcher/rte_dispatcher.h
>> @@ -0,0 +1,480 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2023 Ericsson AB
>> + */
>> +
>> +#ifndef __RTE_DISPATCHER_H__
>> +#define __RTE_DISPATCHER_H__
>> +
>> +/**
>> + * @file
>> + *
>> + * RTE Dispatcher
>> + *
>> + * The purpose of the dispatcher is to help decouple different parts
>> + * of an application (e.g., modules), sharing the same underlying
>> + * event device.
>> + */
>> +
>> +#ifdef __cplusplus
>> +extern "C" {
>> +#endif
>> +
>> +#include <rte_eventdev.h>
>> +
>> +/**
>> + * Function prototype for match callbacks.
>> + *
>> + * Match callbacks are used by an application to decide how the
>> + * dispatcher distributes events to different parts of the
>> + * application.
>> + *
>> + * The application is not expected to process the event at the point
>> + * of the match call. Such matters should be deferred to the process
>> + * callback invocation.
>> + *
>> + * The match callback may be used as an opportunity to prefetch data.
>> + *
>> + * @param event
>> + * Pointer to event
>> + *
>> + * @param cb_data
>> + * The pointer supplied by the application in
>> + * rte_dispatcher_register().
>> + *
>> + * @return
>> + * Returns true in case this events should be delivered (via
>> + * the process callback), and false otherwise.
>> + */
>> +typedef bool
>> +(*rte_dispatcher_match_t)(const struct rte_event *event, void
>> +*cb_data);
>> +
>> +/**
>> + * Function prototype for process callbacks.
>> + *
>> + * The process callbacks are used by the dispatcher to deliver
>> + * events for processing.
>> + *
>> + * @param event_dev_id
>> + * The originating event device id.
>> + *
>> + * @param event_port_id
>> + * The originating event port.
>> + *
>> + * @param events
>> + * Pointer to an array of events.
>> + *
>> + * @param num
>> + * The number of events in the @p events array.
>> + *
>> + * @param cb_data
>> + * The pointer supplied by the application in
>> + * rte_dispatcher_register().
>> + */
>> +
>> +typedef void
>> +(*rte_dispatcher_process_t)(uint8_t event_dev_id, uint8_t event_port_id,
>> + struct rte_event *events, uint16_t num,
>> + void *cb_data);
>> +
>> +/**
>> + * Function prototype for finalize callbacks.
>> + *
>> + * The finalize callbacks are used by the dispatcher to notify the
>> + * application it has delivered all events from a particular batch
>> + * dequeued from the event device.
>> + *
>> + * @param event_dev_id
>> + * The originating event device id.
>> + *
>> + * @param event_port_id
>> + * The originating event port.
>> + *
>> + * @param cb_data
>> + * The pointer supplied by the application in
>> + * rte_dispatcher_finalize_register().
>> + */
>> +
>> +typedef void
>> +(*rte_dispatcher_finalize_t)(uint8_t event_dev_id, uint8_t event_port_id,
>> + void *cb_data);
>> +
>> +/**
>> + * Dispatcher statistics
>> + */
>> +struct rte_dispatcher_stats {
>> + uint64_t poll_count;
>> + /**< Number of event dequeue calls made toward the event device. */
>> + uint64_t ev_batch_count;
>> + /**< Number of non-empty event batches dequeued from event
>> device.*/
>> + uint64_t ev_dispatch_count;
>> + /**< Number of events dispatched to a handler.*/
>> + uint64_t ev_drop_count;
>> + /**< Number of events dropped because no handler was found. */ };
>> +
>> +/**
>> + * Create a dispatcher with the specified id.
>> + *
>> + * @param id
>> + * An application-specified, unique (across all dispatcher
>> + * instances) identifier.
>> + *
>> + * @param event_dev_id
>> + * The identifier of the event device from which this dispatcher
>> + * will dequeue events.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - <0: Error code on failure
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_create(uint8_t id, uint8_t event_dev_id);
>> +
>> +/**
>> + * Free a dispatcher.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - <0: Error code on failure
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_free(uint8_t id);
>> +
>> +/**
>> + * Retrieve the service identifier of a dispatcher.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @param [out] service_id
>> + * A pointer to a caller-supplied buffer where the dispatcher's
>> + * service id will be stored.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - <0: Error code on failure.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_service_id_get(uint8_t id, uint32_t *service_id);
>> +
>> +/**
>> + * Binds an event device port to a specific lcore on the specified
>> + * dispatcher.
>> + *
>> + * This function configures the event port id to be used by the event
>> + * dispatcher service, if run on the specified lcore.
>> + *
>> + * Multiple event device ports may be bound to the same lcore. A
>> + * particular port must not be bound to more than one lcore.
>> + *
>> + * If the dispatcher service is mapped (with
>> +rte_service_map_lcore_set())
>> + * to a lcore to which no ports are bound, the service function will be
>> +a
>> + * no-operation.
>> + *
>> + * This function may be called by any thread (including unregistered
>> + * non-EAL threads), but not while the dispatcher is running on lcore
>> + * specified by @c lcore_id.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @param event_port_id
>> + * The event device port identifier.
>> + *
>> + * @param batch_size
>> + * The batch size to use in rte_event_dequeue_burst(), for the
>> + * configured event device port and lcore.
>> + *
>> + * @param timeout
>> + * The timeout parameter to use in rte_event_dequeue_burst(), for the
>> + * configured event device port and lcore.
>> + *
>> + * @param lcore_id
>> + * The lcore by which this event port will be used.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - -ENOMEM: Unable to allocate sufficient resources.
>> + * - -EEXISTS: Event port is already configured.
>> + * - -EINVAL: Invalid arguments.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_bind_port_to_lcore(uint8_t id, uint8_t event_port_id,
>> + uint16_t batch_size, uint64_t
>> timeout,
>> + unsigned int lcore_id);
>> +
>> +/**
>> + * Unbind an event device port from a specific lcore.
>> + *
>> + * This function may be called by any thread (including unregistered
>> + * non-EAL threads), but not while the dispatcher is running on
>> + * lcore specified by @c lcore_id.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @param event_port_id
>> + * The event device port identifier.
>> + *
>> + * @param lcore_id
>> + * The lcore which was using this event port.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - -EINVAL: Invalid @c id.
>> + * - -ENOENT: Event port id not bound to this @c lcore_id.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_unbind_port_from_lcore(uint8_t id, uint8_t event_port_id,
>> + unsigned int lcore_id);
>> +
>> +/**
>> + * Register an event handler.
>> + *
>> + * The match callback function is used to select if a particular event
>> + * should be delivered, using the corresponding process callback
>> + * function.
>> + *
>> + * The reason for having two distinct steps is to allow the dispatcher
>> + * to deliver all events as a batch. This in turn will cause
>> + * processing of a particular kind of events to happen in a
>> + * back-to-back manner, improving cache locality.
>> + *
>> + * The list of handler callback functions is shared among all lcores,
>> + * but will only be executed on lcores which has an eventdev port
>> + * bound to them, and which are running the dispatcher service.
>> + *
>> + * An event is delivered to at most one handler. Events where no
>> + * handler is found are dropped.
>> + *
>> + * The application must not depend on the order of which the match
>> + * functions are invoked.
>> + *
>> + * Ordering of events is not guaranteed to be maintained between
>> + * different deliver callbacks. For example, suppose there are two
>> + * callbacks registered, matching different subsets of events arriving
>> + * on an atomic queue. A batch of events [ev0, ev1, ev2] are dequeued
>> + * on a particular port, all pertaining to the same flow. The match
>> + * callback for registration A returns true for ev0 and ev2, and the
>> + * matching function for registration B for ev1. In that scenario, the
>> + * dispatcher may choose to deliver first [ev0, ev2] using A's deliver
>> + * function, and then [ev1] to B - or vice versa.
>> + *
>> + * rte_dispatcher_register() may be called by any thread
>> + * (including unregistered non-EAL threads), but not while the event
>> + * dispatcher is running on any service lcore.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @param match_fun
>> + * The match callback function.
>> + *
>> + * @param match_cb_data
>> + * A pointer to some application-specific opaque data (or NULL),
>> + * which is supplied back to the application when match_fun is
>> + * called.
>> + *
>> + * @param process_fun
>> + * The process callback function.
>> + *
>> + * @param process_cb_data
>> + * A pointer to some application-specific opaque data (or NULL),
>> + * which is supplied back to the application when process_fun is
>> + * called.
>> + *
>> + * @return
>> + * - >= 0: The identifier for this registration.
>> + * - -ENOMEM: Unable to allocate sufficient resources.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_register(uint8_t id,
>> + rte_dispatcher_match_t match_fun,
>> + void *match_cb_data,
>> + rte_dispatcher_process_t process_fun,
>> + void *process_cb_data);
>> +
>> +/**
>> + * Unregister an event handler.
>> + *
>> + * This function may be called by any thread (including unregistered
>> + * non-EAL threads), but not while the dispatcher is running on
>> + * any service lcore.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @param handler_id
>> + * The handler registration id returned by the original
>> + * rte_dispatcher_register() call.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - -EINVAL: The @c id and/or the @c handler_id parameter was invalid.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_unregister(uint8_t id, int handler_id);
>> +
>> +/**
>> + * Register a finalize callback function.
>> + *
>> + * An application may optionally install one or more finalize
>> + * callbacks.
>> + *
>> + * All finalize callbacks are invoked by the dispatcher when a
>> + * complete batch of events (retrieve using rte_event_dequeue_burst())
>> + * have been delivered to the application (or have been dropped).
>> + *
>> + * The finalize callback is not tied to any particular handler.
>> + *
>> + * The finalize callback provides an opportunity for the application
>> + * to do per-batch processing. One case where this may be useful is if
>> + * an event output buffer is used, and is shared among several
>> + * handlers. In such a case, proper output buffer flushing may be
>> + * assured using a finalize callback.
>> + *
>> + * rte_dispatcher_finalize_register() may be called by any thread
>> + * (including unregistered non-EAL threads), but not while the
>> + * dispatcher is running on any service lcore.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @param finalize_fun
>> + * The function called after completing the processing of a
>> + * dequeue batch.
>> + *
>> + * @param finalize_data
>> + * A pointer to some application-specific opaque data (or NULL),
>> + * which is supplied back to the application when @c finalize_fun is
>> + * called.
>> + *
>> + * @return
>> + * - >= 0: The identifier for this registration.
>> + * - -ENOMEM: Unable to allocate sufficient resources.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_finalize_register(uint8_t id,
>> + rte_dispatcher_finalize_t finalize_fun,
>> + void *finalize_data);
>> +
>> +/**
>> + * Unregister a finalize callback.
>> + *
>> + * This function may be called by any thread (including unregistered
>> + * non-EAL threads), but not while the dispatcher is running on
>> + * any service lcore.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @param reg_id
>> + * The finalize registration id returned by the original
>> + * rte_dispatcher_finalize_register() call.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - -EINVAL: The @c id and/or the @c reg_id parameter was invalid.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_finalize_unregister(uint8_t id, int reg_id);
>> +
>> +/**
>> + * Start a dispatcher instance.
>> + *
>> + * Enables the dispatcher service.
>> + *
>> + * The underlying event device must have been started prior to calling
>> + * rte_dispatcher_start().
>> + *
>> + * For the dispatcher to actually perform work (i.e., dispatch
>> + * events), its service must have been mapped to one or more service
>> + * lcores, and its service run state set to '1'. A dispatcher's
>> + * service is retrieved using rte_dispatcher_service_id_get().
>> + *
>> + * Each service lcore to which the dispatcher is mapped should
>> + * have at least one event port configured. Such configuration is
>> + * performed by calling rte_dispatcher_bind_port_to_lcore(), prior to
>> + * starting the dispatcher.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - -EINVAL: Invalid @c id.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_start(uint8_t id);
>> +
>> +/**
>> + * Stop a running dispatcher instance.
>> + *
>> + * Disables the dispatcher service.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - -EINVAL: Invalid @c id.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_stop(uint8_t id);
>> +
>> +/**
>> + * Retrieve statistics for a dispatcher instance.
>> + *
>> + * This function is MT safe and may be called by any thread
>> + * (including unregistered non-EAL threads).
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + * @param[out] stats
>> + * A pointer to a structure to fill with statistics.
>> + * @return
>> + * - 0: Success
>> + * - -EINVAL: The @c id parameter was invalid.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_stats_get(uint8_t id,
>> + struct rte_dispatcher_stats *stats);
>> +
>> +/**
>> + * Reset statistics for a dispatcher instance.
>> + *
>> + * This function may be called by any thread (including unregistered
>> + * non-EAL threads), but may not produce the correct result if the
>> + * dispatcher is running on any service lcore.
>> + *
>> + * @param id
>> + * The dispatcher identifier.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - -EINVAL: The @c id parameter was invalid.
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_stats_reset(uint8_t id);
>> +
>> +#ifdef __cplusplus
>> +}
>> +#endif
>> +
>> +#endif /* __RTE_DISPATCHER__ */
>> diff --git a/lib/dispatcher/version.map b/lib/dispatcher/version.map new file
>> mode 100644 index 0000000000..8f9ad96522
>> --- /dev/null
>> +++ b/lib/dispatcher/version.map
>> @@ -0,0 +1,20 @@
>> +EXPERIMENTAL {
>> + global:
>> +
>> + # added in 23.11
>> + rte_dispatcher_create;
>> + rte_dispatcher_free;
>> + rte_dispatcher_service_id_get;
>> + rte_dispatcher_bind_port_to_lcore;
>> + rte_dispatcher_unbind_port_from_lcore;
>> + rte_dispatcher_register;
>> + rte_dispatcher_unregister;
>> + rte_dispatcher_finalize_register;
>> + rte_dispatcher_finalize_unregister;
>> + rte_dispatcher_start;
>> + rte_dispatcher_stop;
>> + rte_dispatcher_stats_get;
>> + rte_dispatcher_stats_reset;
>> +
>> + local: *;
>> +};
>> diff --git a/lib/meson.build b/lib/meson.build index
>> 099b0ed18a..3093b338d2 100644
>> --- a/lib/meson.build
>> +++ b/lib/meson.build
>> @@ -35,6 +35,7 @@ libraries = [
>> 'distributor',
>> 'efd',
>> 'eventdev',
>> + 'dispatcher', # dispatcher depends on eventdev
>> 'gpudev',
>> 'gro',
>> 'gso',
>> @@ -81,6 +82,7 @@ optional_libs = [
>> 'cfgfile',
>> 'compressdev',
>> 'cryptodev',
>> + 'dispatcher',
>> 'distributor',
>> 'dmadev',
>> 'efd',
>> --
>> 2.34.1
>
On Mon, Sep 4, 2023 at 6:39 PM Mattias Rönnblom
<mattias.ronnblom@ericsson.com> wrote:
>
> The purpose of the dispatcher library is to help reduce coupling in an
> Eventdev-based DPDK application.
>
> In addition, the dispatcher also provides a convenient and flexible
> way for the application to use service cores for application-level
> processing.
>
> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
High level architecture comment
--------------------------------
1) I think, we don't need tie this library ONLY to event dev
application. It can be used with poll mode as well,
that way traditiona pipeline application with ethdev as source could
use this library dispatch the packets.
We dont need to implement that first version but API can make room for
such abstractions.
Based on my understanding in fast-path it has means to
a)Pull out the events using rte_event_dequeue()
b)Compare with registered match functions and call process upon match.
if we abstract (a) as rte_dispatcher_source, We could pull from ethdev
via rte_eth_rx_burst() or
from ring via dequeue_burst API or so based on rte_dispatcher_source
selected for dispatch configuration
and we can use different sevice function pointers to have different service core
implementation without effecting performance each sources.
High level cosmetic comment
----------------------------------------------------
1)Missing doxygen connection- See doc/api/doxy-api-index.md
Process related comment
------------------------------------
1) Documentation does not need need separate patch. All recent library
changes documentation in same file.
You could have doc and API header file as first patch and
implementation as subsequent patches.
> diff --git a/lib/dispatcher/rte_dispatcher.h b/lib/dispatcher/rte_dispatcher.h
> new file mode 100644
> index 0000000000..6712687a08
> --- /dev/null
> +++ b/lib/dispatcher/rte_dispatcher.h
> @@ -0,0 +1,480 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2023 Ericsson AB
> + */
> +
> +#ifndef __RTE_DISPATCHER_H__
> +#define __RTE_DISPATCHER_H__
> +
All new API should be experimental. See
https://elixir.bootlin.com/dpdk/latest/source/lib/graph/rte_graph.h#L12
example.
> +/**
> + * @file
> + *
> + * RTE Dispatcher
> + *
> + * The purpose of the dispatcher is to help decouple different parts
> + * of an application (e.g., modules), sharing the same underlying
> + * event device.
> +
> +/**
> + * Function prototype for match callbacks.
> + *
> + * Match callbacks are used by an application to decide how the
> + * dispatcher distributes events to different parts of the
> + * application.
> + *
> + * The application is not expected to process the event at the point
> + * of the match call. Such matters should be deferred to the process
> + * callback invocation.
> + *
> + * The match callback may be used as an opportunity to prefetch data.
> + *
> + * @param event
> + * Pointer to event
> + *
> + * @param cb_data
> + * The pointer supplied by the application in
> + * rte_dispatcher_register().
> + *
> + * @return
> + * Returns true in case this events should be delivered (via
> + * the process callback), and false otherwise.
> + */
> +typedef bool
> +(*rte_dispatcher_match_t)(const struct rte_event *event, void *cb_data);
a) Can we use void* event, so that it can be used with mbuf or other
type by casting in the call back implementer.
b) I was thinking, How we can avoid this function pointer and enable
more have better performance at architecture level.
Both x86, ARM has vector instructions[1] to form a vector from various
offset from memory and compare N events
in one shot. That is, if express match data like offset = X has value
is Y and offset = X has value = A.
I know, it may not good existing application using this APIs. But I
believe, it will be more performance
effective. If make sense, you can adapt to this.(Something to think about)
[1]
https://developer.arm.com/documentation/den0018/a/NEON-and-VFP-Instruction-Summary/NEON-general-data-processing-instructions/VTBL
> +
> +/**
> + * Function prototype for process callbacks.
> + *
> + * The process callbacks are used by the dispatcher to deliver
> + * events for processing.
> + *
> + * @param event_dev_id
> + * The originating event device id.
> + *
> + * @param event_port_id
> + * The originating event port.
> + *
> + * @param events
> + * Pointer to an array of events.
> + *
> + * @param num
> + * The number of events in the @p events array.
> + *
> + * @param cb_data
> + * The pointer supplied by the application in
> + * rte_dispatcher_register().
> + */
> +
> +typedef void
> +(*rte_dispatcher_process_t)(uint8_t event_dev_id, uint8_t event_port_id,
> + struct rte_event *events, uint16_t num,
> + void *cb_data);
Same as above comment, can event_port_id can be change to source_id?
> +/**
> + * Create a dispatcher with the specified id.
> + *
> + * @param id
> + * An application-specified, unique (across all dispatcher
> + * instances) identifier.
> + *
> + * @param event_dev_id
> + * The identifier of the event device from which this dispatcher
> + * will dequeue events.
> + *
> + * @return
> + * - 0: Success
> + * - <0: Error code on failure
> + */
> +__rte_experimental
> +int
> +rte_dispatcher_create(uint8_t id, uint8_t event_dev_id);
Following could be used to abstract more dispatcher sources, like
enum rte_dispatcher_source {
RTE_DISPATCHER_SOURCE_EVENTDEV, // Use rte_event_dequeue() to
pull the packet
RTE_DISPATCHER_SOURCE_ETHDEV, // Use rte_ethdev_rx_burst() to
pull the packet
};
struct rte_dispatcher_params {
enum rte_dispatcher_source source;
union {
/* Valid when source == RTE_DISPATCHER_SOURCE_EVENTDEV */
struct event_source {
uint8_t event_dev_id;
uin8_t event_port_id;
};
/* Valid when source == RTE_DISPATCHER_SOURCE_ETHDEV*/
struct ethdev_source {
uint16_t ethdev__dev_id;
uin16_t ethdev_rx_queue_id;
};
}
};
rte_dispatcher_create(uint8_t id, struct rte_dispatcher_params *parms);
I will stop reviewing at this point. Will review based on direction agree on.
> -----Original Message-----
> From: Mattias Rönnblom <hofors@lysator.liu.se>
> Sent: Tuesday, September 19, 2023 2:51 PM
> To: Naga Harish K, S V <s.v.naga.harish.k@intel.com>; mattias.ronnblom
> <mattias.ronnblom@ericsson.com>; dev@dpdk.org
> Cc: Jerin Jacob <jerinj@marvell.com>; techboard@dpdk.org; Van Haaren,
> Harry <harry.van.haaren@intel.com>; Nilsson, Peter
> <peter.j.nilsson@ericsson.com>; Heng Wang <heng.wang@ericsson.com>;
> Pavan Nikhilesh <pbhagavatula@marvell.com>; Gujjar, Abhinandan S
> <abhinandan.gujjar@intel.com>; Carrillo, Erik G <erik.g.carrillo@intel.com>;
> Shijith Thotton <sthotton@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
> Liang Ma <liangma@liangbit.com>; Mccarthy, Peter
> <peter.mccarthy@intel.com>; Yan, Zhirun <zhirun.yan@intel.com>
> Subject: Re: [PATCH v3 1/3] lib: introduce dispatcher library
>
> On 2023-09-17 18:46, Naga Harish K, S V wrote:
> >
> >
> >> -----Original Message-----
> >> From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> >> Sent: Monday, September 4, 2023 6:33 PM
> >> To: dev@dpdk.org
> >> Cc: Jerin Jacob <jerinj@marvell.com>; techboard@dpdk.org; Van Haaren,
> >> Harry <harry.van.haaren@intel.com>; hofors@lysator.liu.se; Nilsson,
> >> Peter <peter.j.nilsson@ericsson.com>; Heng Wang
> >> <heng.wang@ericsson.com>; Naga Harish K, S V
> >> <s.v.naga.harish.k@intel.com>; Pavan Nikhilesh
> >> <pbhagavatula@marvell.com>; Gujjar, Abhinandan S
> >> <abhinandan.gujjar@intel.com>; Carrillo, Erik G
> >> <Erik.G.Carrillo@intel.com>; Shijith Thotton <sthotton@marvell.com>;
> >> Hemant Agrawal <hemant.agrawal@nxp.com>; Sachin Saxena
> >> <sachin.saxena@oss.nxp.com>; Liang Ma <liangma@liangbit.com>;
> >> Mccarthy, Peter <Peter.Mccarthy@intel.com>; Yan, Zhirun
> >> <Zhirun.Yan@intel.com>; mattias.ronnblom
> >> <mattias.ronnblom@ericsson.com>
> >> Subject: [PATCH v3 1/3] lib: introduce dispatcher library
> >>
> >> The purpose of the dispatcher library is to help reduce coupling in
> >> an Eventdev-based DPDK application.
> >>
> >> In addition, the dispatcher also provides a convenient and flexible
> >> way for the application to use service cores for application-level processing.
> >>
> >> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> >> Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
> >> Reviewed-by: Heng Wang <heng.wang@ericsson.com>
> >>
> >> --
> >>
> >> PATCH v3:
> >> o To underline its optional character and since it does not provide
> >> hardware abstraction, the event dispatcher is now a separate
> >> library.
> >> o Change name from rte_event_dispatcher -> rte_dispatcher, to make it
> >> shorter and to avoid the rte_event_* namespace.
> >>
> >
> > Rte_dispatcher is basically dispatching events but it feels like the name does
> not convey that.
> > Also, it is like any other adapter service that can reside within the eventdev
> directory.
> >
> > I can see some discussion in previous threads related to the placement of the
> dispatcher library.
> >
> > It is an optional eventdev application service, not enforcing this
> programming model to the application.
> > The documentation may need to be updated and mention that this is
> optional.
> >
> > If any hardware comes up with the dispatcher feature, then this library may
> need to be moved inside eventdev library later.
> >
>
> It seems to me that the deciding factor for what functionality goes into a DPDK
> library or not is not such much dependent on if it's implemented in hardware,
> in software, or some combination thereof. The important thing is that the
> library is be able to present a coherent API to the application (or other
> libraries).
>
> That said, as I've mentioned before, I have no strong opionion on this subject.
>
What is the next step here?
The response is not conclusive as It looks like both yes and no to change the directory structure.
> > So, It makes sense to keep this optional service in the eventdev folder as an
> optional feature.
> >
> >> PATCH v2:
> >> o Add dequeue batch count statistic.
> >> o Add statistics reset function to API.
> >> o Clarify MT safety guarantees (or lack thereof) in the API documentation.
> >> o Change loop variable type in evd_lcore_get_handler_by_id() to uint16_t,
> >> to be consistent with similar loops elsewhere in the dispatcher.
> >> o Fix variable names in finalizer unregister function.
> >>
> >> PATCH:
> >> o Change prefix from RED to EVD, to avoid confusion with random
> >> early detection.
> >>
> >> RFC v4:
> >> o Move handlers to per-lcore data structures.
> >> o Introduce mechanism which rearranges handlers so that often-used
> >> handlers tend to be tried first.
> >> o Terminate dispatch loop in case all events are delivered.
> >> o To avoid the dispatcher's service function hogging the CPU, process
> >> only one batch per call.
> >> o Have service function return -EAGAIN if no work is performed.
> >> o Events delivered in the process function is no longer marked 'const',
> >> since modifying them may be useful for the application and cause
> >> no difficulties for the dispatcher.
> >> o Various minor API documentation improvements.
> >>
> >> RFC v3:
> >> o Add stats_get() function to the version.map file.
> >> ---
> >> MAINTAINERS | 3 +
> >> lib/dispatcher/meson.build | 17 +
> >> lib/dispatcher/rte_dispatcher.c | 791
> >> ++++++++++++++++++++++++++++++++ lib/dispatcher/rte_dispatcher.h
> |
> >> 480 +++++++++++++++++++
> >> lib/dispatcher/version.map | 20 +
> >> lib/meson.build | 2 +
> >> 6 files changed, 1313 insertions(+)
> >> create mode 100644 lib/dispatcher/meson.build create mode 100644
> >> lib/dispatcher/rte_dispatcher.c create mode 100644
> >> lib/dispatcher/rte_dispatcher.h create mode 100644
> >> lib/dispatcher/version.map
> >>
> >> diff --git a/MAINTAINERS b/MAINTAINERS index a926155f26..6704cd5b2c
> >> 100644
> >> --- a/MAINTAINERS
> >> +++ b/MAINTAINERS
> >> @@ -1726,6 +1726,9 @@ M: Nithin Dabilpuram
> <ndabilpuram@marvell.com>
> >> M: Pavan Nikhilesh <pbhagavatula@marvell.com>
> >> F: lib/node/
> >>
> >> +Dispatcher - EXPERIMENTAL
> >> +M: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> >> +F: lib/dispatcher/
> >>
> >> Test Applications
> >> -----------------
> >> diff --git a/lib/dispatcher/meson.build b/lib/dispatcher/meson.build
> >> new file mode 100644 index 0000000000..c6054a3a5d
> >> --- /dev/null
> >> +++ b/lib/dispatcher/meson.build
> >> @@ -0,0 +1,17 @@
> >> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 Ericsson
> >> +AB
> >> +
> >> +if is_windows
> >> + build = false
> >> + reason = 'not supported on Windows'
> >> + subdir_done()
> >> +endif
> >> +
> >> +sources = files(
> >> + 'rte_dispatcher.c',
> >> +)
> >> +headers = files(
> >> + 'rte_dispatcher.h',
> >> +)
> >> +
> >> +deps += ['eventdev']
> >> diff --git a/lib/dispatcher/rte_dispatcher.c
> >> b/lib/dispatcher/rte_dispatcher.c new file mode 100644 index
> >> 0000000000..3319fe09f2
> >> --- /dev/null
> >> +++ b/lib/dispatcher/rte_dispatcher.c
> >> @@ -0,0 +1,791 @@
> >> +/* SPDX-License-Identifier: BSD-3-Clause
> >> + * Copyright(c) 2023 Ericsson AB
> >> + */
> >> +
> >> +#include <stdbool.h>
> >> +#include <stdint.h>
> >> +
> >> +#include <rte_branch_prediction.h>
> >> +#include <rte_common.h>
> >> +#include <rte_lcore.h>
> >> +#include <rte_random.h>
> >> +#include <rte_service_component.h>
> >> +
> >> +#include "eventdev_pmd.h"
> >> +
> >> +#include <rte_dispatcher.h>
> >> +
> >> +#define EVD_MAX_PORTS_PER_LCORE 4
> >> +#define EVD_MAX_HANDLERS 32
> >> +#define EVD_MAX_FINALIZERS 16
> >> +#define EVD_AVG_PRIO_INTERVAL 2000
> >> +
> >> +struct rte_dispatcher_lcore_port {
> >> + uint8_t port_id;
> >> + uint16_t batch_size;
> >> + uint64_t timeout;
> >> +};
> >> +
> >> +struct rte_dispatcher_handler {
> >> + int id;
> >> + rte_dispatcher_match_t match_fun;
> >> + void *match_data;
> >> + rte_dispatcher_process_t process_fun;
> >> + void *process_data;
> >> +};
> >> +
> >> +struct rte_dispatcher_finalizer {
> >> + int id;
> >> + rte_dispatcher_finalize_t finalize_fun;
> >> + void *finalize_data;
> >> +};
> >> +
> >> +struct rte_dispatcher_lcore {
> >> + uint8_t num_ports;
> >> + uint16_t num_handlers;
> >> + int32_t prio_count;
> >> + struct rte_dispatcher_lcore_port
> >> ports[EVD_MAX_PORTS_PER_LCORE];
> >> + struct rte_dispatcher_handler handlers[EVD_MAX_HANDLERS];
> >> + struct rte_dispatcher_stats stats;
> >> +} __rte_cache_aligned;
> >> +
> >> +struct rte_dispatcher {
> >> + uint8_t id;
> >> + uint8_t event_dev_id;
> >> + int socket_id;
> >> + uint32_t service_id;
> >> + struct rte_dispatcher_lcore lcores[RTE_MAX_LCORE];
> >> + uint16_t num_finalizers;
> >> + struct rte_dispatcher_finalizer finalizers[EVD_MAX_FINALIZERS]; };
> >> +
> >> +static struct rte_dispatcher *dispatchers[UINT8_MAX];
> >> +
> >> +static bool
> >> +evd_has_dispatcher(uint8_t id)
> >> +{
> >> + return dispatchers[id] != NULL;
> >> +}
> >> +
> >> +static struct rte_dispatcher *
> >> +evd_get_dispatcher(uint8_t id)
> >> +{
> >> + return dispatchers[id];
> >> +}
> >> +
> >> +static void
> >> +evd_set_dispatcher(uint8_t id, struct rte_dispatcher *dispatcher) {
> >> + dispatchers[id] = dispatcher;
> >> +}
> >> +
> >> +#define EVD_VALID_ID_OR_RET_EINVAL(id)
> >> \
> >> + do { \
> >> + if (unlikely(!evd_has_dispatcher(id))) { \
> >> + RTE_EDEV_LOG_ERR("Invalid dispatcher id %d\n", id);
> >> \
> >> + return -EINVAL; \
> >> + } \
> >> + } while (0)
> >> +
> >> +static int
> >> +evd_lookup_handler_idx(struct rte_dispatcher_lcore *lcore,
> >> + const struct rte_event *event) {
> >> + uint16_t i;
> >> +
> >> + for (i = 0; i < lcore->num_handlers; i++) {
> >> + struct rte_dispatcher_handler *handler =
> >> + &lcore->handlers[i];
> >> +
> >> + if (handler->match_fun(event, handler->match_data))
> >> + return i;
> >> + }
> >> +
> >> + return -1;
> >> +}
> >> +
> >> +static void
> >> +evd_prioritize_handler(struct rte_dispatcher_lcore *lcore,
> >> + int handler_idx)
> >> +{
> >> + struct rte_dispatcher_handler tmp;
> >> +
> >> + if (handler_idx == 0)
> >> + return;
> >> +
> >> + /* Let the lucky handler "bubble" up the list */
> >> +
> >> + tmp = lcore->handlers[handler_idx - 1];
> >> +
> >> + lcore->handlers[handler_idx - 1] = lcore->handlers[handler_idx];
> >> +
> >> + lcore->handlers[handler_idx] = tmp; }
> >> +
> >> +static inline void
> >> +evd_consider_prioritize_handler(struct rte_dispatcher_lcore *lcore,
> >> + int handler_idx, uint16_t handler_events) {
> >> + lcore->prio_count -= handler_events;
> >> +
> >> + if (unlikely(lcore->prio_count <= 0)) {
> >> + evd_prioritize_handler(lcore, handler_idx);
> >> +
> >> + /*
> >> + * Randomize the interval in the unlikely case
> >> + * the traffic follow some very strict pattern.
> >> + */
> >> + lcore->prio_count =
> >> + rte_rand_max(EVD_AVG_PRIO_INTERVAL) +
> >> + EVD_AVG_PRIO_INTERVAL / 2;
> >> + }
> >> +}
> >> +
> >> +static inline void
> >> +evd_dispatch_events(struct rte_dispatcher *dispatcher,
> >> + struct rte_dispatcher_lcore *lcore,
> >> + struct rte_dispatcher_lcore_port *port,
> >> + struct rte_event *events, uint16_t num_events) {
> >> + int i;
> >> + struct rte_event bursts[EVD_MAX_HANDLERS][num_events];
> >> + uint16_t burst_lens[EVD_MAX_HANDLERS] = { 0 };
> >> + uint16_t drop_count = 0;
> >> + uint16_t dispatch_count;
> >> + uint16_t dispatched = 0;
> >> +
> >> + for (i = 0; i < num_events; i++) {
> >> + struct rte_event *event = &events[i];
> >> + int handler_idx;
> >> +
> >> + handler_idx = evd_lookup_handler_idx(lcore, event);
> >> +
> >> + if (unlikely(handler_idx < 0)) {
> >> + drop_count++;
> >> + continue;
> >> + }
> >> +
> >> + bursts[handler_idx][burst_lens[handler_idx]] = *event;
> >> + burst_lens[handler_idx]++;
> >> + }
> >> +
> >> + dispatch_count = num_events - drop_count;
> >> +
> >> + for (i = 0; i < lcore->num_handlers &&
> >> + dispatched < dispatch_count; i++) {
> >> + struct rte_dispatcher_handler *handler =
> >> + &lcore->handlers[i];
> >> + uint16_t len = burst_lens[i];
> >> +
> >> + if (len == 0)
> >> + continue;
> >> +
> >> + handler->process_fun(dispatcher->event_dev_id, port-
> >>> port_id,
> >> + bursts[i], len, handler->process_data);
> >> +
> >> + dispatched += len;
> >> +
> >> + /*
> >> + * Safe, since any reshuffling will only involve
> >> + * already-processed handlers.
> >> + */
> >> + evd_consider_prioritize_handler(lcore, i, len);
> >> + }
> >> +
> >> + lcore->stats.ev_batch_count++;
> >> + lcore->stats.ev_dispatch_count += dispatch_count;
> >> + lcore->stats.ev_drop_count += drop_count;
> >> +
> >> + for (i = 0; i < dispatcher->num_finalizers; i++) {
> >> + struct rte_dispatcher_finalizer *finalizer =
> >> + &dispatcher->finalizers[i];
> >> +
> >> + finalizer->finalize_fun(dispatcher->event_dev_id,
> >> + port->port_id,
> >> + finalizer->finalize_data);
> >> + }
> >> +}
> >> +
> >> +static __rte_always_inline uint16_t
> >> +evd_port_dequeue(struct rte_dispatcher *dispatcher,
> >> + struct rte_dispatcher_lcore *lcore,
> >> + struct rte_dispatcher_lcore_port *port) {
> >> + uint16_t batch_size = port->batch_size;
> >> + struct rte_event events[batch_size];
> >> + uint16_t n;
> >> +
> >> + n = rte_event_dequeue_burst(dispatcher->event_dev_id, port-
> >>> port_id,
> >> + events, batch_size, port->timeout);
> >> +
> >> + if (likely(n > 0))
> >> + evd_dispatch_events(dispatcher, lcore, port, events, n);
> >> +
> >> + lcore->stats.poll_count++;
> >> +
> >> + return n;
> >> +}
> >> +
> >> +static __rte_always_inline uint16_t
> >> +evd_lcore_process(struct rte_dispatcher *dispatcher,
> >> + struct rte_dispatcher_lcore *lcore) {
> >> + uint16_t i;
> >> + uint16_t event_count = 0;
> >> +
> >> + for (i = 0; i < lcore->num_ports; i++) {
> >> + struct rte_dispatcher_lcore_port *port =
> >> + &lcore->ports[i];
> >> +
> >> + event_count += evd_port_dequeue(dispatcher, lcore, port);
> >> + }
> >> +
> >> + return event_count;
> >> +}
> >> +
> >> +static int32_t
> >> +evd_process(void *userdata)
> >> +{
> >> + struct rte_dispatcher *dispatcher = userdata;
> >> + unsigned int lcore_id = rte_lcore_id();
> >> + struct rte_dispatcher_lcore *lcore =
> >> + &dispatcher->lcores[lcore_id];
> >> + uint64_t event_count;
> >> +
> >> + event_count = evd_lcore_process(dispatcher, lcore);
> >> +
> >> + if (unlikely(event_count == 0))
> >> + return -EAGAIN;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int
> >> +evd_service_register(struct rte_dispatcher *dispatcher) {
> >> + struct rte_service_spec service = {
> >> + .callback = evd_process,
> >> + .callback_userdata = dispatcher,
> >> + .capabilities = RTE_SERVICE_CAP_MT_SAFE,
> >> + .socket_id = dispatcher->socket_id
> >> + };
> >> + int rc;
> >> +
> >> + snprintf(service.name, RTE_SERVICE_NAME_MAX - 1, "evd_%d",
> >> + dispatcher->id);
> >> +
> >> + rc = rte_service_component_register(&service,
> >> +&dispatcher->service_id);
> >> +
> >> + if (rc)
> >> + RTE_EDEV_LOG_ERR("Registration of dispatcher service "
> >> + "%s failed with error code %d\n",
> >> + service.name, rc);
> >> +
> >> + return rc;
> >> +}
> >> +
> >> +static int
> >> +evd_service_unregister(struct rte_dispatcher *dispatcher) {
> >> + int rc;
> >> +
> >> + rc = rte_service_component_unregister(dispatcher->service_id);
> >> +
> >> + if (rc)
> >> + RTE_EDEV_LOG_ERR("Unregistration of dispatcher service "
> >> + "failed with error code %d\n", rc);
> >> +
> >> + return rc;
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_create(uint8_t id, uint8_t event_dev_id) {
> >> + int socket_id;
> >> + struct rte_dispatcher *dispatcher;
> >> + int rc;
> >> +
> >> + if (evd_has_dispatcher(id)) {
> >> + RTE_EDEV_LOG_ERR("Dispatcher with id %d already exists\n",
> >> + id);
> >> + return -EEXIST;
> >> + }
> >> +
> >> + socket_id = rte_event_dev_socket_id(event_dev_id);
> >> +
> >> + dispatcher =
> >> + rte_malloc_socket("dispatcher", sizeof(struct rte_dispatcher),
> >> + RTE_CACHE_LINE_SIZE, socket_id);
> >> +
> >> + if (dispatcher == NULL) {
> >> + RTE_EDEV_LOG_ERR("Unable to allocate memory for
> >> dispatcher\n");
> >> + return -ENOMEM;
> >> + }
> >> +
> >> + *dispatcher = (struct rte_dispatcher) {
> >> + .id = id,
> >> + .event_dev_id = event_dev_id,
> >> + .socket_id = socket_id
> >> + };
> >> +
> >> + rc = evd_service_register(dispatcher);
> >> +
> >> + if (rc < 0) {
> >> + rte_free(dispatcher);
> >> + return rc;
> >> + }
> >> +
> >> + evd_set_dispatcher(id, dispatcher);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_free(uint8_t id)
> >> +{
> >> + struct rte_dispatcher *dispatcher;
> >> + int rc;
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >> + rc = evd_service_unregister(dispatcher);
> >> +
> >> + if (rc)
> >> + return rc;
> >> +
> >> + evd_set_dispatcher(id, NULL);
> >> +
> >> + rte_free(dispatcher);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_service_id_get(uint8_t id, uint32_t *service_id) {
> >> + struct rte_dispatcher *dispatcher;
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >
> > Service_id pointer needs to be validated for NULL before accessing
> >
> >
>
> Noted.
>
> Returning error codes on API violations is bad practice (such should be met
> with assertions), but it is DPDK standard practice and how Eventdev does
> things as well, so I'll change it.
>
> It would also be consistent with allowing invalid 'id' parameter values.
>
> >> + *service_id = dispatcher->service_id;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int
> >> +lcore_port_index(struct rte_dispatcher_lcore *lcore,
> >> + uint8_t event_port_id)
> >> +{
> >> + uint16_t i;
> >> +
> >> + for (i = 0; i < lcore->num_ports; i++) {
> >> + struct rte_dispatcher_lcore_port *port =
> >> + &lcore->ports[i];
> >> +
> >> + if (port->port_id == event_port_id)
> >> + return i;
> >> + }
> >> +
> >> + return -1;
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_bind_port_to_lcore(uint8_t id, uint8_t event_port_id,
> >> + uint16_t batch_size, uint64_t
> >> timeout,
> >> + unsigned int lcore_id)
> >> +{
> >> + struct rte_dispatcher *dispatcher;
> >> + struct rte_dispatcher_lcore *lcore;
> >> + struct rte_dispatcher_lcore_port *port;
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >> + lcore = &dispatcher->lcores[lcore_id];
> >> +
> >> + if (lcore->num_ports == EVD_MAX_PORTS_PER_LCORE)
> >> + return -ENOMEM;
> >> +
> >> + if (lcore_port_index(lcore, event_port_id) >= 0)
> >> + return -EEXIST;
> >> +
> >> + port = &lcore->ports[lcore->num_ports];
> >> +
> >> + *port = (struct rte_dispatcher_lcore_port) {
> >> + .port_id = event_port_id,
> >> + .batch_size = batch_size,
> >> + .timeout = timeout
> >> + };
> >> +
> >> + lcore->num_ports++;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_unbind_port_from_lcore(uint8_t id, uint8_t
> event_port_id,
> >> + unsigned int lcore_id)
> >> +{
> >> + struct rte_dispatcher *dispatcher;
> >> + struct rte_dispatcher_lcore *lcore;
> >> + int port_idx;
> >> + struct rte_dispatcher_lcore_port *port;
> >> + struct rte_dispatcher_lcore_port *last;
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >> + lcore = &dispatcher->lcores[lcore_id];
> >> +
> >> + port_idx = lcore_port_index(lcore, event_port_id);
> >> +
> >> + if (port_idx < 0)
> >> + return -ENOENT;
> >> +
> >> + port = &lcore->ports[port_idx];
> >> + last = &lcore->ports[lcore->num_ports - 1];
> >> +
> >> + if (port != last)
> >> + *port = *last;
> >> +
> >> + lcore->num_ports--;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static struct rte_dispatcher_handler*
> >> +evd_lcore_get_handler_by_id(struct rte_dispatcher_lcore *lcore,
> >> + int handler_id)
> >> +{
> >> + uint16_t i;
> >> +
> >> + for (i = 0; i < lcore->num_handlers; i++) {
> >> + struct rte_dispatcher_handler *handler =
> >> + &lcore->handlers[i];
> >> +
> >> + if (handler->id == handler_id)
> >> + return handler;
> >> + }
> >> +
> >> + return NULL;
> >> +}
> >> +
> >> +static int
> >> +evd_alloc_handler_id(struct rte_dispatcher *dispatcher) {
> >> + int handler_id = 0;
> >> + struct rte_dispatcher_lcore *reference_lcore =
> >> + &dispatcher->lcores[0];
> >> +
> >> + if (reference_lcore->num_handlers == EVD_MAX_HANDLERS)
> >> + return -1;
> >> +
> >> + while (evd_lcore_get_handler_by_id(reference_lcore, handler_id) !=
> >> NULL)
> >> + handler_id++;
> >> +
> >> + return handler_id;
> >> +}
> >> +
> >> +static void
> >> +evd_lcore_install_handler(struct rte_dispatcher_lcore *lcore,
> >> + const struct rte_dispatcher_handler *handler) {
> >> + int handler_idx = lcore->num_handlers;
> >> +
> >> + lcore->handlers[handler_idx] = *handler;
> >> + lcore->num_handlers++;
> >> +}
> >> +
> >> +static void
> >> +evd_install_handler(struct rte_dispatcher *dispatcher,
> >> + const struct rte_dispatcher_handler *handler) {
> >> + int i;
> >> +
> >> + for (i = 0; i < RTE_MAX_LCORE; i++) {
> >> + struct rte_dispatcher_lcore *lcore =
> >> + &dispatcher->lcores[i];
> >> + evd_lcore_install_handler(lcore, handler);
> >> + }
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_register(uint8_t id,
> >> + rte_dispatcher_match_t match_fun,
> >> + void *match_data,
> >> + rte_dispatcher_process_t process_fun,
> >> + void *process_data)
> >> +{
> >> + struct rte_dispatcher *dispatcher;
> >> + struct rte_dispatcher_handler handler = {
> >> + .match_fun = match_fun,
> >> + .match_data = match_data,
> >
> > We can have a default function which uses queue_id as matching data.
> > This reduces the application load to provide two callbacks, one for matching
> and one for processing the event.
> > Application can pass NULL parameter for "match_fun", in that case default
> function pointer can be used here.
> >
>
> But which queue id would this default function pointer match?
>
> I think you need more API calls to allow for something like this. I've discussed
> this kind of API in some previous messager on this list, if I recall correctly.
>
Agree, it may require some more APIs to implement this functionality.
I am fine to continue with the current logic.
> >
> >> + .process_fun = process_fun,
> >> + .process_data = process_data
> >> + };
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >> + handler.id = evd_alloc_handler_id(dispatcher);
> >> +
> >> + if (handler.id < 0)
> >> + return -ENOMEM;
> >> +
> >> + evd_install_handler(dispatcher, &handler);
> >> +
> >> + return handler.id;
> >> +}
> >> +
> >> +static int
> >> +evd_lcore_uninstall_handler(struct rte_dispatcher_lcore *lcore,
> >> + int handler_id)
> >> +{
> >> + struct rte_dispatcher_handler *unreg_handler;
> >> + int handler_idx;
> >> + uint16_t last_idx;
> >> +
> >> + unreg_handler = evd_lcore_get_handler_by_id(lcore, handler_id);
> >> +
> >> + if (unreg_handler == NULL)
> >> + return -EINVAL;
> >> +
> >
> > Shouldn't the logic be " handler_idx = unreg_handler - &lcore->handlers[0];"
> > Because, unreg_handler will be a higher or equal address to the
> > handler base address (&lcore->handlers[0])
> >
>
> True. Will fix.
>
> >
> >> + handler_idx = &lcore->handlers[0] - unreg_handler;
> >> +
> >> + last_idx = lcore->num_handlers - 1;
> >> +
> >> + if (handler_idx != last_idx) {
> >> + /* move all handlers to maintain handler order */
> >> + int n = last_idx - handler_idx;
> >> + memmove(unreg_handler, unreg_handler + 1,
> >> + sizeof(struct rte_dispatcher_handler) * n);
> >> + }
> >> +
> >> + lcore->num_handlers--;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int
> >> +evd_uninstall_handler(struct rte_dispatcher *dispatcher,
> >> + int handler_id)
> >> +{
> >> + unsigned int lcore_id;
> >> +
> >> + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
> >> + struct rte_dispatcher_lcore *lcore =
> >> + &dispatcher->lcores[lcore_id];
> >> + int rc;
> >> +
> >> + rc = evd_lcore_uninstall_handler(lcore, handler_id);
> >> +
> >> + if (rc < 0)
> >> + return rc;
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_unregister(uint8_t id, int handler_id) {
> >> + struct rte_dispatcher *dispatcher;
> >> + int rc;
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >> + rc = evd_uninstall_handler(dispatcher, handler_id);
> >> +
> >> + return rc;
> >> +}
> >> +
> >> +static struct rte_dispatcher_finalizer*
> >> +evd_get_finalizer_by_id(struct rte_dispatcher *dispatcher,
> >> + int handler_id)
> >> +{
> >> + int i;
> >> +
> >> + for (i = 0; i < dispatcher->num_finalizers; i++) {
> >> + struct rte_dispatcher_finalizer *finalizer =
> >> + &dispatcher->finalizers[i];
> >> +
> >> + if (finalizer->id == handler_id)
> >> + return finalizer;
> >> + }
> >> +
> >> + return NULL;
> >> +}
> >> +
> >> +static int
> >> +evd_alloc_finalizer_id(struct rte_dispatcher *dispatcher) {
> >> + int finalizer_id = 0;
> >> +
> >> + while (evd_get_finalizer_by_id(dispatcher, finalizer_id) != NULL)
> >> + finalizer_id++;
> >> +
> >> + return finalizer_id;
> >> +}
> >> +
> >> +static struct rte_dispatcher_finalizer * evd_alloc_finalizer(struct
> >> +rte_dispatcher *dispatcher) {
> >> + int finalizer_idx;
> >> + struct rte_dispatcher_finalizer *finalizer;
> >> +
> >> + if (dispatcher->num_finalizers == EVD_MAX_FINALIZERS)
> >> + return NULL;
> >> +
> >> + finalizer_idx = dispatcher->num_finalizers;
> >> + finalizer = &dispatcher->finalizers[finalizer_idx];
> >> +
> >> + finalizer->id = evd_alloc_finalizer_id(dispatcher);
> >> +
> >> + dispatcher->num_finalizers++;
> >> +
> >> + return finalizer;
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_finalize_register(uint8_t id,
> >> + rte_dispatcher_finalize_t finalize_fun,
> >> + void *finalize_data)
> >> +{
> >> + struct rte_dispatcher *dispatcher;
> >> + struct rte_dispatcher_finalizer *finalizer;
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >> + finalizer = evd_alloc_finalizer(dispatcher);
> >> +
> >> + if (finalizer == NULL)
> >> + return -ENOMEM;
> >> +
> >> + finalizer->finalize_fun = finalize_fun;
> >> + finalizer->finalize_data = finalize_data;
> >> +
> >> + return finalizer->id;
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_finalize_unregister(uint8_t id, int handler_id) {
> >> + struct rte_dispatcher *dispatcher;
> >> + struct rte_dispatcher_finalizer *unreg_finalizer;
> >> + int finalizer_idx;
> >> + uint16_t last_idx;
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >> + unreg_finalizer = evd_get_finalizer_by_id(dispatcher, handler_id);
> >> +
> >> + if (unreg_finalizer == NULL)
> >> + return -EINVAL;
> >> +
> >
> > Same as above comment in rte_dispatcher_unregister, base address needs
> > to be subtracted from unreg_finalizer
> >
>
> Yes.
>
> >
> >> + finalizer_idx = &dispatcher->finalizers[0] - unreg_finalizer;
> >> +
> >> + last_idx = dispatcher->num_finalizers - 1;
> >> +
> >> + if (finalizer_idx != last_idx) {
> >> + /* move all finalizers to maintain order */
> >> + int n = last_idx - finalizer_idx;
> >> + memmove(unreg_finalizer, unreg_finalizer + 1,
> >> + sizeof(struct rte_dispatcher_finalizer) * n);
> >> + }
> >> +
> >> + dispatcher->num_finalizers--;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int
> >> +evd_set_service_runstate(uint8_t id, int state) {
> >> + struct rte_dispatcher *dispatcher;
> >> + int rc;
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >> + rc = rte_service_component_runstate_set(dispatcher->service_id,
> >> + state);
> >> +
> >> + if (rc != 0) {
> >> + RTE_EDEV_LOG_ERR("Unexpected error %d occurred while
> >> setting "
> >> + "service component run state to %d\n", rc,
> >> + state);
> >> + RTE_ASSERT(0);
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_start(uint8_t id)
> >> +{
> >> + return evd_set_service_runstate(id, 1); }
> >> +
> >> +int
> >> +rte_dispatcher_stop(uint8_t id)
> >> +{
> >> + return evd_set_service_runstate(id, 0); }
> >> +
> >> +static void
> >> +evd_aggregate_stats(struct rte_dispatcher_stats *result,
> >> + const struct rte_dispatcher_stats *part) {
> >> + result->poll_count += part->poll_count;
> >> + result->ev_batch_count += part->ev_batch_count;
> >> + result->ev_dispatch_count += part->ev_dispatch_count;
> >> + result->ev_drop_count += part->ev_drop_count; }
> >> +
> >> +int
> >> +rte_dispatcher_stats_get(uint8_t id,
> >> + struct rte_dispatcher_stats *stats) {
> >> + struct rte_dispatcher *dispatcher;
> >> + unsigned int lcore_id;
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >
> > Stats pointer needs to be validated for NULL before accessing
> >
> >
>
> Yes.
>
> Thanks a lot for your review comments.
>
> >> + *stats = (struct rte_dispatcher_stats) {};
> >> +
> >> + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
> >> + struct rte_dispatcher_lcore *lcore =
> >> + &dispatcher->lcores[lcore_id];
> >> +
> >> + evd_aggregate_stats(stats, &lcore->stats);
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +int
> >> +rte_dispatcher_stats_reset(uint8_t id) {
> >> + struct rte_dispatcher *dispatcher;
> >> + unsigned int lcore_id;
> >> +
> >> + EVD_VALID_ID_OR_RET_EINVAL(id);
> >> + dispatcher = evd_get_dispatcher(id);
> >> +
> >> +
> >> + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
> >> + struct rte_dispatcher_lcore *lcore =
> >> + &dispatcher->lcores[lcore_id];
> >> +
> >> + lcore->stats = (struct rte_dispatcher_stats) {};
> >> + }
> >> +
> >> + return 0;
> >> +
> >> +}
> >> diff --git a/lib/dispatcher/rte_dispatcher.h
> >> b/lib/dispatcher/rte_dispatcher.h new file mode 100644 index
> >> 0000000000..6712687a08
> >> --- /dev/null
> >> +++ b/lib/dispatcher/rte_dispatcher.h
> >> @@ -0,0 +1,480 @@
> >> +/* SPDX-License-Identifier: BSD-3-Clause
> >> + * Copyright(c) 2023 Ericsson AB
> >> + */
> >> +
> >> +#ifndef __RTE_DISPATCHER_H__
> >> +#define __RTE_DISPATCHER_H__
> >> +
> >> +/**
> >> + * @file
> >> + *
> >> + * RTE Dispatcher
> >> + *
> >> + * The purpose of the dispatcher is to help decouple different parts
> >> + * of an application (e.g., modules), sharing the same underlying
> >> + * event device.
> >> + */
> >> +
> >> +#ifdef __cplusplus
> >> +extern "C" {
> >> +#endif
> >> +
> >> +#include <rte_eventdev.h>
> >> +
> >> +/**
> >> + * Function prototype for match callbacks.
> >> + *
> >> + * Match callbacks are used by an application to decide how the
> >> + * dispatcher distributes events to different parts of the
> >> + * application.
> >> + *
> >> + * The application is not expected to process the event at the point
> >> + * of the match call. Such matters should be deferred to the process
> >> + * callback invocation.
> >> + *
> >> + * The match callback may be used as an opportunity to prefetch data.
> >> + *
> >> + * @param event
> >> + * Pointer to event
> >> + *
> >> + * @param cb_data
> >> + * The pointer supplied by the application in
> >> + * rte_dispatcher_register().
> >> + *
> >> + * @return
> >> + * Returns true in case this events should be delivered (via
> >> + * the process callback), and false otherwise.
> >> + */
> >> +typedef bool
> >> +(*rte_dispatcher_match_t)(const struct rte_event *event, void
> >> +*cb_data);
> >> +
> >> +/**
> >> + * Function prototype for process callbacks.
> >> + *
> >> + * The process callbacks are used by the dispatcher to deliver
> >> + * events for processing.
> >> + *
> >> + * @param event_dev_id
> >> + * The originating event device id.
> >> + *
> >> + * @param event_port_id
> >> + * The originating event port.
> >> + *
> >> + * @param events
> >> + * Pointer to an array of events.
> >> + *
> >> + * @param num
> >> + * The number of events in the @p events array.
> >> + *
> >> + * @param cb_data
> >> + * The pointer supplied by the application in
> >> + * rte_dispatcher_register().
> >> + */
> >> +
> >> +typedef void
> >> +(*rte_dispatcher_process_t)(uint8_t event_dev_id, uint8_t event_port_id,
> >> + struct rte_event *events, uint16_t num,
> >> + void *cb_data);
> >> +
> >> +/**
> >> + * Function prototype for finalize callbacks.
> >> + *
> >> + * The finalize callbacks are used by the dispatcher to notify the
> >> + * application it has delivered all events from a particular batch
> >> + * dequeued from the event device.
> >> + *
> >> + * @param event_dev_id
> >> + * The originating event device id.
> >> + *
> >> + * @param event_port_id
> >> + * The originating event port.
> >> + *
> >> + * @param cb_data
> >> + * The pointer supplied by the application in
> >> + * rte_dispatcher_finalize_register().
> >> + */
> >> +
> >> +typedef void
> >> +(*rte_dispatcher_finalize_t)(uint8_t event_dev_id, uint8_t event_port_id,
> >> + void *cb_data);
> >> +
> >> +/**
> >> + * Dispatcher statistics
> >> + */
> >> +struct rte_dispatcher_stats {
> >> + uint64_t poll_count;
> >> + /**< Number of event dequeue calls made toward the event device. */
> >> + uint64_t ev_batch_count;
> >> + /**< Number of non-empty event batches dequeued from event
> >> device.*/
> >> + uint64_t ev_dispatch_count;
> >> + /**< Number of events dispatched to a handler.*/
> >> + uint64_t ev_drop_count;
> >> + /**< Number of events dropped because no handler was found. */ };
> >> +
> >> +/**
> >> + * Create a dispatcher with the specified id.
> >> + *
> >> + * @param id
> >> + * An application-specified, unique (across all dispatcher
> >> + * instances) identifier.
> >> + *
> >> + * @param event_dev_id
> >> + * The identifier of the event device from which this dispatcher
> >> + * will dequeue events.
> >> + *
> >> + * @return
> >> + * - 0: Success
> >> + * - <0: Error code on failure
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_create(uint8_t id, uint8_t event_dev_id);
> >> +
> >> +/**
> >> + * Free a dispatcher.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @return
> >> + * - 0: Success
> >> + * - <0: Error code on failure
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_free(uint8_t id);
> >> +
> >> +/**
> >> + * Retrieve the service identifier of a dispatcher.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @param [out] service_id
> >> + * A pointer to a caller-supplied buffer where the dispatcher's
> >> + * service id will be stored.
> >> + *
> >> + * @return
> >> + * - 0: Success
> >> + * - <0: Error code on failure.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_service_id_get(uint8_t id, uint32_t *service_id);
> >> +
> >> +/**
> >> + * Binds an event device port to a specific lcore on the specified
> >> + * dispatcher.
> >> + *
> >> + * This function configures the event port id to be used by the
> >> +event
> >> + * dispatcher service, if run on the specified lcore.
> >> + *
> >> + * Multiple event device ports may be bound to the same lcore. A
> >> + * particular port must not be bound to more than one lcore.
> >> + *
> >> + * If the dispatcher service is mapped (with
> >> +rte_service_map_lcore_set())
> >> + * to a lcore to which no ports are bound, the service function will
> >> +be a
> >> + * no-operation.
> >> + *
> >> + * This function may be called by any thread (including unregistered
> >> + * non-EAL threads), but not while the dispatcher is running on
> >> +lcore
> >> + * specified by @c lcore_id.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @param event_port_id
> >> + * The event device port identifier.
> >> + *
> >> + * @param batch_size
> >> + * The batch size to use in rte_event_dequeue_burst(), for the
> >> + * configured event device port and lcore.
> >> + *
> >> + * @param timeout
> >> + * The timeout parameter to use in rte_event_dequeue_burst(), for
> >> +the
> >> + * configured event device port and lcore.
> >> + *
> >> + * @param lcore_id
> >> + * The lcore by which this event port will be used.
> >> + *
> >> + * @return
> >> + * - 0: Success
> >> + * - -ENOMEM: Unable to allocate sufficient resources.
> >> + * - -EEXISTS: Event port is already configured.
> >> + * - -EINVAL: Invalid arguments.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_bind_port_to_lcore(uint8_t id, uint8_t event_port_id,
> >> + uint16_t batch_size, uint64_t
> >> timeout,
> >> + unsigned int lcore_id);
> >> +
> >> +/**
> >> + * Unbind an event device port from a specific lcore.
> >> + *
> >> + * This function may be called by any thread (including unregistered
> >> + * non-EAL threads), but not while the dispatcher is running on
> >> + * lcore specified by @c lcore_id.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @param event_port_id
> >> + * The event device port identifier.
> >> + *
> >> + * @param lcore_id
> >> + * The lcore which was using this event port.
> >> + *
> >> + * @return
> >> + * - 0: Success
> >> + * - -EINVAL: Invalid @c id.
> >> + * - -ENOENT: Event port id not bound to this @c lcore_id.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_unbind_port_from_lcore(uint8_t id, uint8_t
> event_port_id,
> >> + unsigned int lcore_id);
> >> +
> >> +/**
> >> + * Register an event handler.
> >> + *
> >> + * The match callback function is used to select if a particular
> >> +event
> >> + * should be delivered, using the corresponding process callback
> >> + * function.
> >> + *
> >> + * The reason for having two distinct steps is to allow the
> >> +dispatcher
> >> + * to deliver all events as a batch. This in turn will cause
> >> + * processing of a particular kind of events to happen in a
> >> + * back-to-back manner, improving cache locality.
> >> + *
> >> + * The list of handler callback functions is shared among all
> >> +lcores,
> >> + * but will only be executed on lcores which has an eventdev port
> >> + * bound to them, and which are running the dispatcher service.
> >> + *
> >> + * An event is delivered to at most one handler. Events where no
> >> + * handler is found are dropped.
> >> + *
> >> + * The application must not depend on the order of which the match
> >> + * functions are invoked.
> >> + *
> >> + * Ordering of events is not guaranteed to be maintained between
> >> + * different deliver callbacks. For example, suppose there are two
> >> + * callbacks registered, matching different subsets of events
> >> +arriving
> >> + * on an atomic queue. A batch of events [ev0, ev1, ev2] are
> >> +dequeued
> >> + * on a particular port, all pertaining to the same flow. The match
> >> + * callback for registration A returns true for ev0 and ev2, and the
> >> + * matching function for registration B for ev1. In that scenario,
> >> +the
> >> + * dispatcher may choose to deliver first [ev0, ev2] using A's
> >> +deliver
> >> + * function, and then [ev1] to B - or vice versa.
> >> + *
> >> + * rte_dispatcher_register() may be called by any thread
> >> + * (including unregistered non-EAL threads), but not while the event
> >> + * dispatcher is running on any service lcore.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @param match_fun
> >> + * The match callback function.
> >> + *
> >> + * @param match_cb_data
> >> + * A pointer to some application-specific opaque data (or NULL),
> >> + * which is supplied back to the application when match_fun is
> >> + * called.
> >> + *
> >> + * @param process_fun
> >> + * The process callback function.
> >> + *
> >> + * @param process_cb_data
> >> + * A pointer to some application-specific opaque data (or NULL),
> >> + * which is supplied back to the application when process_fun is
> >> + * called.
> >> + *
> >> + * @return
> >> + * - >= 0: The identifier for this registration.
> >> + * - -ENOMEM: Unable to allocate sufficient resources.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_register(uint8_t id,
> >> + rte_dispatcher_match_t match_fun,
> >> + void *match_cb_data,
> >> + rte_dispatcher_process_t process_fun,
> >> + void *process_cb_data);
> >> +
> >> +/**
> >> + * Unregister an event handler.
> >> + *
> >> + * This function may be called by any thread (including unregistered
> >> + * non-EAL threads), but not while the dispatcher is running on
> >> + * any service lcore.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @param handler_id
> >> + * The handler registration id returned by the original
> >> + * rte_dispatcher_register() call.
> >> + *
> >> + * @return
> >> + * - 0: Success
> >> + * - -EINVAL: The @c id and/or the @c handler_id parameter was invalid.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_unregister(uint8_t id, int handler_id);
> >> +
> >> +/**
> >> + * Register a finalize callback function.
> >> + *
> >> + * An application may optionally install one or more finalize
> >> + * callbacks.
> >> + *
> >> + * All finalize callbacks are invoked by the dispatcher when a
> >> + * complete batch of events (retrieve using
> >> +rte_event_dequeue_burst())
> >> + * have been delivered to the application (or have been dropped).
> >> + *
> >> + * The finalize callback is not tied to any particular handler.
> >> + *
> >> + * The finalize callback provides an opportunity for the application
> >> + * to do per-batch processing. One case where this may be useful is
> >> +if
> >> + * an event output buffer is used, and is shared among several
> >> + * handlers. In such a case, proper output buffer flushing may be
> >> + * assured using a finalize callback.
> >> + *
> >> + * rte_dispatcher_finalize_register() may be called by any thread
> >> + * (including unregistered non-EAL threads), but not while the
> >> + * dispatcher is running on any service lcore.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @param finalize_fun
> >> + * The function called after completing the processing of a
> >> + * dequeue batch.
> >> + *
> >> + * @param finalize_data
> >> + * A pointer to some application-specific opaque data (or NULL),
> >> + * which is supplied back to the application when @c finalize_fun
> >> +is
> >> + * called.
> >> + *
> >> + * @return
> >> + * - >= 0: The identifier for this registration.
> >> + * - -ENOMEM: Unable to allocate sufficient resources.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_finalize_register(uint8_t id,
> >> + rte_dispatcher_finalize_t finalize_fun,
> >> + void *finalize_data);
> >> +
> >> +/**
> >> + * Unregister a finalize callback.
> >> + *
> >> + * This function may be called by any thread (including unregistered
> >> + * non-EAL threads), but not while the dispatcher is running on
> >> + * any service lcore.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @param reg_id
> >> + * The finalize registration id returned by the original
> >> + * rte_dispatcher_finalize_register() call.
> >> + *
> >> + * @return
> >> + * - 0: Success
> >> + * - -EINVAL: The @c id and/or the @c reg_id parameter was invalid.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_finalize_unregister(uint8_t id, int reg_id);
> >> +
> >> +/**
> >> + * Start a dispatcher instance.
> >> + *
> >> + * Enables the dispatcher service.
> >> + *
> >> + * The underlying event device must have been started prior to
> >> +calling
> >> + * rte_dispatcher_start().
> >> + *
> >> + * For the dispatcher to actually perform work (i.e., dispatch
> >> + * events), its service must have been mapped to one or more service
> >> + * lcores, and its service run state set to '1'. A dispatcher's
> >> + * service is retrieved using rte_dispatcher_service_id_get().
> >> + *
> >> + * Each service lcore to which the dispatcher is mapped should
> >> + * have at least one event port configured. Such configuration is
> >> + * performed by calling rte_dispatcher_bind_port_to_lcore(), prior
> >> +to
> >> + * starting the dispatcher.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @return
> >> + * - 0: Success
> >> + * - -EINVAL: Invalid @c id.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_start(uint8_t id);
> >> +
> >> +/**
> >> + * Stop a running dispatcher instance.
> >> + *
> >> + * Disables the dispatcher service.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @return
> >> + * - 0: Success
> >> + * - -EINVAL: Invalid @c id.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_stop(uint8_t id);
> >> +
> >> +/**
> >> + * Retrieve statistics for a dispatcher instance.
> >> + *
> >> + * This function is MT safe and may be called by any thread
> >> + * (including unregistered non-EAL threads).
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + * @param[out] stats
> >> + * A pointer to a structure to fill with statistics.
> >> + * @return
> >> + * - 0: Success
> >> + * - -EINVAL: The @c id parameter was invalid.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_stats_get(uint8_t id,
> >> + struct rte_dispatcher_stats *stats);
> >> +
> >> +/**
> >> + * Reset statistics for a dispatcher instance.
> >> + *
> >> + * This function may be called by any thread (including unregistered
> >> + * non-EAL threads), but may not produce the correct result if the
> >> + * dispatcher is running on any service lcore.
> >> + *
> >> + * @param id
> >> + * The dispatcher identifier.
> >> + *
> >> + * @return
> >> + * - 0: Success
> >> + * - -EINVAL: The @c id parameter was invalid.
> >> + */
> >> +__rte_experimental
> >> +int
> >> +rte_dispatcher_stats_reset(uint8_t id);
> >> +
> >> +#ifdef __cplusplus
> >> +}
> >> +#endif
> >> +
> >> +#endif /* __RTE_DISPATCHER__ */
> >> diff --git a/lib/dispatcher/version.map b/lib/dispatcher/version.map
> >> new file mode 100644 index 0000000000..8f9ad96522
> >> --- /dev/null
> >> +++ b/lib/dispatcher/version.map
> >> @@ -0,0 +1,20 @@
> >> +EXPERIMENTAL {
> >> + global:
> >> +
> >> + # added in 23.11
> >> + rte_dispatcher_create;
> >> + rte_dispatcher_free;
> >> + rte_dispatcher_service_id_get;
> >> + rte_dispatcher_bind_port_to_lcore;
> >> + rte_dispatcher_unbind_port_from_lcore;
> >> + rte_dispatcher_register;
> >> + rte_dispatcher_unregister;
> >> + rte_dispatcher_finalize_register;
> >> + rte_dispatcher_finalize_unregister;
> >> + rte_dispatcher_start;
> >> + rte_dispatcher_stop;
> >> + rte_dispatcher_stats_get;
> >> + rte_dispatcher_stats_reset;
> >> +
> >> + local: *;
> >> +};
> >> diff --git a/lib/meson.build b/lib/meson.build index
> >> 099b0ed18a..3093b338d2 100644
> >> --- a/lib/meson.build
> >> +++ b/lib/meson.build
> >> @@ -35,6 +35,7 @@ libraries = [
> >> 'distributor',
> >> 'efd',
> >> 'eventdev',
> >> + 'dispatcher', # dispatcher depends on eventdev
> >> 'gpudev',
> >> 'gro',
> >> 'gso',
> >> @@ -81,6 +82,7 @@ optional_libs = [
> >> 'cfgfile',
> >> 'compressdev',
> >> 'cryptodev',
> >> + 'dispatcher',
> >> 'distributor',
> >> 'dmadev',
> >> 'efd',
> >> --
> >> 2.34.1
> >
On Mon, Sep 18, 2023 at 5:26 AM Naga Harish K, S V
<s.v.naga.harish.k@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> > Sent: Monday, September 4, 2023 6:33 PM
> > To: dev@dpdk.org
> > Cc: Jerin Jacob <jerinj@marvell.com>; techboard@dpdk.org; Van Haaren,
> > Harry <harry.van.haaren@intel.com>; hofors@lysator.liu.se; Nilsson, Peter
> > <peter.j.nilsson@ericsson.com>; Heng Wang <heng.wang@ericsson.com>;
> > Naga Harish K, S V <s.v.naga.harish.k@intel.com>; Pavan Nikhilesh
> > <pbhagavatula@marvell.com>; Gujjar, Abhinandan S
> > <abhinandan.gujjar@intel.com>; Carrillo, Erik G <Erik.G.Carrillo@intel.com>;
> > Shijith Thotton <sthotton@marvell.com>; Hemant Agrawal
> > <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
> > Liang Ma <liangma@liangbit.com>; Mccarthy, Peter
> > <Peter.Mccarthy@intel.com>; Yan, Zhirun <Zhirun.Yan@intel.com>;
> > mattias.ronnblom <mattias.ronnblom@ericsson.com>
> > Subject: [PATCH v3 1/3] lib: introduce dispatcher library
> >
> > The purpose of the dispatcher library is to help reduce coupling in an
> > Eventdev-based DPDK application.
> >
> > In addition, the dispatcher also provides a convenient and flexible way for the
> > application to use service cores for application-level processing.
> >
> > Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> > Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
> > Reviewed-by: Heng Wang <heng.wang@ericsson.com>
> >
> > --
> >
> > PATCH v3:
> > o To underline its optional character and since it does not provide
> > hardware abstraction, the event dispatcher is now a separate
> > library.
> > o Change name from rte_event_dispatcher -> rte_dispatcher, to make it
> > shorter and to avoid the rte_event_* namespace.
> >
>
> Rte_dispatcher is basically dispatching events but it feels like the name does not convey that.
> Also, it is like any other adapter service that can reside within the eventdev directory.
>
> I can see some discussion in previous threads related to the placement of the dispatcher library.
>
> It is an optional eventdev application service, not enforcing this programming model to the application.
> The documentation may need to be updated and mention that this is optional.
>
> If any hardware comes up with the dispatcher feature, then this library may need to be moved inside eventdev library later.
I would like to follow YAGNI principle in eventdev library.
Even if a HW comes(I assume not), the interface should not look like that.
None of the HW will be comparing a bunch of function pointers and call
the callback.
So interface will look different for HW enablement. We need to model
the API based on HW for device libraries and SW libraries based on CPU
modeling dynamics.
Also, There is no need to tie up this library/framework only event
]dev, other than using rte_event_dequeue() to pull packet it has no
eventdev significance.
The library scope if just pull the packet from a source and compare
with in N number of matches and call respective process callback.
The dispatcher source can rte_ethdev_rx_burst or ring.
> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Wednesday, September 20, 2023 3:02 PM
> To: Naga Harish K, S V <s.v.naga.harish.k@intel.com>
> Cc: mattias.ronnblom <mattias.ronnblom@ericsson.com>; dev@dpdk.org;
> Jerin Jacob <jerinj@marvell.com>; techboard@dpdk.org; Van Haaren, Harry
> <harry.van.haaren@intel.com>; hofors@lysator.liu.se; Nilsson, Peter
> <peter.j.nilsson@ericsson.com>; Heng Wang <heng.wang@ericsson.com>;
> Pavan Nikhilesh <pbhagavatula@marvell.com>; Gujjar, Abhinandan S
> <abhinandan.gujjar@intel.com>; Carrillo, Erik G <erik.g.carrillo@intel.com>;
> Shijith Thotton <sthotton@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
> Liang Ma <liangma@liangbit.com>; Mccarthy, Peter
> <peter.mccarthy@intel.com>; Yan, Zhirun <zhirun.yan@intel.com>
> Subject: Re: [PATCH v3 1/3] lib: introduce dispatcher library
>
> On Mon, Sep 18, 2023 at 5:26 AM Naga Harish K, S V
> <s.v.naga.harish.k@intel.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> > > Sent: Monday, September 4, 2023 6:33 PM
> > > To: dev@dpdk.org
> > > Cc: Jerin Jacob <jerinj@marvell.com>; techboard@dpdk.org; Van
> > > Haaren, Harry <harry.van.haaren@intel.com>; hofors@lysator.liu.se;
> > > Nilsson, Peter <peter.j.nilsson@ericsson.com>; Heng Wang
> > > <heng.wang@ericsson.com>; Naga Harish K, S V
> > > <s.v.naga.harish.k@intel.com>; Pavan Nikhilesh
> > > <pbhagavatula@marvell.com>; Gujjar, Abhinandan S
> > > <abhinandan.gujjar@intel.com>; Carrillo, Erik G
> > > <Erik.G.Carrillo@intel.com>; Shijith Thotton <sthotton@marvell.com>;
> > > Hemant Agrawal <hemant.agrawal@nxp.com>; Sachin Saxena
> > > <sachin.saxena@oss.nxp.com>; Liang Ma <liangma@liangbit.com>;
> > > Mccarthy, Peter <Peter.Mccarthy@intel.com>; Yan, Zhirun
> > > <Zhirun.Yan@intel.com>; mattias.ronnblom
> > > <mattias.ronnblom@ericsson.com>
> > > Subject: [PATCH v3 1/3] lib: introduce dispatcher library
> > >
> > > The purpose of the dispatcher library is to help reduce coupling in
> > > an Eventdev-based DPDK application.
> > >
> > > In addition, the dispatcher also provides a convenient and flexible
> > > way for the application to use service cores for application-level processing.
> > >
> > > Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> > > Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
> > > Reviewed-by: Heng Wang <heng.wang@ericsson.com>
> > >
> > > --
> > >
> > > PATCH v3:
> > > o To underline its optional character and since it does not provide
> > > hardware abstraction, the event dispatcher is now a separate
> > > library.
> > > o Change name from rte_event_dispatcher -> rte_dispatcher, to make it
> > > shorter and to avoid the rte_event_* namespace.
> > >
> >
> > Rte_dispatcher is basically dispatching events but it feels like the name does
> not convey that.
> > Also, it is like any other adapter service that can reside within the eventdev
> directory.
> >
> > I can see some discussion in previous threads related to the placement of the
> dispatcher library.
> >
> > It is an optional eventdev application service, not enforcing this
> programming model to the application.
> > The documentation may need to be updated and mention that this is
> optional.
> >
> > If any hardware comes up with the dispatcher feature, then this library may
> need to be moved inside eventdev library later.
>
> I would like to follow YAGNI principle in eventdev library.
What is YAGNI principle? for understanding purposes.
> Even if a HW comes(I assume not), the interface should not look like that.
> None of the HW will be comparing a bunch of function pointers and call the
> callback.
> So interface will look different for HW enablement. We need to model the API
> based on HW for device libraries and SW libraries based on CPU modeling
> dynamics.
>
> Also, There is no need to tie up this library/framework only event ]dev, other
> than using rte_event_dequeue() to pull packet it has no eventdev significance.
> The library scope if just pull the packet from a source and compare with in N
> number of matches and call respective process callback.
> The dispatcher source can rte_ethdev_rx_burst or ring.
The current implementation of rte_dispatcher is event-dev centric.
All the data structures are defined around the eventdev.
The documentation also mentions it is for eventdev-based applications.
Maybe the documentation needs to have the info on supporting different sources (from ethdev, ring etc).
On Thu, Sep 21, 2023 at 11:29 AM Naga Harish K, S V
<s.v.naga.harish.k@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > Sent: Wednesday, September 20, 2023 3:02 PM
> > To: Naga Harish K, S V <s.v.naga.harish.k@intel.com>
> > Cc: mattias.ronnblom <mattias.ronnblom@ericsson.com>; dev@dpdk.org;
> > Jerin Jacob <jerinj@marvell.com>; techboard@dpdk.org; Van Haaren, Harry
> > <harry.van.haaren@intel.com>; hofors@lysator.liu.se; Nilsson, Peter
> > <peter.j.nilsson@ericsson.com>; Heng Wang <heng.wang@ericsson.com>;
> > Pavan Nikhilesh <pbhagavatula@marvell.com>; Gujjar, Abhinandan S
> > <abhinandan.gujjar@intel.com>; Carrillo, Erik G <erik.g.carrillo@intel.com>;
> > Shijith Thotton <sthotton@marvell.com>; Hemant Agrawal
> > <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
> > Liang Ma <liangma@liangbit.com>; Mccarthy, Peter
> > <peter.mccarthy@intel.com>; Yan, Zhirun <zhirun.yan@intel.com>
> > Subject: Re: [PATCH v3 1/3] lib: introduce dispatcher library
> >
> > On Mon, Sep 18, 2023 at 5:26 AM Naga Harish K, S V
> > <s.v.naga.harish.k@intel.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> > > > Sent: Monday, September 4, 2023 6:33 PM
> > > > To: dev@dpdk.org
> > > > Cc: Jerin Jacob <jerinj@marvell.com>; techboard@dpdk.org; Van
> > > > Haaren, Harry <harry.van.haaren@intel.com>; hofors@lysator.liu.se;
> > > > Nilsson, Peter <peter.j.nilsson@ericsson.com>; Heng Wang
> > > > <heng.wang@ericsson.com>; Naga Harish K, S V
> > > > <s.v.naga.harish.k@intel.com>; Pavan Nikhilesh
> > > > <pbhagavatula@marvell.com>; Gujjar, Abhinandan S
> > > > <abhinandan.gujjar@intel.com>; Carrillo, Erik G
> > > > <Erik.G.Carrillo@intel.com>; Shijith Thotton <sthotton@marvell.com>;
> > > > Hemant Agrawal <hemant.agrawal@nxp.com>; Sachin Saxena
> > > > <sachin.saxena@oss.nxp.com>; Liang Ma <liangma@liangbit.com>;
> > > > Mccarthy, Peter <Peter.Mccarthy@intel.com>; Yan, Zhirun
> > > > <Zhirun.Yan@intel.com>; mattias.ronnblom
> > > > <mattias.ronnblom@ericsson.com>
> > > > Subject: [PATCH v3 1/3] lib: introduce dispatcher library
> > > >
> > > > The purpose of the dispatcher library is to help reduce coupling in
> > > > an Eventdev-based DPDK application.
> > > >
> > > > In addition, the dispatcher also provides a convenient and flexible
> > > > way for the application to use service cores for application-level processing.
> > > >
> > > > Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> > > > Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
> > > > Reviewed-by: Heng Wang <heng.wang@ericsson.com>
> > > >
> > > > --
> > > >
> > > > PATCH v3:
> > > > o To underline its optional character and since it does not provide
> > > > hardware abstraction, the event dispatcher is now a separate
> > > > library.
> > > > o Change name from rte_event_dispatcher -> rte_dispatcher, to make it
> > > > shorter and to avoid the rte_event_* namespace.
> > > >
> > >
> > > Rte_dispatcher is basically dispatching events but it feels like the name does
> > not convey that.
> > > Also, it is like any other adapter service that can reside within the eventdev
> > directory.
> > >
> > > I can see some discussion in previous threads related to the placement of the
> > dispatcher library.
> > >
> > > It is an optional eventdev application service, not enforcing this
> > programming model to the application.
> > > The documentation may need to be updated and mention that this is
> > optional.
> > >
> > > If any hardware comes up with the dispatcher feature, then this library may
> > need to be moved inside eventdev library later.
> >
> > I would like to follow YAGNI principle in eventdev library.
>
> What is YAGNI principle? for understanding purposes.
https://www.techtarget.com/whatis/definition/You-arent-gonna-need-it#:~:text=YAGNI%20principle%20(%22You%20Aren',desired%20increased%20frequency%20of%20releases.
What meant by that, If it meant to be abstracting any HW Device
feature(on the point of above: If any hardware comes up with the
dispatcher feature, then this library may
need to be moved inside eventdev library later) lest define API based
on a Device definition and its capability. If it is targeting for an
SW library align API based on that.
For example, If this needed to be implemented in HW as device feature,
I would implement as following
- CAM as ACL or EM to store the rules
- Key extractor to extract the interested filed(match actions).
Similar to rte_flow flow patterns
- Action will something similar to rte_flow mark action
(https://doc.dpdk.org/guides/prog_guide/rte_flow.html#action-mark).
- When packet/event delivered to application. HW stores the mark value
in the metadata up on match.
For such HW, if we try to abstract with current API proposal, it is
not efficient.
On the other way, if the use case, addressing the existing SW API
library and model around the above HW primitives also not efficient.
On 2023-09-19 12:58, Jerin Jacob wrote:
> On Mon, Sep 4, 2023 at 6:39 PM Mattias Rönnblom
> <mattias.ronnblom@ericsson.com> wrote:
>>
>> The purpose of the dispatcher library is to help reduce coupling in an
>> Eventdev-based DPDK application.
>>
>> In addition, the dispatcher also provides a convenient and flexible
>> way for the application to use service cores for application-level
>> processing.
>>
>> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>> Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
>
> High level architecture comment
> --------------------------------
>
> 1) I think, we don't need tie this library ONLY to event dev
> application. It can be used with poll mode as well,
> that way traditiona pipeline application with ethdev as source could
> use this library dispatch the packets.
>
They could potentially use a library *like this*, but I'm not sure it
should be this library, or at least I think it should be a different API
(better type checking, plus no obvious benefit of being more generic).
Another option for a traditional app calling rte_eth_rx_burst() directly
is to start using eventdev. :)
> We dont need to implement that first version but API can make room for
> such abstractions.
>
> Based on my understanding in fast-path it has means to
> a)Pull out the events using rte_event_dequeue()
> b)Compare with registered match functions and call process upon match.
>
> if we abstract (a) as rte_dispatcher_source, We could pull from ethdev
> via rte_eth_rx_burst() or
> from ring via dequeue_burst API or so based on rte_dispatcher_source
> selected for dispatch configuration
> and we can use different sevice function pointers to have different service core
> implementation without effecting performance each sources.
>
It could be generalized, for sure. I don't think it should be, at this
point at least.
Non-event dev events could - and at this point I'm leaning toward
*should* - be consumed as a different DPDK service, but potentially on
the same lcore.
If you would want to prepare for a future with several different event
sources, one could consider reintroducing the word "event" somewhere in
the dispatcher's name. So you would have
rte_event_dispatcher.h
rte_eth_dispatcher.h
or
rte_dispatcher_event.h
rte_dispatcher_eth.h
> High level cosmetic comment
> ----------------------------------------------------
> 1)Missing doxygen connection- See doc/api/doxy-api-index.md
>
rte_dispatcher.h is listed under **classification**, but this change is
in the programming guide patch. I'll move it to the patch containing the
header file.
> Process related comment
> ------------------------------------
> 1) Documentation does not need need separate patch. All recent library
> changes documentation in same file.
> You could have doc and API header file as first patch and
> implementation as subsequent patches.
>
>
I'm not sure how this is an improvement. Can you elaborate? For me, it
just seems like a change.
Are there some guidelines on how to split a larger change into a patch
set? A section on this matter in the contribution guide would be great.
>
>> diff --git a/lib/dispatcher/rte_dispatcher.h b/lib/dispatcher/rte_dispatcher.h
>> new file mode 100644
>> index 0000000000..6712687a08
>> --- /dev/null
>> +++ b/lib/dispatcher/rte_dispatcher.h
>> @@ -0,0 +1,480 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2023 Ericsson AB
>> + */
>> +
>> +#ifndef __RTE_DISPATCHER_H__
>> +#define __RTE_DISPATCHER_H__
>> +
>
>
> All new API should be experimental. See
> https://elixir.bootlin.com/dpdk/latest/source/lib/graph/rte_graph.h#L12
> example.
>
Noted.
>
>> +/**
>> + * @file
>> + *
>> + * RTE Dispatcher
>> + *
>> + * The purpose of the dispatcher is to help decouple different parts
>> + * of an application (e.g., modules), sharing the same underlying
>> + * event device.
>> +
>> +/**
>> + * Function prototype for match callbacks.
>> + *
>> + * Match callbacks are used by an application to decide how the
>> + * dispatcher distributes events to different parts of the
>> + * application.
>> + *
>> + * The application is not expected to process the event at the point
>> + * of the match call. Such matters should be deferred to the process
>> + * callback invocation.
>> + *
>> + * The match callback may be used as an opportunity to prefetch data.
>> + *
>> + * @param event
>> + * Pointer to event
>> + *
>> + * @param cb_data
>> + * The pointer supplied by the application in
>> + * rte_dispatcher_register().
>> + *
>> + * @return
>> + * Returns true in case this events should be delivered (via
>> + * the process callback), and false otherwise.
>> + */
>> +typedef bool
>> +(*rte_dispatcher_match_t)(const struct rte_event *event, void *cb_data);
>
>
> a) Can we use void* event, so that it can be used with mbuf or other
> type by casting in the call back implementer.
>
> b) I was thinking, How we can avoid this function pointer and enable
> more have better performance at architecture level.
>
> Both x86, ARM has vector instructions[1] to form a vector from various
> offset from memory and compare N events
> in one shot. That is, if express match data like offset = X has value
> is Y and offset = X has value = A.
> I know, it may not good existing application using this APIs. But I
> believe, it will be more performance
> effective. If make sense, you can adapt to this.(Something to think about)
>
There may be a future development where you try to shave off a few of
the circa 10 clock cycles per event of overhead that the current
implementation incur. 10 cc is not a lot though. Your eventdev will need
something like 10x that.
With link-time optimizations, the matching function call will go away.
I'm not sure how much auto-vectorization will happen. (The number quoted
above is without LTO.)
The event dispatcher as a separate library will result in an extra
function call across a shared object boundary, which is not exactly for
free. (The 10 cc are for static linking.)
>
> [1]
> https://developer.arm.com/documentation/den0018/a/NEON-and-VFP-Instruction-Summary/NEON-general-data-processing-instructions/VTBL
>
>> +
>> +/**
>> + * Function prototype for process callbacks.
>> + *
>> + * The process callbacks are used by the dispatcher to deliver
>> + * events for processing.
>> + *
>> + * @param event_dev_id
>> + * The originating event device id.
>> + *
>> + * @param event_port_id
>> + * The originating event port.
>> + *
>> + * @param events
>> + * Pointer to an array of events.
>> + *
>> + * @param num
>> + * The number of events in the @p events array.
>> + *
>> + * @param cb_data
>> + * The pointer supplied by the application in
>> + * rte_dispatcher_register().
>> + */
>> +
>> +typedef void
>> +(*rte_dispatcher_process_t)(uint8_t event_dev_id, uint8_t event_port_id,
>> + struct rte_event *events, uint16_t num,
>> + void *cb_data);
>
> Same as above comment, can event_port_id can be change to source_id?
>
>
>> +/**
>> + * Create a dispatcher with the specified id.
>> + *
>> + * @param id
>> + * An application-specified, unique (across all dispatcher
>> + * instances) identifier.
>> + *
>> + * @param event_dev_id
>> + * The identifier of the event device from which this dispatcher
>> + * will dequeue events.
>> + *
>> + * @return
>> + * - 0: Success
>> + * - <0: Error code on failure
>> + */
>> +__rte_experimental
>> +int
>> +rte_dispatcher_create(uint8_t id, uint8_t event_dev_id);
>
> Following could be used to abstract more dispatcher sources, like
>
> enum rte_dispatcher_source {
> RTE_DISPATCHER_SOURCE_EVENTDEV, // Use rte_event_dequeue() to
> pull the packet
> RTE_DISPATCHER_SOURCE_ETHDEV, // Use rte_ethdev_rx_burst() to
> pull the packet
> };
>
> struct rte_dispatcher_params {
> enum rte_dispatcher_source source;
> union {
> /* Valid when source == RTE_DISPATCHER_SOURCE_EVENTDEV */
> struct event_source {
> uint8_t event_dev_id;
> uin8_t event_port_id;
> };
> /* Valid when source == RTE_DISPATCHER_SOURCE_ETHDEV*/
> struct ethdev_source {
> uint16_t ethdev__dev_id;
> uin16_t ethdev_rx_queue_id;
> };
> }
> };
>
> rte_dispatcher_create(uint8_t id, struct rte_dispatcher_params *parms);
>
> I will stop reviewing at this point. Will review based on direction agree on.
Thanks!
On Thu, Sep 21, 2023 at 10:17 PM Mattias Rönnblom <hofors@lysator.liu.se> wrote:
>
> On 2023-09-19 12:58, Jerin Jacob wrote:
> > On Mon, Sep 4, 2023 at 6:39 PM Mattias Rönnblom
> > <mattias.ronnblom@ericsson.com> wrote:
> >>
> >> The purpose of the dispatcher library is to help reduce coupling in an
> >> Eventdev-based DPDK application.
> >>
> >> In addition, the dispatcher also provides a convenient and flexible
> >> way for the application to use service cores for application-level
> >> processing.
> >>
> >> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> >> Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
> >
> > High level architecture comment
> > --------------------------------
> >
> > 1) I think, we don't need tie this library ONLY to event dev
> > application. It can be used with poll mode as well,
> > that way traditiona pipeline application with ethdev as source could
> > use this library dispatch the packets.
> >
>
> They could potentially use a library *like this*, but I'm not sure it
> should be this library, or at least I think it should be a different API
> (better type checking, plus no obvious benefit of being more generic).
The only reason why I thought of this, It is cheap to do as all the logic
for comparing match actions, packet/event aggregation and calling the
action function is _same_
and better type checking can be added by separate callback for each source.
and allow more user to use this library.
I don't have a strong opinion of API semantic on this library API
other than the placement.
Feel free to ignore.
> Another option for a traditional app calling rte_eth_rx_burst() directly
> is to start using eventdev. :)
Yes. Those who can afford extra SW cores to emulate eventdev or has evendev HW.
>
> > We dont need to implement that first version but API can make room for
> > such abstractions.
> >
> > Based on my understanding in fast-path it has means to
> > a)Pull out the events using rte_event_dequeue()
> > b)Compare with registered match functions and call process upon match.
> >
> > if we abstract (a) as rte_dispatcher_source, We could pull from ethdev
> > via rte_eth_rx_burst() or
> > from ring via dequeue_burst API or so based on rte_dispatcher_source
> > selected for dispatch configuration
> > and we can use different sevice function pointers to have different service core
> > implementation without effecting performance each sources.
> >
>
> It could be generalized, for sure. I don't think it should be, at this
> point at least.
>
> Non-event dev events could - and at this point I'm leaning toward
> *should* - be consumed as a different DPDK service, but potentially on
> the same lcore.
>
> If you would want to prepare for a future with several different event
> sources, one could consider reintroducing the word "event" somewhere in
> the dispatcher's name. So you would have
> rte_event_dispatcher.h
> rte_eth_dispatcher.h
>
> or
>
> rte_dispatcher_event.h
> rte_dispatcher_eth.h
Yes.
> > High level cosmetic comment
> > ----------------------------------------------------
> > 1)Missing doxygen connection- See doc/api/doxy-api-index.md
> >
>
> rte_dispatcher.h is listed under **classification**, but this change is
> in the programming guide patch. I'll move it to the patch containing the
> header file.
>
>
> > Process related comment
> > ------------------------------------
> > 1) Documentation does not need need separate patch. All recent library
> > changes documentation in same file.
> > You could have doc and API header file as first patch and
> > implementation as subsequent patches.
> >
> >
>
> I'm not sure how this is an improvement. Can you elaborate? For me, it
> just seems like a change.
>
> Are there some guidelines on how to split a larger change into a patch
> set? A section on this matter in the contribution guide would be great.
In general, more patches easy review and attract more reviewers.
Last library was added to dpdk is lib/mldev. You can see
git log lib/mldev/
There operations like _create/free() etc made as separate patches.
I leave it up to you and Thomas as this library will be merged though main tree.
No strong opinion.
>
> >
> >> diff --git a/lib/dispatcher/rte_dispatcher.h b/lib/dispatcher/rte_dispatcher.h
> >> new file mode 100644
> >> index 0000000000..6712687a08
> >> --- /dev/null
> >> +++ b/lib/dispatcher/rte_dispatcher.h
> >> @@ -0,0 +1,480 @@
> >> +/* SPDX-License-Identifier: BSD-3-Clause
> >> + * Copyright(c) 2023 Ericsson AB
> >> + */
> >> +
> >> +#ifndef __RTE_DISPATCHER_H__
> >> +#define __RTE_DISPATCHER_H__
> >> +
> >
> >
> > All new API should be experimental. See
> > https://elixir.bootlin.com/dpdk/latest/source/lib/graph/rte_graph.h#L12
> > example.
> >
>
> Noted.
>
> >
> >> +/**
> >> + * @file
> >> + *
> >> + * RTE Dispatcher
> >> + *
> >> + * The purpose of the dispatcher is to help decouple different parts
> >> + * of an application (e.g., modules), sharing the same underlying
> >> + * event device.
> >> +
> >> +/**
> >> + * Function prototype for match callbacks.
> >> + *
> >> + * Match callbacks are used by an application to decide how the
> >> + * dispatcher distributes events to different parts of the
> >> + * application.
> >> + *
> >> + * The application is not expected to process the event at the point
> >> + * of the match call. Such matters should be deferred to the process
> >> + * callback invocation.
> >> + *
> >> + * The match callback may be used as an opportunity to prefetch data.
> >> + *
> >> + * @param event
> >> + * Pointer to event
> >> + *
> >> + * @param cb_data
> >> + * The pointer supplied by the application in
> >> + * rte_dispatcher_register().
> >> + *
> >> + * @return
> >> + * Returns true in case this events should be delivered (via
> >> + * the process callback), and false otherwise.
> >> + */
> >> +typedef bool
> >> +(*rte_dispatcher_match_t)(const struct rte_event *event, void *cb_data);
> >
> >
> > a) Can we use void* event, so that it can be used with mbuf or other
> > type by casting in the call back implementer.
> >
> > b) I was thinking, How we can avoid this function pointer and enable
> > more have better performance at architecture level.
> >
> > Both x86, ARM has vector instructions[1] to form a vector from various
> > offset from memory and compare N events
> > in one shot. That is, if express match data like offset = X has value
> > is Y and offset = X has value = A.
> > I know, it may not good existing application using this APIs. But I
> > believe, it will be more performance
> > effective. If make sense, you can adapt to this.(Something to think about)
> >
>
> There may be a future development where you try to shave off a few of
> the circa 10 clock cycles per event of overhead that the current
OK, as you wish.
On Mon, Sep 4, 2023 at 6:39 PM Mattias Rönnblom
<mattias.ronnblom@ericsson.com> wrote:
>
> The purpose of the dispatcher library is to help reduce coupling in an
> Eventdev-based DPDK application.
>
> In addition, the dispatcher also provides a convenient and flexible
> way for the application to use service cores for application-level
> processing.
>
> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
> Reviewed-by: Heng Wang <heng.wang@ericsson.com>
>
> +static inline void
> +evd_dispatch_events(struct rte_dispatcher *dispatcher,
> + struct rte_dispatcher_lcore *lcore,
> + struct rte_dispatcher_lcore_port *port,
> + struct rte_event *events, uint16_t num_events)
> +{
> + int i;
> + struct rte_event bursts[EVD_MAX_HANDLERS][num_events];
> + uint16_t burst_lens[EVD_MAX_HANDLERS] = { 0 };
> + uint16_t drop_count = 0;
> + uint16_t dispatch_count;
> + uint16_t dispatched = 0;
> +
> + for (i = 0; i < num_events; i++) {
> + struct rte_event *event = &events[i];
> + int handler_idx;
> +
> + handler_idx = evd_lookup_handler_idx(lcore, event);
> +
> + if (unlikely(handler_idx < 0)) {
> + drop_count++;
> + continue;
> + }
> +
> + bursts[handler_idx][burst_lens[handler_idx]] = *event;
Looks like it caching the event to accumulate ? If flow or queue is
configured as RTE_SCHED_TYPE_ORDERED?
Will it completely lose ordering as next rte_event_enqueue_burst will
release context?
Definition of RTE_SCHED_TYPE_ORDERED
#define RTE_SCHED_TYPE_ORDERED 0
/**< Ordered scheduling
*
* Events from an ordered flow of an event queue can be scheduled to multiple
* ports for concurrent processing while maintaining the original event order.
* This scheme enables the user to achieve high single flow throughput by
* avoiding SW synchronization for ordering between ports which bound to cores.
*
* The source flow ordering from an event queue is maintained when events are
* enqueued to their destination queue within the same ordered flow context.
* An event port holds the context until application call
* rte_event_dequeue_burst() from the same port, which implicitly releases
* the context.
* User may allow the scheduler to release the context earlier than that
* by invoking rte_event_enqueue_burst() with RTE_EVENT_OP_RELEASE operation.
*
* Events from the source queue appear in their original order when dequeued
* from a destination queue.
* Event ordering is based on the received event(s), but also other
* (newly allocated or stored) events are ordered when enqueued within the same
* ordered context. Events not enqueued (e.g. released or stored) within the
* context are considered missing from reordering and are skipped at this time
* (but can be ordered again within another context).
*
* @see rte_event_queue_setup(), rte_event_dequeue_burst(), RTE_EVENT_OP_RELEASE
*/
On 2023-09-21 20:36, Jerin Jacob wrote:
> On Mon, Sep 4, 2023 at 6:39 PM Mattias Rönnblom
> <mattias.ronnblom@ericsson.com> wrote:
>>
>> The purpose of the dispatcher library is to help reduce coupling in an
>> Eventdev-based DPDK application.
>>
>> In addition, the dispatcher also provides a convenient and flexible
>> way for the application to use service cores for application-level
>> processing.
>>
>> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>> Tested-by: Peter Nilsson <peter.j.nilsson@ericsson.com>
>> Reviewed-by: Heng Wang <heng.wang@ericsson.com>
>>
>
>> +static inline void
>> +evd_dispatch_events(struct rte_dispatcher *dispatcher,
>> + struct rte_dispatcher_lcore *lcore,
>> + struct rte_dispatcher_lcore_port *port,
>> + struct rte_event *events, uint16_t num_events)
>> +{
>> + int i;
>> + struct rte_event bursts[EVD_MAX_HANDLERS][num_events];
>> + uint16_t burst_lens[EVD_MAX_HANDLERS] = { 0 };
>> + uint16_t drop_count = 0;
>> + uint16_t dispatch_count;
>> + uint16_t dispatched = 0;
>> +
>> + for (i = 0; i < num_events; i++) {
>> + struct rte_event *event = &events[i];
>> + int handler_idx;
>> +
>> + handler_idx = evd_lookup_handler_idx(lcore, event);
>> +
>> + if (unlikely(handler_idx < 0)) {
>> + drop_count++;
>> + continue;
>> + }
>> +
>> + bursts[handler_idx][burst_lens[handler_idx]] = *event;
>
> Looks like it caching the event to accumulate ? If flow or queue is
> configured as RTE_SCHED_TYPE_ORDERED?
The ordering guarantees (and lack thereof) are covered in detail in the
programming guide.
"Delivery order" (the order the callbacks see the events) is maintained
only for events destined for the same handler.
I have considered adding a flags field to the create function, to then
in turn (now, or in the future) add an option to maintain strict
ordering between handlers. In my mind, and in the applications where
this pattern has been used in the past, the "clustering" of events going
to the same handler is a feature, not a bug, since it much improves
cache temporal locality and provides more opportunity for software
prefetching/preloading. (Prefetching may be done already in the match
function.)
If your event device does clustering already, or if the application
implements this pattern already, you will obviously see no gains. If
neither of those are true, the application will likely suffer fewer
cache misses, much outweighing the tiny bit of extra processing required
in the event dispatcher.
This reshuffling ("clustering") of events is the only thing I think
could be offloaded to hardware. The event device is already free to
reshuffle events as long as it conforms to whatever ordering guarantees
the eventdev scheduling types in questions require, but the event
dispatcher relaxes those further, and give further hints to the
platform, what events are actually related.
> Will it completely lose ordering as next rte_event_enqueue_burst will
> release context? >
It is the dequeue operation that will release the context (provided
"implicit release" is not disabled). See the documentation you quote below.
(Total) ordering is guaranteed between dequeue bursts.
>
> Definition of RTE_SCHED_TYPE_ORDERED
>
> #define RTE_SCHED_TYPE_ORDERED 0
> /**< Ordered scheduling
> *
> * Events from an ordered flow of an event queue can be scheduled to multiple
> * ports for concurrent processing while maintaining the original event order.
> * This scheme enables the user to achieve high single flow throughput by
> * avoiding SW synchronization for ordering between ports which bound to cores.
> *
> * The source flow ordering from an event queue is maintained when events are
> * enqueued to their destination queue within the same ordered flow context.
> * An event port holds the context until application call
> * rte_event_dequeue_burst() from the same port, which implicitly releases
> * the context.
> * User may allow the scheduler to release the context earlier than that
> * by invoking rte_event_enqueue_burst() with RTE_EVENT_OP_RELEASE operation.
> *
> * Events from the source queue appear in their original order when dequeued
> * from a destination queue.
> * Event ordering is based on the received event(s), but also other
> * (newly allocated or stored) events are ordered when enqueued within the same
> * ordered context. Events not enqueued (e.g. released or stored) within the
> * context are considered missing from reordering and are skipped at this time
> * (but can be ordered again within another context).
> *
> * @see rte_event_queue_setup(), rte_event_dequeue_burst(), RTE_EVENT_OP_RELEASE
> */
@@ -1726,6 +1726,9 @@ M: Nithin Dabilpuram <ndabilpuram@marvell.com>
M: Pavan Nikhilesh <pbhagavatula@marvell.com>
F: lib/node/
+Dispatcher - EXPERIMENTAL
+M: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
+F: lib/dispatcher/
Test Applications
-----------------
new file mode 100644
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Ericsson AB
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+sources = files(
+ 'rte_dispatcher.c',
+)
+headers = files(
+ 'rte_dispatcher.h',
+)
+
+deps += ['eventdev']
new file mode 100644
@@ -0,0 +1,791 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Ericsson AB
+ */
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <rte_branch_prediction.h>
+#include <rte_common.h>
+#include <rte_lcore.h>
+#include <rte_random.h>
+#include <rte_service_component.h>
+
+#include "eventdev_pmd.h"
+
+#include <rte_dispatcher.h>
+
+#define EVD_MAX_PORTS_PER_LCORE 4
+#define EVD_MAX_HANDLERS 32
+#define EVD_MAX_FINALIZERS 16
+#define EVD_AVG_PRIO_INTERVAL 2000
+
+struct rte_dispatcher_lcore_port {
+ uint8_t port_id;
+ uint16_t batch_size;
+ uint64_t timeout;
+};
+
+struct rte_dispatcher_handler {
+ int id;
+ rte_dispatcher_match_t match_fun;
+ void *match_data;
+ rte_dispatcher_process_t process_fun;
+ void *process_data;
+};
+
+struct rte_dispatcher_finalizer {
+ int id;
+ rte_dispatcher_finalize_t finalize_fun;
+ void *finalize_data;
+};
+
+struct rte_dispatcher_lcore {
+ uint8_t num_ports;
+ uint16_t num_handlers;
+ int32_t prio_count;
+ struct rte_dispatcher_lcore_port ports[EVD_MAX_PORTS_PER_LCORE];
+ struct rte_dispatcher_handler handlers[EVD_MAX_HANDLERS];
+ struct rte_dispatcher_stats stats;
+} __rte_cache_aligned;
+
+struct rte_dispatcher {
+ uint8_t id;
+ uint8_t event_dev_id;
+ int socket_id;
+ uint32_t service_id;
+ struct rte_dispatcher_lcore lcores[RTE_MAX_LCORE];
+ uint16_t num_finalizers;
+ struct rte_dispatcher_finalizer finalizers[EVD_MAX_FINALIZERS];
+};
+
+static struct rte_dispatcher *dispatchers[UINT8_MAX];
+
+static bool
+evd_has_dispatcher(uint8_t id)
+{
+ return dispatchers[id] != NULL;
+}
+
+static struct rte_dispatcher *
+evd_get_dispatcher(uint8_t id)
+{
+ return dispatchers[id];
+}
+
+static void
+evd_set_dispatcher(uint8_t id, struct rte_dispatcher *dispatcher)
+{
+ dispatchers[id] = dispatcher;
+}
+
+#define EVD_VALID_ID_OR_RET_EINVAL(id) \
+ do { \
+ if (unlikely(!evd_has_dispatcher(id))) { \
+ RTE_EDEV_LOG_ERR("Invalid dispatcher id %d\n", id); \
+ return -EINVAL; \
+ } \
+ } while (0)
+
+static int
+evd_lookup_handler_idx(struct rte_dispatcher_lcore *lcore,
+ const struct rte_event *event)
+{
+ uint16_t i;
+
+ for (i = 0; i < lcore->num_handlers; i++) {
+ struct rte_dispatcher_handler *handler =
+ &lcore->handlers[i];
+
+ if (handler->match_fun(event, handler->match_data))
+ return i;
+ }
+
+ return -1;
+}
+
+static void
+evd_prioritize_handler(struct rte_dispatcher_lcore *lcore,
+ int handler_idx)
+{
+ struct rte_dispatcher_handler tmp;
+
+ if (handler_idx == 0)
+ return;
+
+ /* Let the lucky handler "bubble" up the list */
+
+ tmp = lcore->handlers[handler_idx - 1];
+
+ lcore->handlers[handler_idx - 1] = lcore->handlers[handler_idx];
+
+ lcore->handlers[handler_idx] = tmp;
+}
+
+static inline void
+evd_consider_prioritize_handler(struct rte_dispatcher_lcore *lcore,
+ int handler_idx, uint16_t handler_events)
+{
+ lcore->prio_count -= handler_events;
+
+ if (unlikely(lcore->prio_count <= 0)) {
+ evd_prioritize_handler(lcore, handler_idx);
+
+ /*
+ * Randomize the interval in the unlikely case
+ * the traffic follow some very strict pattern.
+ */
+ lcore->prio_count =
+ rte_rand_max(EVD_AVG_PRIO_INTERVAL) +
+ EVD_AVG_PRIO_INTERVAL / 2;
+ }
+}
+
+static inline void
+evd_dispatch_events(struct rte_dispatcher *dispatcher,
+ struct rte_dispatcher_lcore *lcore,
+ struct rte_dispatcher_lcore_port *port,
+ struct rte_event *events, uint16_t num_events)
+{
+ int i;
+ struct rte_event bursts[EVD_MAX_HANDLERS][num_events];
+ uint16_t burst_lens[EVD_MAX_HANDLERS] = { 0 };
+ uint16_t drop_count = 0;
+ uint16_t dispatch_count;
+ uint16_t dispatched = 0;
+
+ for (i = 0; i < num_events; i++) {
+ struct rte_event *event = &events[i];
+ int handler_idx;
+
+ handler_idx = evd_lookup_handler_idx(lcore, event);
+
+ if (unlikely(handler_idx < 0)) {
+ drop_count++;
+ continue;
+ }
+
+ bursts[handler_idx][burst_lens[handler_idx]] = *event;
+ burst_lens[handler_idx]++;
+ }
+
+ dispatch_count = num_events - drop_count;
+
+ for (i = 0; i < lcore->num_handlers &&
+ dispatched < dispatch_count; i++) {
+ struct rte_dispatcher_handler *handler =
+ &lcore->handlers[i];
+ uint16_t len = burst_lens[i];
+
+ if (len == 0)
+ continue;
+
+ handler->process_fun(dispatcher->event_dev_id, port->port_id,
+ bursts[i], len, handler->process_data);
+
+ dispatched += len;
+
+ /*
+ * Safe, since any reshuffling will only involve
+ * already-processed handlers.
+ */
+ evd_consider_prioritize_handler(lcore, i, len);
+ }
+
+ lcore->stats.ev_batch_count++;
+ lcore->stats.ev_dispatch_count += dispatch_count;
+ lcore->stats.ev_drop_count += drop_count;
+
+ for (i = 0; i < dispatcher->num_finalizers; i++) {
+ struct rte_dispatcher_finalizer *finalizer =
+ &dispatcher->finalizers[i];
+
+ finalizer->finalize_fun(dispatcher->event_dev_id,
+ port->port_id,
+ finalizer->finalize_data);
+ }
+}
+
+static __rte_always_inline uint16_t
+evd_port_dequeue(struct rte_dispatcher *dispatcher,
+ struct rte_dispatcher_lcore *lcore,
+ struct rte_dispatcher_lcore_port *port)
+{
+ uint16_t batch_size = port->batch_size;
+ struct rte_event events[batch_size];
+ uint16_t n;
+
+ n = rte_event_dequeue_burst(dispatcher->event_dev_id, port->port_id,
+ events, batch_size, port->timeout);
+
+ if (likely(n > 0))
+ evd_dispatch_events(dispatcher, lcore, port, events, n);
+
+ lcore->stats.poll_count++;
+
+ return n;
+}
+
+static __rte_always_inline uint16_t
+evd_lcore_process(struct rte_dispatcher *dispatcher,
+ struct rte_dispatcher_lcore *lcore)
+{
+ uint16_t i;
+ uint16_t event_count = 0;
+
+ for (i = 0; i < lcore->num_ports; i++) {
+ struct rte_dispatcher_lcore_port *port =
+ &lcore->ports[i];
+
+ event_count += evd_port_dequeue(dispatcher, lcore, port);
+ }
+
+ return event_count;
+}
+
+static int32_t
+evd_process(void *userdata)
+{
+ struct rte_dispatcher *dispatcher = userdata;
+ unsigned int lcore_id = rte_lcore_id();
+ struct rte_dispatcher_lcore *lcore =
+ &dispatcher->lcores[lcore_id];
+ uint64_t event_count;
+
+ event_count = evd_lcore_process(dispatcher, lcore);
+
+ if (unlikely(event_count == 0))
+ return -EAGAIN;
+
+ return 0;
+}
+
+static int
+evd_service_register(struct rte_dispatcher *dispatcher)
+{
+ struct rte_service_spec service = {
+ .callback = evd_process,
+ .callback_userdata = dispatcher,
+ .capabilities = RTE_SERVICE_CAP_MT_SAFE,
+ .socket_id = dispatcher->socket_id
+ };
+ int rc;
+
+ snprintf(service.name, RTE_SERVICE_NAME_MAX - 1, "evd_%d",
+ dispatcher->id);
+
+ rc = rte_service_component_register(&service, &dispatcher->service_id);
+
+ if (rc)
+ RTE_EDEV_LOG_ERR("Registration of dispatcher service "
+ "%s failed with error code %d\n",
+ service.name, rc);
+
+ return rc;
+}
+
+static int
+evd_service_unregister(struct rte_dispatcher *dispatcher)
+{
+ int rc;
+
+ rc = rte_service_component_unregister(dispatcher->service_id);
+
+ if (rc)
+ RTE_EDEV_LOG_ERR("Unregistration of dispatcher service "
+ "failed with error code %d\n", rc);
+
+ return rc;
+}
+
+int
+rte_dispatcher_create(uint8_t id, uint8_t event_dev_id)
+{
+ int socket_id;
+ struct rte_dispatcher *dispatcher;
+ int rc;
+
+ if (evd_has_dispatcher(id)) {
+ RTE_EDEV_LOG_ERR("Dispatcher with id %d already exists\n",
+ id);
+ return -EEXIST;
+ }
+
+ socket_id = rte_event_dev_socket_id(event_dev_id);
+
+ dispatcher =
+ rte_malloc_socket("dispatcher", sizeof(struct rte_dispatcher),
+ RTE_CACHE_LINE_SIZE, socket_id);
+
+ if (dispatcher == NULL) {
+ RTE_EDEV_LOG_ERR("Unable to allocate memory for dispatcher\n");
+ return -ENOMEM;
+ }
+
+ *dispatcher = (struct rte_dispatcher) {
+ .id = id,
+ .event_dev_id = event_dev_id,
+ .socket_id = socket_id
+ };
+
+ rc = evd_service_register(dispatcher);
+
+ if (rc < 0) {
+ rte_free(dispatcher);
+ return rc;
+ }
+
+ evd_set_dispatcher(id, dispatcher);
+
+ return 0;
+}
+
+int
+rte_dispatcher_free(uint8_t id)
+{
+ struct rte_dispatcher *dispatcher;
+ int rc;
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+ rc = evd_service_unregister(dispatcher);
+
+ if (rc)
+ return rc;
+
+ evd_set_dispatcher(id, NULL);
+
+ rte_free(dispatcher);
+
+ return 0;
+}
+
+int
+rte_dispatcher_service_id_get(uint8_t id, uint32_t *service_id)
+{
+ struct rte_dispatcher *dispatcher;
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+ *service_id = dispatcher->service_id;
+
+ return 0;
+}
+
+static int
+lcore_port_index(struct rte_dispatcher_lcore *lcore,
+ uint8_t event_port_id)
+{
+ uint16_t i;
+
+ for (i = 0; i < lcore->num_ports; i++) {
+ struct rte_dispatcher_lcore_port *port =
+ &lcore->ports[i];
+
+ if (port->port_id == event_port_id)
+ return i;
+ }
+
+ return -1;
+}
+
+int
+rte_dispatcher_bind_port_to_lcore(uint8_t id, uint8_t event_port_id,
+ uint16_t batch_size, uint64_t timeout,
+ unsigned int lcore_id)
+{
+ struct rte_dispatcher *dispatcher;
+ struct rte_dispatcher_lcore *lcore;
+ struct rte_dispatcher_lcore_port *port;
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+ lcore = &dispatcher->lcores[lcore_id];
+
+ if (lcore->num_ports == EVD_MAX_PORTS_PER_LCORE)
+ return -ENOMEM;
+
+ if (lcore_port_index(lcore, event_port_id) >= 0)
+ return -EEXIST;
+
+ port = &lcore->ports[lcore->num_ports];
+
+ *port = (struct rte_dispatcher_lcore_port) {
+ .port_id = event_port_id,
+ .batch_size = batch_size,
+ .timeout = timeout
+ };
+
+ lcore->num_ports++;
+
+ return 0;
+}
+
+int
+rte_dispatcher_unbind_port_from_lcore(uint8_t id, uint8_t event_port_id,
+ unsigned int lcore_id)
+{
+ struct rte_dispatcher *dispatcher;
+ struct rte_dispatcher_lcore *lcore;
+ int port_idx;
+ struct rte_dispatcher_lcore_port *port;
+ struct rte_dispatcher_lcore_port *last;
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+ lcore = &dispatcher->lcores[lcore_id];
+
+ port_idx = lcore_port_index(lcore, event_port_id);
+
+ if (port_idx < 0)
+ return -ENOENT;
+
+ port = &lcore->ports[port_idx];
+ last = &lcore->ports[lcore->num_ports - 1];
+
+ if (port != last)
+ *port = *last;
+
+ lcore->num_ports--;
+
+ return 0;
+}
+
+static struct rte_dispatcher_handler*
+evd_lcore_get_handler_by_id(struct rte_dispatcher_lcore *lcore,
+ int handler_id)
+{
+ uint16_t i;
+
+ for (i = 0; i < lcore->num_handlers; i++) {
+ struct rte_dispatcher_handler *handler =
+ &lcore->handlers[i];
+
+ if (handler->id == handler_id)
+ return handler;
+ }
+
+ return NULL;
+}
+
+static int
+evd_alloc_handler_id(struct rte_dispatcher *dispatcher)
+{
+ int handler_id = 0;
+ struct rte_dispatcher_lcore *reference_lcore =
+ &dispatcher->lcores[0];
+
+ if (reference_lcore->num_handlers == EVD_MAX_HANDLERS)
+ return -1;
+
+ while (evd_lcore_get_handler_by_id(reference_lcore, handler_id) != NULL)
+ handler_id++;
+
+ return handler_id;
+}
+
+static void
+evd_lcore_install_handler(struct rte_dispatcher_lcore *lcore,
+ const struct rte_dispatcher_handler *handler)
+{
+ int handler_idx = lcore->num_handlers;
+
+ lcore->handlers[handler_idx] = *handler;
+ lcore->num_handlers++;
+}
+
+static void
+evd_install_handler(struct rte_dispatcher *dispatcher,
+ const struct rte_dispatcher_handler *handler)
+{
+ int i;
+
+ for (i = 0; i < RTE_MAX_LCORE; i++) {
+ struct rte_dispatcher_lcore *lcore =
+ &dispatcher->lcores[i];
+ evd_lcore_install_handler(lcore, handler);
+ }
+}
+
+int
+rte_dispatcher_register(uint8_t id,
+ rte_dispatcher_match_t match_fun,
+ void *match_data,
+ rte_dispatcher_process_t process_fun,
+ void *process_data)
+{
+ struct rte_dispatcher *dispatcher;
+ struct rte_dispatcher_handler handler = {
+ .match_fun = match_fun,
+ .match_data = match_data,
+ .process_fun = process_fun,
+ .process_data = process_data
+ };
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+ handler.id = evd_alloc_handler_id(dispatcher);
+
+ if (handler.id < 0)
+ return -ENOMEM;
+
+ evd_install_handler(dispatcher, &handler);
+
+ return handler.id;
+}
+
+static int
+evd_lcore_uninstall_handler(struct rte_dispatcher_lcore *lcore,
+ int handler_id)
+{
+ struct rte_dispatcher_handler *unreg_handler;
+ int handler_idx;
+ uint16_t last_idx;
+
+ unreg_handler = evd_lcore_get_handler_by_id(lcore, handler_id);
+
+ if (unreg_handler == NULL)
+ return -EINVAL;
+
+ handler_idx = &lcore->handlers[0] - unreg_handler;
+
+ last_idx = lcore->num_handlers - 1;
+
+ if (handler_idx != last_idx) {
+ /* move all handlers to maintain handler order */
+ int n = last_idx - handler_idx;
+ memmove(unreg_handler, unreg_handler + 1,
+ sizeof(struct rte_dispatcher_handler) * n);
+ }
+
+ lcore->num_handlers--;
+
+ return 0;
+}
+
+static int
+evd_uninstall_handler(struct rte_dispatcher *dispatcher,
+ int handler_id)
+{
+ unsigned int lcore_id;
+
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ struct rte_dispatcher_lcore *lcore =
+ &dispatcher->lcores[lcore_id];
+ int rc;
+
+ rc = evd_lcore_uninstall_handler(lcore, handler_id);
+
+ if (rc < 0)
+ return rc;
+ }
+
+ return 0;
+}
+
+int
+rte_dispatcher_unregister(uint8_t id, int handler_id)
+{
+ struct rte_dispatcher *dispatcher;
+ int rc;
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+ rc = evd_uninstall_handler(dispatcher, handler_id);
+
+ return rc;
+}
+
+static struct rte_dispatcher_finalizer*
+evd_get_finalizer_by_id(struct rte_dispatcher *dispatcher,
+ int handler_id)
+{
+ int i;
+
+ for (i = 0; i < dispatcher->num_finalizers; i++) {
+ struct rte_dispatcher_finalizer *finalizer =
+ &dispatcher->finalizers[i];
+
+ if (finalizer->id == handler_id)
+ return finalizer;
+ }
+
+ return NULL;
+}
+
+static int
+evd_alloc_finalizer_id(struct rte_dispatcher *dispatcher)
+{
+ int finalizer_id = 0;
+
+ while (evd_get_finalizer_by_id(dispatcher, finalizer_id) != NULL)
+ finalizer_id++;
+
+ return finalizer_id;
+}
+
+static struct rte_dispatcher_finalizer *
+evd_alloc_finalizer(struct rte_dispatcher *dispatcher)
+{
+ int finalizer_idx;
+ struct rte_dispatcher_finalizer *finalizer;
+
+ if (dispatcher->num_finalizers == EVD_MAX_FINALIZERS)
+ return NULL;
+
+ finalizer_idx = dispatcher->num_finalizers;
+ finalizer = &dispatcher->finalizers[finalizer_idx];
+
+ finalizer->id = evd_alloc_finalizer_id(dispatcher);
+
+ dispatcher->num_finalizers++;
+
+ return finalizer;
+}
+
+int
+rte_dispatcher_finalize_register(uint8_t id,
+ rte_dispatcher_finalize_t finalize_fun,
+ void *finalize_data)
+{
+ struct rte_dispatcher *dispatcher;
+ struct rte_dispatcher_finalizer *finalizer;
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+ finalizer = evd_alloc_finalizer(dispatcher);
+
+ if (finalizer == NULL)
+ return -ENOMEM;
+
+ finalizer->finalize_fun = finalize_fun;
+ finalizer->finalize_data = finalize_data;
+
+ return finalizer->id;
+}
+
+int
+rte_dispatcher_finalize_unregister(uint8_t id, int handler_id)
+{
+ struct rte_dispatcher *dispatcher;
+ struct rte_dispatcher_finalizer *unreg_finalizer;
+ int finalizer_idx;
+ uint16_t last_idx;
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+ unreg_finalizer = evd_get_finalizer_by_id(dispatcher, handler_id);
+
+ if (unreg_finalizer == NULL)
+ return -EINVAL;
+
+ finalizer_idx = &dispatcher->finalizers[0] - unreg_finalizer;
+
+ last_idx = dispatcher->num_finalizers - 1;
+
+ if (finalizer_idx != last_idx) {
+ /* move all finalizers to maintain order */
+ int n = last_idx - finalizer_idx;
+ memmove(unreg_finalizer, unreg_finalizer + 1,
+ sizeof(struct rte_dispatcher_finalizer) * n);
+ }
+
+ dispatcher->num_finalizers--;
+
+ return 0;
+}
+
+static int
+evd_set_service_runstate(uint8_t id, int state)
+{
+ struct rte_dispatcher *dispatcher;
+ int rc;
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+ rc = rte_service_component_runstate_set(dispatcher->service_id,
+ state);
+
+ if (rc != 0) {
+ RTE_EDEV_LOG_ERR("Unexpected error %d occurred while setting "
+ "service component run state to %d\n", rc,
+ state);
+ RTE_ASSERT(0);
+ }
+
+ return 0;
+}
+
+int
+rte_dispatcher_start(uint8_t id)
+{
+ return evd_set_service_runstate(id, 1);
+}
+
+int
+rte_dispatcher_stop(uint8_t id)
+{
+ return evd_set_service_runstate(id, 0);
+}
+
+static void
+evd_aggregate_stats(struct rte_dispatcher_stats *result,
+ const struct rte_dispatcher_stats *part)
+{
+ result->poll_count += part->poll_count;
+ result->ev_batch_count += part->ev_batch_count;
+ result->ev_dispatch_count += part->ev_dispatch_count;
+ result->ev_drop_count += part->ev_drop_count;
+}
+
+int
+rte_dispatcher_stats_get(uint8_t id,
+ struct rte_dispatcher_stats *stats)
+{
+ struct rte_dispatcher *dispatcher;
+ unsigned int lcore_id;
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+ *stats = (struct rte_dispatcher_stats) {};
+
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ struct rte_dispatcher_lcore *lcore =
+ &dispatcher->lcores[lcore_id];
+
+ evd_aggregate_stats(stats, &lcore->stats);
+ }
+
+ return 0;
+}
+
+int
+rte_dispatcher_stats_reset(uint8_t id)
+{
+ struct rte_dispatcher *dispatcher;
+ unsigned int lcore_id;
+
+ EVD_VALID_ID_OR_RET_EINVAL(id);
+ dispatcher = evd_get_dispatcher(id);
+
+
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ struct rte_dispatcher_lcore *lcore =
+ &dispatcher->lcores[lcore_id];
+
+ lcore->stats = (struct rte_dispatcher_stats) {};
+ }
+
+ return 0;
+
+}
new file mode 100644
@@ -0,0 +1,480 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Ericsson AB
+ */
+
+#ifndef __RTE_DISPATCHER_H__
+#define __RTE_DISPATCHER_H__
+
+/**
+ * @file
+ *
+ * RTE Dispatcher
+ *
+ * The purpose of the dispatcher is to help decouple different parts
+ * of an application (e.g., modules), sharing the same underlying
+ * event device.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <rte_eventdev.h>
+
+/**
+ * Function prototype for match callbacks.
+ *
+ * Match callbacks are used by an application to decide how the
+ * dispatcher distributes events to different parts of the
+ * application.
+ *
+ * The application is not expected to process the event at the point
+ * of the match call. Such matters should be deferred to the process
+ * callback invocation.
+ *
+ * The match callback may be used as an opportunity to prefetch data.
+ *
+ * @param event
+ * Pointer to event
+ *
+ * @param cb_data
+ * The pointer supplied by the application in
+ * rte_dispatcher_register().
+ *
+ * @return
+ * Returns true in case this events should be delivered (via
+ * the process callback), and false otherwise.
+ */
+typedef bool
+(*rte_dispatcher_match_t)(const struct rte_event *event, void *cb_data);
+
+/**
+ * Function prototype for process callbacks.
+ *
+ * The process callbacks are used by the dispatcher to deliver
+ * events for processing.
+ *
+ * @param event_dev_id
+ * The originating event device id.
+ *
+ * @param event_port_id
+ * The originating event port.
+ *
+ * @param events
+ * Pointer to an array of events.
+ *
+ * @param num
+ * The number of events in the @p events array.
+ *
+ * @param cb_data
+ * The pointer supplied by the application in
+ * rte_dispatcher_register().
+ */
+
+typedef void
+(*rte_dispatcher_process_t)(uint8_t event_dev_id, uint8_t event_port_id,
+ struct rte_event *events, uint16_t num,
+ void *cb_data);
+
+/**
+ * Function prototype for finalize callbacks.
+ *
+ * The finalize callbacks are used by the dispatcher to notify the
+ * application it has delivered all events from a particular batch
+ * dequeued from the event device.
+ *
+ * @param event_dev_id
+ * The originating event device id.
+ *
+ * @param event_port_id
+ * The originating event port.
+ *
+ * @param cb_data
+ * The pointer supplied by the application in
+ * rte_dispatcher_finalize_register().
+ */
+
+typedef void
+(*rte_dispatcher_finalize_t)(uint8_t event_dev_id, uint8_t event_port_id,
+ void *cb_data);
+
+/**
+ * Dispatcher statistics
+ */
+struct rte_dispatcher_stats {
+ uint64_t poll_count;
+ /**< Number of event dequeue calls made toward the event device. */
+ uint64_t ev_batch_count;
+ /**< Number of non-empty event batches dequeued from event device.*/
+ uint64_t ev_dispatch_count;
+ /**< Number of events dispatched to a handler.*/
+ uint64_t ev_drop_count;
+ /**< Number of events dropped because no handler was found. */
+};
+
+/**
+ * Create a dispatcher with the specified id.
+ *
+ * @param id
+ * An application-specified, unique (across all dispatcher
+ * instances) identifier.
+ *
+ * @param event_dev_id
+ * The identifier of the event device from which this dispatcher
+ * will dequeue events.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int
+rte_dispatcher_create(uint8_t id, uint8_t event_dev_id);
+
+/**
+ * Free a dispatcher.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int
+rte_dispatcher_free(uint8_t id);
+
+/**
+ * Retrieve the service identifier of a dispatcher.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @param [out] service_id
+ * A pointer to a caller-supplied buffer where the dispatcher's
+ * service id will be stored.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int
+rte_dispatcher_service_id_get(uint8_t id, uint32_t *service_id);
+
+/**
+ * Binds an event device port to a specific lcore on the specified
+ * dispatcher.
+ *
+ * This function configures the event port id to be used by the event
+ * dispatcher service, if run on the specified lcore.
+ *
+ * Multiple event device ports may be bound to the same lcore. A
+ * particular port must not be bound to more than one lcore.
+ *
+ * If the dispatcher service is mapped (with rte_service_map_lcore_set())
+ * to a lcore to which no ports are bound, the service function will be a
+ * no-operation.
+ *
+ * This function may be called by any thread (including unregistered
+ * non-EAL threads), but not while the dispatcher is running on lcore
+ * specified by @c lcore_id.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @param event_port_id
+ * The event device port identifier.
+ *
+ * @param batch_size
+ * The batch size to use in rte_event_dequeue_burst(), for the
+ * configured event device port and lcore.
+ *
+ * @param timeout
+ * The timeout parameter to use in rte_event_dequeue_burst(), for the
+ * configured event device port and lcore.
+ *
+ * @param lcore_id
+ * The lcore by which this event port will be used.
+ *
+ * @return
+ * - 0: Success
+ * - -ENOMEM: Unable to allocate sufficient resources.
+ * - -EEXISTS: Event port is already configured.
+ * - -EINVAL: Invalid arguments.
+ */
+__rte_experimental
+int
+rte_dispatcher_bind_port_to_lcore(uint8_t id, uint8_t event_port_id,
+ uint16_t batch_size, uint64_t timeout,
+ unsigned int lcore_id);
+
+/**
+ * Unbind an event device port from a specific lcore.
+ *
+ * This function may be called by any thread (including unregistered
+ * non-EAL threads), but not while the dispatcher is running on
+ * lcore specified by @c lcore_id.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @param event_port_id
+ * The event device port identifier.
+ *
+ * @param lcore_id
+ * The lcore which was using this event port.
+ *
+ * @return
+ * - 0: Success
+ * - -EINVAL: Invalid @c id.
+ * - -ENOENT: Event port id not bound to this @c lcore_id.
+ */
+__rte_experimental
+int
+rte_dispatcher_unbind_port_from_lcore(uint8_t id, uint8_t event_port_id,
+ unsigned int lcore_id);
+
+/**
+ * Register an event handler.
+ *
+ * The match callback function is used to select if a particular event
+ * should be delivered, using the corresponding process callback
+ * function.
+ *
+ * The reason for having two distinct steps is to allow the dispatcher
+ * to deliver all events as a batch. This in turn will cause
+ * processing of a particular kind of events to happen in a
+ * back-to-back manner, improving cache locality.
+ *
+ * The list of handler callback functions is shared among all lcores,
+ * but will only be executed on lcores which has an eventdev port
+ * bound to them, and which are running the dispatcher service.
+ *
+ * An event is delivered to at most one handler. Events where no
+ * handler is found are dropped.
+ *
+ * The application must not depend on the order of which the match
+ * functions are invoked.
+ *
+ * Ordering of events is not guaranteed to be maintained between
+ * different deliver callbacks. For example, suppose there are two
+ * callbacks registered, matching different subsets of events arriving
+ * on an atomic queue. A batch of events [ev0, ev1, ev2] are dequeued
+ * on a particular port, all pertaining to the same flow. The match
+ * callback for registration A returns true for ev0 and ev2, and the
+ * matching function for registration B for ev1. In that scenario, the
+ * dispatcher may choose to deliver first [ev0, ev2] using A's deliver
+ * function, and then [ev1] to B - or vice versa.
+ *
+ * rte_dispatcher_register() may be called by any thread
+ * (including unregistered non-EAL threads), but not while the event
+ * dispatcher is running on any service lcore.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @param match_fun
+ * The match callback function.
+ *
+ * @param match_cb_data
+ * A pointer to some application-specific opaque data (or NULL),
+ * which is supplied back to the application when match_fun is
+ * called.
+ *
+ * @param process_fun
+ * The process callback function.
+ *
+ * @param process_cb_data
+ * A pointer to some application-specific opaque data (or NULL),
+ * which is supplied back to the application when process_fun is
+ * called.
+ *
+ * @return
+ * - >= 0: The identifier for this registration.
+ * - -ENOMEM: Unable to allocate sufficient resources.
+ */
+__rte_experimental
+int
+rte_dispatcher_register(uint8_t id,
+ rte_dispatcher_match_t match_fun,
+ void *match_cb_data,
+ rte_dispatcher_process_t process_fun,
+ void *process_cb_data);
+
+/**
+ * Unregister an event handler.
+ *
+ * This function may be called by any thread (including unregistered
+ * non-EAL threads), but not while the dispatcher is running on
+ * any service lcore.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @param handler_id
+ * The handler registration id returned by the original
+ * rte_dispatcher_register() call.
+ *
+ * @return
+ * - 0: Success
+ * - -EINVAL: The @c id and/or the @c handler_id parameter was invalid.
+ */
+__rte_experimental
+int
+rte_dispatcher_unregister(uint8_t id, int handler_id);
+
+/**
+ * Register a finalize callback function.
+ *
+ * An application may optionally install one or more finalize
+ * callbacks.
+ *
+ * All finalize callbacks are invoked by the dispatcher when a
+ * complete batch of events (retrieve using rte_event_dequeue_burst())
+ * have been delivered to the application (or have been dropped).
+ *
+ * The finalize callback is not tied to any particular handler.
+ *
+ * The finalize callback provides an opportunity for the application
+ * to do per-batch processing. One case where this may be useful is if
+ * an event output buffer is used, and is shared among several
+ * handlers. In such a case, proper output buffer flushing may be
+ * assured using a finalize callback.
+ *
+ * rte_dispatcher_finalize_register() may be called by any thread
+ * (including unregistered non-EAL threads), but not while the
+ * dispatcher is running on any service lcore.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @param finalize_fun
+ * The function called after completing the processing of a
+ * dequeue batch.
+ *
+ * @param finalize_data
+ * A pointer to some application-specific opaque data (or NULL),
+ * which is supplied back to the application when @c finalize_fun is
+ * called.
+ *
+ * @return
+ * - >= 0: The identifier for this registration.
+ * - -ENOMEM: Unable to allocate sufficient resources.
+ */
+__rte_experimental
+int
+rte_dispatcher_finalize_register(uint8_t id,
+ rte_dispatcher_finalize_t finalize_fun,
+ void *finalize_data);
+
+/**
+ * Unregister a finalize callback.
+ *
+ * This function may be called by any thread (including unregistered
+ * non-EAL threads), but not while the dispatcher is running on
+ * any service lcore.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @param reg_id
+ * The finalize registration id returned by the original
+ * rte_dispatcher_finalize_register() call.
+ *
+ * @return
+ * - 0: Success
+ * - -EINVAL: The @c id and/or the @c reg_id parameter was invalid.
+ */
+__rte_experimental
+int
+rte_dispatcher_finalize_unregister(uint8_t id, int reg_id);
+
+/**
+ * Start a dispatcher instance.
+ *
+ * Enables the dispatcher service.
+ *
+ * The underlying event device must have been started prior to calling
+ * rte_dispatcher_start().
+ *
+ * For the dispatcher to actually perform work (i.e., dispatch
+ * events), its service must have been mapped to one or more service
+ * lcores, and its service run state set to '1'. A dispatcher's
+ * service is retrieved using rte_dispatcher_service_id_get().
+ *
+ * Each service lcore to which the dispatcher is mapped should
+ * have at least one event port configured. Such configuration is
+ * performed by calling rte_dispatcher_bind_port_to_lcore(), prior to
+ * starting the dispatcher.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @return
+ * - 0: Success
+ * - -EINVAL: Invalid @c id.
+ */
+__rte_experimental
+int
+rte_dispatcher_start(uint8_t id);
+
+/**
+ * Stop a running dispatcher instance.
+ *
+ * Disables the dispatcher service.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @return
+ * - 0: Success
+ * - -EINVAL: Invalid @c id.
+ */
+__rte_experimental
+int
+rte_dispatcher_stop(uint8_t id);
+
+/**
+ * Retrieve statistics for a dispatcher instance.
+ *
+ * This function is MT safe and may be called by any thread
+ * (including unregistered non-EAL threads).
+ *
+ * @param id
+ * The dispatcher identifier.
+ * @param[out] stats
+ * A pointer to a structure to fill with statistics.
+ * @return
+ * - 0: Success
+ * - -EINVAL: The @c id parameter was invalid.
+ */
+__rte_experimental
+int
+rte_dispatcher_stats_get(uint8_t id,
+ struct rte_dispatcher_stats *stats);
+
+/**
+ * Reset statistics for a dispatcher instance.
+ *
+ * This function may be called by any thread (including unregistered
+ * non-EAL threads), but may not produce the correct result if the
+ * dispatcher is running on any service lcore.
+ *
+ * @param id
+ * The dispatcher identifier.
+ *
+ * @return
+ * - 0: Success
+ * - -EINVAL: The @c id parameter was invalid.
+ */
+__rte_experimental
+int
+rte_dispatcher_stats_reset(uint8_t id);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* __RTE_DISPATCHER__ */
new file mode 100644
@@ -0,0 +1,20 @@
+EXPERIMENTAL {
+ global:
+
+ # added in 23.11
+ rte_dispatcher_create;
+ rte_dispatcher_free;
+ rte_dispatcher_service_id_get;
+ rte_dispatcher_bind_port_to_lcore;
+ rte_dispatcher_unbind_port_from_lcore;
+ rte_dispatcher_register;
+ rte_dispatcher_unregister;
+ rte_dispatcher_finalize_register;
+ rte_dispatcher_finalize_unregister;
+ rte_dispatcher_start;
+ rte_dispatcher_stop;
+ rte_dispatcher_stats_get;
+ rte_dispatcher_stats_reset;
+
+ local: *;
+};
@@ -35,6 +35,7 @@ libraries = [
'distributor',
'efd',
'eventdev',
+ 'dispatcher', # dispatcher depends on eventdev
'gpudev',
'gro',
'gso',
@@ -81,6 +82,7 @@ optional_libs = [
'cfgfile',
'compressdev',
'cryptodev',
+ 'dispatcher',
'distributor',
'dmadev',
'efd',