Message ID | 20230809142617.6482-1-pbhagavatula@marvell.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A5B2F43019; Wed, 9 Aug 2023 16:26:41 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6625B40691; Wed, 9 Aug 2023 16:26:40 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 096DC400D6 for <dev@dpdk.org>; Wed, 9 Aug 2023 16:26:38 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 379CjPQ3024126; Wed, 9 Aug 2023 07:26:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=AKW4kYkT7ZFElFRZ4A6QWpf+8u+LqYA7GYA3WeUXGdE=; b=ddi6a2MoYVbhle53eBziggtx7ppug5mYioFySN4G1PabUcDETbkHZsJTF4nJfmprmqyx rUHSnvxY29OMPP3c7Q7OZjq70bqDqk1wTj7+CxeCJebjLpwbVuC0cPI+yc+NmHQ7VF8P dsFNUmJXRokSF8ufKywEqWYcf6vTcCO+/t0jFGZmU9dN+c6Hv/PcZYlWo6iFvz38qR16 CdH3gHycWk01Cmi4s7kFvloTZzJkLzaAKMIzECU9JCrgE9YcuO0jpjSSiUzwKyr2rolW wUk/zDizQkhwt/jBO7U9aduihg0oQlf0a5iH/7bUEVHOgVm7Gy468jqNfQsoQEUDMF7J /w== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3sbkntn1b2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 09 Aug 2023 07:26:38 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Wed, 9 Aug 2023 07:26:36 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Wed, 9 Aug 2023 07:26:36 -0700 Received: from MININT-80QBFE8.corp.innovium.com (MININT-80QBFE8.marvell.com [10.28.164.106]) by maili.marvell.com (Postfix) with ESMTP id 595B45B6928; Wed, 9 Aug 2023 07:26:31 -0700 (PDT) From: <pbhagavatula@marvell.com> To: <jerinj@marvell.com>, <pbhagavatula@marvell.com>, <sthotton@marvell.com>, <timothy.mcdaniel@intel.com>, <hemant.agrawal@nxp.com>, <sachin.saxena@nxp.com>, <mattias.ronnblom@ericsson.com>, <liangma@liangbit.com>, <peter.mccarthy@intel.com>, <harry.van.haaren@intel.com>, <erik.g.carrillo@intel.com>, <abhinandan.gujjar@intel.com>, <s.v.naga.harish.k@intel.com>, <anatoly.burakov@intel.com> CC: <dev@dpdk.org> Subject: [RFC 0/3] Introduce event link profiles Date: Wed, 9 Aug 2023 19:56:14 +0530 Message-ID: <20230809142617.6482-1-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: F8fTPK0GI7Yb8dduZIXtbDnVTMVupyy9 X-Proofpoint-ORIG-GUID: F8fTPK0GI7Yb8dduZIXtbDnVTMVupyy9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-08-09_11,2023-08-09_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org |
Series |
Introduce event link profiles
|
|
Message
Pavan Nikhilesh Bhagavatula
Aug. 9, 2023, 2:26 p.m. UTC
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
A collection of event queues linked to an event port can be associated
with unique identifier called as a profile, multiple such profiles can
be configured based on the event device capability using the function
`rte_event_port_link_with_profile` which takes arguments similar to
`rte_event_port_link` in addition to the profile identifier.
The maximum link profiles that are supported by an event device is
advertised through the structure member
`rte_event_dev_info::max_profiles_per_port`.
By default, event ports are configured to use the link profile 0 on
initialization.
Once multiple link profiles are set up and the event device is started, the
application can use the function `rte_event_port_change_profile` to change
the currently active profile on an event port. This effects the next
`rte_event_dequeue_burst` call, where the event queues associated with the
newly active link profile will participate in scheduling.
Rudementary work flow would something like:
Config path:
uint8_t lowQ[4] = {4, 5, 6, 7};
uint8_t highQ[4] = {0, 1, 2, 3};
if (rte_event_dev_info.max_profiles_per_port < 2)
return -ENOTSUP;
rte_event_port_link_with_profile(0, 0, highQ, NULL, 4, 0);
rte_event_port_link_with_profile(0, 0, lowQ, NULL, 4, 1);
Worker path:
empty_high_deq = 0;
empty_low_deq = 0;
is_low_deq = 0;
while (1) {
deq = rte_event_dequeue_burst(0, 0, &ev, 1, 0);
if (deq == 0) {
/**
* Change link profile based on work activity on current
* active profile
*/
if (is_low_deq) {
empty_low_deq++;
if (empty_low_deq == MAX_LOW_RETRY) {
rte_event_port_change_profile(0, 0, 0);
is_low_deq = 0;
empty_low_deq = 0;
}
continue;
}
if (empty_high_deq == MAX_HIGH_RETRY) {
rte_event_port_change_profile(0, 0, 1);
is_low_deq = 1;
empty_high_deq = 0;
}
continue;
}
// Process the event received.
if (is_low_deq++ == MAX_LOW_EVENTS) {
rte_event_port_change_profile(0, 0, 0);
is_low_deq = 0;
}
}
An application could use heuristic data of load/activity of a given event
port and change its active profile to adapt to the traffic pattern.
An unlink function `rte_event_port_unlink_with_profile` is provided to
modify the links associated to a profile, and
`rte_event_port_links_get_with_profile` can be used to retrieve the links
associated with a profile.
Pavan Nikhilesh (3):
eventdev: introduce link profiles
event/cnxk: implement event link profiles
test/event: add event link profile test
app/test/test_eventdev.c | 110 ++++++++++
config/rte_config.h | 1 +
doc/guides/eventdevs/cnxk.rst | 1 +
doc/guides/prog_guide/eventdev.rst | 58 ++++++
drivers/common/cnxk/roc_nix_inl_dev.c | 4 +-
drivers/common/cnxk/roc_sso.c | 18 +-
drivers/common/cnxk/roc_sso.h | 8 +-
drivers/common/cnxk/roc_sso_priv.h | 4 +-
drivers/event/cnxk/cn10k_eventdev.c | 45 ++--
drivers/event/cnxk/cn10k_worker.c | 11 +
drivers/event/cnxk/cn10k_worker.h | 1 +
drivers/event/cnxk/cn9k_eventdev.c | 72 ++++---
drivers/event/cnxk/cn9k_worker.c | 22 ++
drivers/event/cnxk/cn9k_worker.h | 2 +
drivers/event/cnxk/cnxk_eventdev.c | 34 ++--
drivers/event/cnxk/cnxk_eventdev.h | 10 +-
drivers/event/dlb2/dlb2.c | 1 +
drivers/event/dpaa/dpaa_eventdev.c | 1 +
drivers/event/dpaa2/dpaa2_eventdev.c | 2 +-
drivers/event/dsw/dsw_evdev.c | 1 +
drivers/event/octeontx/ssovf_evdev.c | 2 +-
drivers/event/opdl/opdl_evdev.c | 1 +
drivers/event/skeleton/skeleton_eventdev.c | 1 +
drivers/event/sw/sw_evdev.c | 1 +
lib/eventdev/eventdev_pmd.h | 59 +++++-
lib/eventdev/eventdev_private.c | 9 +
lib/eventdev/eventdev_trace.h | 22 ++
lib/eventdev/eventdev_trace_points.c | 6 +
lib/eventdev/rte_eventdev.c | 146 ++++++++++---
lib/eventdev/rte_eventdev.h | 226 +++++++++++++++++++++
lib/eventdev/rte_eventdev_core.h | 4 +
lib/eventdev/rte_eventdev_trace_fp.h | 8 +
lib/eventdev/version.map | 5 +
33 files changed, 788 insertions(+), 108 deletions(-)
--
2.25.1
Comments
On 2023-08-09 16:26, pbhagavatula@marvell.com wrote: > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > A collection of event queues linked to an event port can be associated > with unique identifier called as a profile, multiple such profiles can > be configured based on the event device capability using the function > `rte_event_port_link_with_profile` which takes arguments similar to > `rte_event_port_link` in addition to the profile identifier. > What is the overall goal with this new API? What problems does it intend to solve, that the old one doesn't. > The maximum link profiles that are supported by an event device is > advertised through the structure member > `rte_event_dev_info::max_profiles_per_port`. > > By default, event ports are configured to use the link profile 0 on > initialization. > > Once multiple link profiles are set up and the event device is started, the > application can use the function `rte_event_port_change_profile` to change > the currently active profile on an event port. This effects the next > `rte_event_dequeue_burst` call, where the event queues associated with the > newly active link profile will participate in scheduling. > > Rudementary work flow would something like: > > Config path: > > uint8_t lowQ[4] = {4, 5, 6, 7}; > uint8_t highQ[4] = {0, 1, 2, 3}; > > if (rte_event_dev_info.max_profiles_per_port < 2) > return -ENOTSUP; > > rte_event_port_link_with_profile(0, 0, highQ, NULL, 4, 0); > rte_event_port_link_with_profile(0, 0, lowQ, NULL, 4, 1); > > Worker path: > > empty_high_deq = 0; > empty_low_deq = 0; > is_low_deq = 0; > while (1) { > deq = rte_event_dequeue_burst(0, 0, &ev, 1, 0); > if (deq == 0) { > /** > * Change link profile based on work activity on current > * active profile > */ > if (is_low_deq) { > empty_low_deq++; > if (empty_low_deq == MAX_LOW_RETRY) { > rte_event_port_change_profile(0, 0, 0); > is_low_deq = 0; > empty_low_deq = 0; > } > continue; > } > > if (empty_high_deq == MAX_HIGH_RETRY) { > rte_event_port_change_profile(0, 0, 1); > is_low_deq = 1; > empty_high_deq = 0; > } > continue; > } > > // Process the event received. > > if (is_low_deq++ == MAX_LOW_EVENTS) { > rte_event_port_change_profile(0, 0, 0); > is_low_deq = 0; > } > } > This thing looks like the application is asked to do work scheduling. That doesn't sound right. That's the job of the work scheduler (i.e., the event device). If this thing is merely a matter of changing what queues are linked to which ports, wouldn't a new call: rte_event_port_link_modify() suffice? > An application could use heuristic data of load/activity of a given event > port and change its active profile to adapt to the traffic pattern. > > An unlink function `rte_event_port_unlink_with_profile` is provided to > modify the links associated to a profile, and > `rte_event_port_links_get_with_profile` can be used to retrieve the links > associated with a profile. > > Pavan Nikhilesh (3): > eventdev: introduce link profiles > event/cnxk: implement event link profiles > test/event: add event link profile test > > app/test/test_eventdev.c | 110 ++++++++++ > config/rte_config.h | 1 + > doc/guides/eventdevs/cnxk.rst | 1 + > doc/guides/prog_guide/eventdev.rst | 58 ++++++ > drivers/common/cnxk/roc_nix_inl_dev.c | 4 +- > drivers/common/cnxk/roc_sso.c | 18 +- > drivers/common/cnxk/roc_sso.h | 8 +- > drivers/common/cnxk/roc_sso_priv.h | 4 +- > drivers/event/cnxk/cn10k_eventdev.c | 45 ++-- > drivers/event/cnxk/cn10k_worker.c | 11 + > drivers/event/cnxk/cn10k_worker.h | 1 + > drivers/event/cnxk/cn9k_eventdev.c | 72 ++++--- > drivers/event/cnxk/cn9k_worker.c | 22 ++ > drivers/event/cnxk/cn9k_worker.h | 2 + > drivers/event/cnxk/cnxk_eventdev.c | 34 ++-- > drivers/event/cnxk/cnxk_eventdev.h | 10 +- > drivers/event/dlb2/dlb2.c | 1 + > drivers/event/dpaa/dpaa_eventdev.c | 1 + > drivers/event/dpaa2/dpaa2_eventdev.c | 2 +- > drivers/event/dsw/dsw_evdev.c | 1 + > drivers/event/octeontx/ssovf_evdev.c | 2 +- > drivers/event/opdl/opdl_evdev.c | 1 + > drivers/event/skeleton/skeleton_eventdev.c | 1 + > drivers/event/sw/sw_evdev.c | 1 + > lib/eventdev/eventdev_pmd.h | 59 +++++- > lib/eventdev/eventdev_private.c | 9 + > lib/eventdev/eventdev_trace.h | 22 ++ > lib/eventdev/eventdev_trace_points.c | 6 + > lib/eventdev/rte_eventdev.c | 146 ++++++++++--- > lib/eventdev/rte_eventdev.h | 226 +++++++++++++++++++++ > lib/eventdev/rte_eventdev_core.h | 4 + > lib/eventdev/rte_eventdev_trace_fp.h | 8 + > lib/eventdev/version.map | 5 + > 33 files changed, 788 insertions(+), 108 deletions(-) > > -- > 2.25.1 >
> On 2023-08-09 16:26, pbhagavatula@marvell.com wrote: > > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > > > A collection of event queues linked to an event port can be associated > > with unique identifier called as a profile, multiple such profiles can > > be configured based on the event device capability using the function > > `rte_event_port_link_with_profile` which takes arguments similar to > > `rte_event_port_link` in addition to the profile identifier. > > > > What is the overall goal with this new API? What problems does it intend > to solve, that the old one doesn't. Linking and unlinking currently has huge overhead and when it needs to be done in fastpath, we have to wait for unlinks to complete and handle other corner cases. This patch set solves it by avoiding linking/unlinking altogether in fastpath by preconfigured set of link profiles out of which only one would be active and can be changed in fastpath with a simple function call. There is no link/unlink waiting for unlink overhead. > > > The maximum link profiles that are supported by an event device is > > advertised through the structure member > > `rte_event_dev_info::max_profiles_per_port`. > > > > By default, event ports are configured to use the link profile 0 on > > initialization. > > > > Once multiple link profiles are set up and the event device is started, the > > application can use the function `rte_event_port_change_profile` to > change > > the currently active profile on an event port. This effects the next > > `rte_event_dequeue_burst` call, where the event queues associated with > the > > newly active link profile will participate in scheduling. > > > > Rudementary work flow would something like: > > > > Config path: > > > > uint8_t lowQ[4] = {4, 5, 6, 7}; > > uint8_t highQ[4] = {0, 1, 2, 3}; > > > > if (rte_event_dev_info.max_profiles_per_port < 2) > > return -ENOTSUP; > > > > rte_event_port_link_with_profile(0, 0, highQ, NULL, 4, 0); > > rte_event_port_link_with_profile(0, 0, lowQ, NULL, 4, 1); > > > > Worker path: > > > > empty_high_deq = 0; > > empty_low_deq = 0; > > is_low_deq = 0; > > while (1) { > > deq = rte_event_dequeue_burst(0, 0, &ev, 1, 0); > > if (deq == 0) { > > /** > > * Change link profile based on work activity on current > > * active profile > > */ > > if (is_low_deq) { > > empty_low_deq++; > > if (empty_low_deq == MAX_LOW_RETRY) { > > rte_event_port_change_profile(0, 0, 0); > > is_low_deq = 0; > > empty_low_deq = 0; > > } > > continue; > > } > > > > if (empty_high_deq == MAX_HIGH_RETRY) { > > rte_event_port_change_profile(0, 0, 1); > > is_low_deq = 1; > > empty_high_deq = 0; > > } > > continue; > > } > > > > // Process the event received. > > > > if (is_low_deq++ == MAX_LOW_EVENTS) { > > rte_event_port_change_profile(0, 0, 0); > > is_low_deq = 0; > > } > > } > > > > This thing looks like the application is asked to do work scheduling. > That doesn't sound right. That's the job of the work scheduler (i.e., > the event device). > > If this thing is merely a matter of changing what queues are linked to > which ports, wouldn't a new call: > rte_event_port_link_modify() > suffice? Some applications divide their available lcores into multiple types of workers which each work on a unique set of event queues, application might need to modify the worker ratio based on various parameters at run time without a lot of overhead. Modifying links wouldn’t work because we might want to restore previous links based on the new traffic pattern etc.,. > > > An application could use heuristic data of load/activity of a given event > > port and change its active profile to adapt to the traffic pattern. > > > > An unlink function `rte_event_port_unlink_with_profile` is provided to > > modify the links associated to a profile, and > > `rte_event_port_links_get_with_profile` can be used to retrieve the links > > associated with a profile. > > > > Pavan Nikhilesh (3): > > eventdev: introduce link profiles > > event/cnxk: implement event link profiles > > test/event: add event link profile test > > > > app/test/test_eventdev.c | 110 ++++++++++ > > config/rte_config.h | 1 + > > doc/guides/eventdevs/cnxk.rst | 1 + > > doc/guides/prog_guide/eventdev.rst | 58 ++++++ > > drivers/common/cnxk/roc_nix_inl_dev.c | 4 +- > > drivers/common/cnxk/roc_sso.c | 18 +- > > drivers/common/cnxk/roc_sso.h | 8 +- > > drivers/common/cnxk/roc_sso_priv.h | 4 +- > > drivers/event/cnxk/cn10k_eventdev.c | 45 ++-- > > drivers/event/cnxk/cn10k_worker.c | 11 + > > drivers/event/cnxk/cn10k_worker.h | 1 + > > drivers/event/cnxk/cn9k_eventdev.c | 72 ++++--- > > drivers/event/cnxk/cn9k_worker.c | 22 ++ > > drivers/event/cnxk/cn9k_worker.h | 2 + > > drivers/event/cnxk/cnxk_eventdev.c | 34 ++-- > > drivers/event/cnxk/cnxk_eventdev.h | 10 +- > > drivers/event/dlb2/dlb2.c | 1 + > > drivers/event/dpaa/dpaa_eventdev.c | 1 + > > drivers/event/dpaa2/dpaa2_eventdev.c | 2 +- > > drivers/event/dsw/dsw_evdev.c | 1 + > > drivers/event/octeontx/ssovf_evdev.c | 2 +- > > drivers/event/opdl/opdl_evdev.c | 1 + > > drivers/event/skeleton/skeleton_eventdev.c | 1 + > > drivers/event/sw/sw_evdev.c | 1 + > > lib/eventdev/eventdev_pmd.h | 59 +++++- > > lib/eventdev/eventdev_private.c | 9 + > > lib/eventdev/eventdev_trace.h | 22 ++ > > lib/eventdev/eventdev_trace_points.c | 6 + > > lib/eventdev/rte_eventdev.c | 146 ++++++++++--- > > lib/eventdev/rte_eventdev.h | 226 +++++++++++++++++++++ > > lib/eventdev/rte_eventdev_core.h | 4 + > > lib/eventdev/rte_eventdev_trace_fp.h | 8 + > > lib/eventdev/version.map | 5 + > > 33 files changed, 788 insertions(+), 108 deletions(-) > > > > -- > > 2.25.1 > >
On 2023-08-10 07:17, Pavan Nikhilesh Bhagavatula wrote: >> On 2023-08-09 16:26, pbhagavatula@marvell.com wrote: >>> From: Pavan Nikhilesh <pbhagavatula@marvell.com> >>> >>> A collection of event queues linked to an event port can be associated >>> with unique identifier called as a profile, multiple such profiles can >>> be configured based on the event device capability using the function >>> `rte_event_port_link_with_profile` which takes arguments similar to >>> `rte_event_port_link` in addition to the profile identifier. >>> >> >> What is the overall goal with this new API? What problems does it intend >> to solve, that the old one doesn't. > > Linking and unlinking currently has huge overhead and when it needs to be done > in fastpath, we have to wait for unlinks to complete and handle other corner cases. > OK, so this API change is specific to some particular hardware? Is this true for some other event devices? That "huge overhead" goes to "simple function call" for unlinking+linking, provided the target configuration is known in advance. What is the overall use case? > This patch set solves it by avoiding linking/unlinking altogether in fastpath by > preconfigured set of link profiles out of which only one would be active and can > be changed in fastpath with a simple function call. There is no link/unlink waiting for > unlink overhead. > >> >>> The maximum link profiles that are supported by an event device is >>> advertised through the structure member >>> `rte_event_dev_info::max_profiles_per_port`. >>> >>> By default, event ports are configured to use the link profile 0 on >>> initialization. >>> >>> Once multiple link profiles are set up and the event device is started, the >>> application can use the function `rte_event_port_change_profile` to >> change >>> the currently active profile on an event port. This effects the next >>> `rte_event_dequeue_burst` call, where the event queues associated with >> the >>> newly active link profile will participate in scheduling. >>> >>> Rudementary work flow would something like: >>> >>> Config path: >>> >>> uint8_t lowQ[4] = {4, 5, 6, 7}; >>> uint8_t highQ[4] = {0, 1, 2, 3}; >>> >>> if (rte_event_dev_info.max_profiles_per_port < 2) >>> return -ENOTSUP; >>> >>> rte_event_port_link_with_profile(0, 0, highQ, NULL, 4, 0); >>> rte_event_port_link_with_profile(0, 0, lowQ, NULL, 4, 1); >>> >>> Worker path: >>> >>> empty_high_deq = 0; >>> empty_low_deq = 0; >>> is_low_deq = 0; >>> while (1) { >>> deq = rte_event_dequeue_burst(0, 0, &ev, 1, 0); >>> if (deq == 0) { >>> /** >>> * Change link profile based on work activity on current >>> * active profile >>> */ >>> if (is_low_deq) { >>> empty_low_deq++; >>> if (empty_low_deq == MAX_LOW_RETRY) { >>> rte_event_port_change_profile(0, 0, 0); >>> is_low_deq = 0; >>> empty_low_deq = 0; >>> } >>> continue; >>> } >>> >>> if (empty_high_deq == MAX_HIGH_RETRY) { >>> rte_event_port_change_profile(0, 0, 1); >>> is_low_deq = 1; >>> empty_high_deq = 0; >>> } >>> continue; >>> } >>> >>> // Process the event received. >>> >>> if (is_low_deq++ == MAX_LOW_EVENTS) { >>> rte_event_port_change_profile(0, 0, 0); >>> is_low_deq = 0; >>> } >>> } >>> >> >> This thing looks like the application is asked to do work scheduling. >> That doesn't sound right. That's the job of the work scheduler (i.e., >> the event device). >> >> If this thing is merely a matter of changing what queues are linked to >> which ports, wouldn't a new call: >> rte_event_port_link_modify() >> suffice? > > > Some applications divide their available lcores into multiple types of > workers which each work on a unique set of event queues, application might > need to modify the worker ratio based on various parameters at run time > without a lot of overhead. > > Modifying links wouldn’t work because we might want to restore previous links > based on the new traffic pattern etc.,. > >> >>> An application could use heuristic data of load/activity of a given event >>> port and change its active profile to adapt to the traffic pattern. >>> >>> An unlink function `rte_event_port_unlink_with_profile` is provided to >>> modify the links associated to a profile, and >>> `rte_event_port_links_get_with_profile` can be used to retrieve the links >>> associated with a profile. >>> >>> Pavan Nikhilesh (3): >>> eventdev: introduce link profiles >>> event/cnxk: implement event link profiles >>> test/event: add event link profile test >>> >>> app/test/test_eventdev.c | 110 ++++++++++ >>> config/rte_config.h | 1 + >>> doc/guides/eventdevs/cnxk.rst | 1 + >>> doc/guides/prog_guide/eventdev.rst | 58 ++++++ >>> drivers/common/cnxk/roc_nix_inl_dev.c | 4 +- >>> drivers/common/cnxk/roc_sso.c | 18 +- >>> drivers/common/cnxk/roc_sso.h | 8 +- >>> drivers/common/cnxk/roc_sso_priv.h | 4 +- >>> drivers/event/cnxk/cn10k_eventdev.c | 45 ++-- >>> drivers/event/cnxk/cn10k_worker.c | 11 + >>> drivers/event/cnxk/cn10k_worker.h | 1 + >>> drivers/event/cnxk/cn9k_eventdev.c | 72 ++++--- >>> drivers/event/cnxk/cn9k_worker.c | 22 ++ >>> drivers/event/cnxk/cn9k_worker.h | 2 + >>> drivers/event/cnxk/cnxk_eventdev.c | 34 ++-- >>> drivers/event/cnxk/cnxk_eventdev.h | 10 +- >>> drivers/event/dlb2/dlb2.c | 1 + >>> drivers/event/dpaa/dpaa_eventdev.c | 1 + >>> drivers/event/dpaa2/dpaa2_eventdev.c | 2 +- >>> drivers/event/dsw/dsw_evdev.c | 1 + >>> drivers/event/octeontx/ssovf_evdev.c | 2 +- >>> drivers/event/opdl/opdl_evdev.c | 1 + >>> drivers/event/skeleton/skeleton_eventdev.c | 1 + >>> drivers/event/sw/sw_evdev.c | 1 + >>> lib/eventdev/eventdev_pmd.h | 59 +++++- >>> lib/eventdev/eventdev_private.c | 9 + >>> lib/eventdev/eventdev_trace.h | 22 ++ >>> lib/eventdev/eventdev_trace_points.c | 6 + >>> lib/eventdev/rte_eventdev.c | 146 ++++++++++--- >>> lib/eventdev/rte_eventdev.h | 226 +++++++++++++++++++++ >>> lib/eventdev/rte_eventdev_core.h | 4 + >>> lib/eventdev/rte_eventdev_trace_fp.h | 8 + >>> lib/eventdev/version.map | 5 + >>> 33 files changed, 788 insertions(+), 108 deletions(-) >>> >>> -- >>> 2.25.1 >>>
> -----Original Message----- > From: Mattias Rönnblom <hofors@lysator.liu.se> > Sent: Saturday, August 12, 2023 11:23 AM > To: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>; Jerin Jacob > Kollanukkaran <jerinj@marvell.com>; Shijith Thotton > <sthotton@marvell.com>; timothy.mcdaniel@intel.com; > hemant.agrawal@nxp.com; sachin.saxena@nxp.com; > mattias.ronnblom@ericsson.com; liangma@liangbit.com; > peter.mccarthy@intel.com; harry.van.haaren@intel.com; > erik.g.carrillo@intel.com; abhinandan.gujjar@intel.com; > s.v.naga.harish.k@intel.com; anatoly.burakov@intel.com > Cc: dev@dpdk.org > Subject: Re: [EXT] Re: [RFC 0/3] Introduce event link profiles > > On 2023-08-10 07:17, Pavan Nikhilesh Bhagavatula wrote: > >> On 2023-08-09 16:26, pbhagavatula@marvell.com wrote: > >>> From: Pavan Nikhilesh <pbhagavatula@marvell.com> > >>> > >>> A collection of event queues linked to an event port can be associated > >>> with unique identifier called as a profile, multiple such profiles can > >>> be configured based on the event device capability using the function > >>> `rte_event_port_link_with_profile` which takes arguments similar to > >>> `rte_event_port_link` in addition to the profile identifier. > >>> > >> > >> What is the overall goal with this new API? What problems does it intend > >> to solve, that the old one doesn't. > > > > Linking and unlinking currently has huge overhead and when it needs to be > done > > in fastpath, we have to wait for unlinks to complete and handle other > corner cases. > > > > OK, so this API change is specific to some particular hardware? Is this > true for some other event devices? That "huge overhead" goes to "simple > function call" for unlinking+linking, provided the target configuration > is known in advance. CNXK supports this feature in HW as an optional feature. Drivers can return -ENOSUP or set info::max_profiles_per_port to 1 if the feature cannot be implemented or decided to not implement this. Although, I believe it can be easily integrated into other SW eventdevices. > > What is the overall use case? > One of the primary use case that our customers are interested is to modify the worker ratio on the fly without the overhead of unlink/relink and also to make sure low-priority queues are at least scheduled once in a while without worrying about affinities and weights. > > This patch set solves it by avoiding linking/unlinking altogether in fastpath > by > > preconfigured set of link profiles out of which only one would be active and > can > > be changed in fastpath with a simple function call. There is no link/unlink > waiting for > > unlink overhead. > > > >> > >>> The maximum link profiles that are supported by an event device is > >>> advertised through the structure member > >>> `rte_event_dev_info::max_profiles_per_port`. > >>> > >>> By default, event ports are configured to use the link profile 0 on > >>> initialization. > >>> > >>> Once multiple link profiles are set up and the event device is started, the > >>> application can use the function `rte_event_port_change_profile` to > >> change > >>> the currently active profile on an event port. This effects the next > >>> `rte_event_dequeue_burst` call, where the event queues associated > with > >> the > >>> newly active link profile will participate in scheduling. > >>> > >>> Rudementary work flow would something like: > >>> > >>> Config path: > >>> > >>> uint8_t lowQ[4] = {4, 5, 6, 7}; > >>> uint8_t highQ[4] = {0, 1, 2, 3}; > >>> > >>> if (rte_event_dev_info.max_profiles_per_port < 2) > >>> return -ENOTSUP; > >>> > >>> rte_event_port_link_with_profile(0, 0, highQ, NULL, 4, 0); > >>> rte_event_port_link_with_profile(0, 0, lowQ, NULL, 4, 1); > >>> > >>> Worker path: > >>> > >>> empty_high_deq = 0; > >>> empty_low_deq = 0; > >>> is_low_deq = 0; > >>> while (1) { > >>> deq = rte_event_dequeue_burst(0, 0, &ev, 1, 0); > >>> if (deq == 0) { > >>> /** > >>> * Change link profile based on work activity on current > >>> * active profile > >>> */ > >>> if (is_low_deq) { > >>> empty_low_deq++; > >>> if (empty_low_deq == MAX_LOW_RETRY) { > >>> rte_event_port_change_profile(0, 0, 0); > >>> is_low_deq = 0; > >>> empty_low_deq = 0; > >>> } > >>> continue; > >>> } > >>> > >>> if (empty_high_deq == MAX_HIGH_RETRY) { > >>> rte_event_port_change_profile(0, 0, 1); > >>> is_low_deq = 1; > >>> empty_high_deq = 0; > >>> } > >>> continue; > >>> } > >>> > >>> // Process the event received. > >>> > >>> if (is_low_deq++ == MAX_LOW_EVENTS) { > >>> rte_event_port_change_profile(0, 0, 0); > >>> is_low_deq = 0; > >>> } > >>> } > >>> > >> > >> This thing looks like the application is asked to do work scheduling. > >> That doesn't sound right. That's the job of the work scheduler (i.e., > >> the event device). > >> > >> If this thing is merely a matter of changing what queues are linked to > >> which ports, wouldn't a new call: > >> rte_event_port_link_modify() > >> suffice? > > > > > > Some applications divide their available lcores into multiple types of > > workers which each work on a unique set of event queues, application > might > > need to modify the worker ratio based on various parameters at run time > > without a lot of overhead. > > > > Modifying links wouldn’t work because we might want to restore previous > links > > based on the new traffic pattern etc.,. > > > >> > >>> An application could use heuristic data of load/activity of a given event > >>> port and change its active profile to adapt to the traffic pattern. > >>> > >>> An unlink function `rte_event_port_unlink_with_profile` is provided to > >>> modify the links associated to a profile, and > >>> `rte_event_port_links_get_with_profile` can be used to retrieve the > links > >>> associated with a profile. > >>> > >>> Pavan Nikhilesh (3): > >>> eventdev: introduce link profiles > >>> event/cnxk: implement event link profiles > >>> test/event: add event link profile test > >>> > >>> app/test/test_eventdev.c | 110 ++++++++++ > >>> config/rte_config.h | 1 + > >>> doc/guides/eventdevs/cnxk.rst | 1 + > >>> doc/guides/prog_guide/eventdev.rst | 58 ++++++ > >>> drivers/common/cnxk/roc_nix_inl_dev.c | 4 +- > >>> drivers/common/cnxk/roc_sso.c | 18 +- > >>> drivers/common/cnxk/roc_sso.h | 8 +- > >>> drivers/common/cnxk/roc_sso_priv.h | 4 +- > >>> drivers/event/cnxk/cn10k_eventdev.c | 45 ++-- > >>> drivers/event/cnxk/cn10k_worker.c | 11 + > >>> drivers/event/cnxk/cn10k_worker.h | 1 + > >>> drivers/event/cnxk/cn9k_eventdev.c | 72 ++++--- > >>> drivers/event/cnxk/cn9k_worker.c | 22 ++ > >>> drivers/event/cnxk/cn9k_worker.h | 2 + > >>> drivers/event/cnxk/cnxk_eventdev.c | 34 ++-- > >>> drivers/event/cnxk/cnxk_eventdev.h | 10 +- > >>> drivers/event/dlb2/dlb2.c | 1 + > >>> drivers/event/dpaa/dpaa_eventdev.c | 1 + > >>> drivers/event/dpaa2/dpaa2_eventdev.c | 2 +- > >>> drivers/event/dsw/dsw_evdev.c | 1 + > >>> drivers/event/octeontx/ssovf_evdev.c | 2 +- > >>> drivers/event/opdl/opdl_evdev.c | 1 + > >>> drivers/event/skeleton/skeleton_eventdev.c | 1 + > >>> drivers/event/sw/sw_evdev.c | 1 + > >>> lib/eventdev/eventdev_pmd.h | 59 +++++- > >>> lib/eventdev/eventdev_private.c | 9 + > >>> lib/eventdev/eventdev_trace.h | 22 ++ > >>> lib/eventdev/eventdev_trace_points.c | 6 + > >>> lib/eventdev/rte_eventdev.c | 146 ++++++++++--- > >>> lib/eventdev/rte_eventdev.h | 226 +++++++++++++++++++++ > >>> lib/eventdev/rte_eventdev_core.h | 4 + > >>> lib/eventdev/rte_eventdev_trace_fp.h | 8 + > >>> lib/eventdev/version.map | 5 + > >>> 33 files changed, 788 insertions(+), 108 deletions(-) > >>> > >>> -- > >>> 2.25.1 > >>>