From patchwork Fri May 13 17:58:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 111138 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AD461A00C3; Fri, 13 May 2022 20:01:06 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8FB6740DF7; Fri, 13 May 2022 20:01:06 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 78FBF40DDE for ; Fri, 13 May 2022 20:01:05 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24DCHX0d010448; Fri, 13 May 2022 10:59:01 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=xV2djo7s/29VT8vBH7sONoO693vfUWHctVQMnBq5+jQ=; b=eqnkY1XeNcG+u6kDgURIF+aYclImi0ZmRE75GQpLup2yWjQRbst/AWdrdAQ/ZTrvjN6N w8okFLtwPolsfe3kOmRBpU7C++ODiud5PFpouSWEFnR9HvW2D5vvLOO2GYjjo1nn9B6x ggT8CzgtKE9A2dL5ExbnzQTexzbdVj+lwp+Zsl58at/KP86ap/hAZX7katzqIZBXrBny MlcYoZjNGT5whRH7ONQc75/4iaWxmDhe1lq1W7q9Z3w5LXRZzhraE0bAcilXRANzYVqM V/kYNBO7u3SPQALWI5B/glbdOC4tJrblY4ariZJZOKL8pEuet1bhKPJ9favT2Fc1lbfb 9Q== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3g1c37bgy2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 13 May 2022 10:59:01 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 13 May 2022 10:58:59 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 13 May 2022 10:58:59 -0700 Received: from MININT-80QBFE8.corp.innovium.com (unknown [10.193.70.72]) by maili.marvell.com (Postfix) with ESMTP id 387563F7052; Fri, 13 May 2022 10:58:54 -0700 (PDT) From: To: , Ray Kinsella CC: , , , , , , , , , , , , Pavan Nikhilesh Subject: [PATCH v3 1/3] eventdev: add function to quiesce an event port Date: Fri, 13 May 2022 23:28:39 +0530 Message-ID: <20220513175841.11853-1-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220427113715.15509-1-pbhagavatula@marvell.com> References: <20220427113715.15509-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: NnuHYgXA10RXB6SgPSebOQGe7yxPwU1l X-Proofpoint-ORIG-GUID: NnuHYgXA10RXB6SgPSebOQGe7yxPwU1l X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-13_09,2022-05-13_01,2022-02-23_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Pavan Nikhilesh Add function to quiesce any core specific resources consumed by the event port. When the application decides to migrate the event port to another lcore or teardown the current lcore it may to call `rte_event_port_quiesce` to make sure that all the data associated with the event port are released from the lcore, this might also include any prefetched events. While releasing the event port from the lcore, this function calls the user-provided flush callback once per event. Signed-off-by: Pavan Nikhilesh --- v3 Changes: - Add `rte_` prefix to callback function. - Fix API documentation issues. - Update eventdev documentation. v2 Changes: - Remove internal Change-Id tag from commit messages. doc/guides/prog_guide/eventdev.rst | 35 ++++++++++++++++++++++++++++ lib/eventdev/eventdev_pmd.h | 19 +++++++++++++++ lib/eventdev/rte_eventdev.c | 19 +++++++++++++++ lib/eventdev/rte_eventdev.h | 37 ++++++++++++++++++++++++++++++ lib/eventdev/version.map | 3 +++ 5 files changed, 113 insertions(+) diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst index a49e486a30..afee674ee1 100644 --- a/doc/guides/prog_guide/eventdev.rst +++ b/doc/guides/prog_guide/eventdev.rst @@ -412,6 +412,41 @@ An event driven worker thread has following typical workflow on fastpath: rte_event_enqueue_burst(...); } +Quiescing Event Ports +~~~~~~~~~~~~~~~~~~~~~ + +To migrate the event port to another lcore or while tearing down a worker core +using an event port ``rte_event_port_quiesce()`` can be invoked to make sure +that all the data associated with the event port are released from the worker +core, this might also include any prefetched events. + +A flush callback can be passed to the function to handle any outstanding events. + +.. code-block:: c + + rte_event_port_quiesce(dev_id, port_id, release_cb, NULL); + +.. Note:: + + The event port specific config shall not be reset when this API is + invoked. + +Stopping the EventDev +~~~~~~~~~~~~~~~~~~~~~ + +A single function call tells the eventdev instance to stop processing +events. A flush callback can be registered to free any inflight events +using ``rte_event_dev_stop_flush_callback_register()`` function. + +.. code-block:: c + + int err = rte_event_dev_stop(dev_id); + +.. Note:: + + The event producers such as event_eth_rx_adapter, event_timer_adapter + and event_crypto_adapter need to be stopped before stopping the event + device. Summary ------- diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h index ce469d47a6..6173f22b9b 100644 --- a/lib/eventdev/eventdev_pmd.h +++ b/lib/eventdev/eventdev_pmd.h @@ -381,6 +381,23 @@ typedef int (*eventdev_port_setup_t)(struct rte_eventdev *dev, */ typedef void (*eventdev_port_release_t)(void *port); +/** + * Quiesce any core specific resources consumed by the event port + * + * @param dev + * Event device pointer. + * @param port + * Event port pointer. + * @param flush_cb + * User-provided event flush function. + * @param args + * Arguments to be passed to the user-provided event flush function. + * + */ +typedef void (*eventdev_port_quiesce_t)(struct rte_eventdev *dev, void *port, + rte_eventdev_port_flush_t flush_cb, + void *args); + /** * Link multiple source event queues to destination event port. * @@ -1218,6 +1235,8 @@ struct eventdev_ops { /**< Set up an event port. */ eventdev_port_release_t port_release; /**< Release an event port. */ + eventdev_port_quiesce_t port_quiesce; + /**< Quiesce an event port. */ eventdev_port_link_t port_link; /**< Link event queues to an event port. */ diff --git a/lib/eventdev/rte_eventdev.c b/lib/eventdev/rte_eventdev.c index 532a253553..0250e57f24 100644 --- a/lib/eventdev/rte_eventdev.c +++ b/lib/eventdev/rte_eventdev.c @@ -730,6 +730,25 @@ rte_event_port_setup(uint8_t dev_id, uint8_t port_id, return 0; } +void +rte_event_port_quiesce(uint8_t dev_id, uint8_t port_id, + rte_eventdev_port_flush_t release_cb, void *args) +{ + struct rte_eventdev *dev; + + RTE_EVENTDEV_VALID_DEVID_OR_RET(dev_id); + dev = &rte_eventdevs[dev_id]; + + if (!is_valid_port(dev, port_id)) { + RTE_EDEV_LOG_ERR("Invalid port_id=%" PRIu8, port_id); + return; + } + + if (dev->dev_ops->port_quiesce) + (*dev->dev_ops->port_quiesce)(dev, dev->data->ports[port_id], + release_cb, args); +} + int rte_event_dev_attr_get(uint8_t dev_id, uint32_t attr_id, uint32_t *attr_value) diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h index 42a5660169..1a46d289a9 100644 --- a/lib/eventdev/rte_eventdev.h +++ b/lib/eventdev/rte_eventdev.h @@ -830,6 +830,43 @@ int rte_event_port_setup(uint8_t dev_id, uint8_t port_id, const struct rte_event_port_conf *port_conf); +typedef void (*rte_eventdev_port_flush_t)(uint8_t dev_id, + struct rte_event event, void *arg); +/**< Callback function prototype that can be passed during + * rte_event_port_release(), invoked once per a released event. + */ + +/** + * Quiesce any core specific resources consumed by the event port. + * + * Event ports are generally coupled with lcores, and a given Hardware + * implementation might require the PMD to store port specific data in the + * lcore. + * When the application decides to migrate the event port to another lcore + * or teardown the current lcore it may to call `rte_event_port_quiesce` + * to make sure that all the data associated with the event port are released + * from the lcore, this might also include any prefetched events. + * While releasing the event port from the lcore, this function calls the + * user-provided flush callback once per event. + * + * @note The event port specific config shall not be reset when this API is + * called. + * + * @param dev_id + * The identifier of the device. + * @param port_id + * The index of the event port to setup. The value must be in the range + * [0, nb_event_ports - 1] previously supplied to rte_event_dev_configure(). + * @param release_cb + * Callback function invoked once per flushed event. + * @param args + * Argument supplied to callback. + */ +__rte_experimental +void +rte_event_port_quiesce(uint8_t dev_id, uint8_t port_id, + rte_eventdev_port_flush_t release_cb, void *args); + /** * The queue depth of the port on the enqueue side */ diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map index cd5dada07f..1907093539 100644 --- a/lib/eventdev/version.map +++ b/lib/eventdev/version.map @@ -108,6 +108,9 @@ EXPERIMENTAL { # added in 22.03 rte_event_eth_rx_adapter_event_port_get; + + # added in 22.07 + rte_event_port_quiesce; }; INTERNAL {