From patchwork Fri Feb 2 12:39:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 136328 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 54EDA43A4F; Fri, 2 Feb 2024 13:41:35 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0612642E8C; Fri, 2 Feb 2024 13:41:06 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) by mails.dpdk.org (Postfix) with ESMTP id A66CA402E8 for ; Fri, 2 Feb 2024 13:41:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706877665; x=1738413665; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5DxhVtBTH0rOnqBaNAwGp13tji+djesWBoIko0QjPO4=; b=PYv5V3oAygesYpdcXNfcSrGSYU8VM8+d5PghJzZuqRaETTltrcD97dd9 afMa6uUJ4MDFdaoI03a5XOS54SoH77r2Tt1lOY6Ct8HKaft2CxPhcUqY1 ncNw22ztcfrxN7D24JuV9Hx3EGc5UFxmuD6MBUaQJ4Xei/TH7hwdbov4P Upk1PcWGTJOWMDt+xGI46EgZagmP48BNlBu3K99yi5/jhFvOthkb0oo4q tBSYsEJTTYFj53wGlsXzK5xnkfzulB00YNdD6/edtvoJdaCw+wFZnSjgb eu2uLDaYwo2BhBbp1RKFnNTLfFM/RVh1jRedGwZ27o0d0DvDSyUiO8RUU g==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="54483" X-IronPort-AV: E=Sophos;i="6.05,238,1701158400"; d="scan'208";a="54483" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Feb 2024 04:40:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,238,1701158400"; d="scan'208";a="347645" Received: from silpixa00401385.ir.intel.com ([10.237.214.38]) by fmviesa006.fm.intel.com with ESMTP; 02 Feb 2024 04:40:55 -0800 From: Bruce Richardson To: dev@dpdk.org, jerinj@marvell.com, mattias.ronnblom@ericsson.com Cc: abdullah.sevincer@intel.com, sachin.saxena@oss.nxp.com, hemant.agrawal@nxp.com, pbhagavatula@marvell.com, pravin.pathak@intel.com, Bruce Richardson Subject: [PATCH v3 09/11] eventdev: improve comments on scheduling types Date: Fri, 2 Feb 2024 12:39:51 +0000 Message-Id: <20240202123953.77166-10-bruce.richardson@intel.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240202123953.77166-1-bruce.richardson@intel.com> References: <20240119174346.108905-1-bruce.richardson@intel.com> <20240202123953.77166-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The description of ordered and atomic scheduling given in the eventdev doxygen documentation was not always clear. Try and simplify this so that it is clearer for the end-user of the application Signed-off-by: Bruce Richardson --- V3: extensive rework following feedback. Please re-review! --- lib/eventdev/rte_eventdev.h | 73 +++++++++++++++++++++++-------------- 1 file changed, 45 insertions(+), 28 deletions(-) diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h index a7d8c28015..8d72765ae7 100644 --- a/lib/eventdev/rte_eventdev.h +++ b/lib/eventdev/rte_eventdev.h @@ -1347,25 +1347,35 @@ struct rte_event_vector { /**< Ordered scheduling * * Events from an ordered flow of an event queue can be scheduled to multiple - * ports for concurrent processing while maintaining the original event order. + * ports for concurrent processing while maintaining the original event order, + * i.e. the order in which they were first enqueued to that queue. * This scheme enables the user to achieve high single flow throughput by - * avoiding SW synchronization for ordering between ports which bound to cores. - * - * The source flow ordering from an event queue is maintained when events are - * enqueued to their destination queue within the same ordered flow context. - * An event port holds the context until application call - * rte_event_dequeue_burst() from the same port, which implicitly releases - * the context. - * User may allow the scheduler to release the context earlier than that - * by invoking rte_event_enqueue_burst() with RTE_EVENT_OP_RELEASE operation. - * - * Events from the source queue appear in their original order when dequeued - * from a destination queue. - * Event ordering is based on the received event(s), but also other - * (newly allocated or stored) events are ordered when enqueued within the same - * ordered context. Events not enqueued (e.g. released or stored) within the - * context are considered missing from reordering and are skipped at this time - * (but can be ordered again within another context). + * avoiding SW synchronization for ordering between ports which are polled + * by different cores. + * + * After events are dequeued from a set of ports, as those events are re-enqueued + * to another queue (with the op field set to @ref RTE_EVENT_OP_FORWARD), the event + * device restores the original event order - including events returned from all + * ports in the set - before the events arrive on the destination queue. + * + * Any events not forwarded i.e. dropped explicitly via RELEASE or implicitly + * released by the next dequeue operation on a port, are skipped by the reordering + * stage and do not affect the reordering of other returned events. + * + * Any NEW events sent on a port are not ordered with respect to FORWARD events sent + * on the same port, since they have no original event order. They also are not + * ordered with respect to NEW events enqueued on other ports. + * However, NEW events to the same destination queue from the same port are guaranteed + * to be enqueued in the order they were submitted via rte_event_enqueue_burst(). + * + * NOTE: + * In restoring event order of forwarded events, the eventdev API guarantees that + * all events from the same flow (i.e. same @ref rte_event.flow_id, + * @ref rte_event.priority and @ref rte_event.queue_id) will be put in the original + * order before being forwarded to the destination queue. + * Some eventdevs may implement stricter ordering to achieve this aim, + * for example, restoring the order across *all* flows dequeued from the same ORDERED + * queue. * * @see rte_event_queue_setup(), rte_event_dequeue_burst(), RTE_EVENT_OP_RELEASE */ @@ -1373,18 +1383,25 @@ struct rte_event_vector { #define RTE_SCHED_TYPE_ATOMIC 1 /**< Atomic scheduling * - * Events from an atomic flow of an event queue can be scheduled only to a + * Events from an atomic flow, identified by a combination of @ref rte_event.flow_id, + * @ref rte_event.queue_id and @ref rte_event.priority, can be scheduled only to a * single port at a time. The port is guaranteed to have exclusive (atomic) * access to the associated flow context, which enables the user to avoid SW - * synchronization. Atomic flows also help to maintain event ordering - * since only one port at a time can process events from a flow of an - * event queue. - * - * The atomic queue synchronization context is dedicated to the port until - * application call rte_event_dequeue_burst() from the same port, - * which implicitly releases the context. User may allow the scheduler to - * release the context earlier than that by invoking rte_event_enqueue_burst() - * with RTE_EVENT_OP_RELEASE operation. + * synchronization. Atomic flows also maintain event ordering + * since only one port at a time can process events from each flow of an + * event queue, and events within a flow are not reordered within the scheduler. + * + * An atomic flow is locked to a port when events from that flow are first + * scheduled to that port. That lock remains in place until the + * application calls rte_event_dequeue_burst() from the same port, + * which implicitly releases the lock (if @ref RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL flag is not set). + * User may allow the scheduler to release the lock earlier than that by invoking + * rte_event_enqueue_burst() with RTE_EVENT_OP_RELEASE operation for each event from that flow. + * + * NOTE: The lock is only released once the last event from the flow, outstanding on the port, + * is released. So long as there is one event from an atomic flow scheduled to + * a port/core (including any events in the port's dequeue queue, not yet read + * by the application), that port will hold the synchronization lock for that flow. * * @see rte_event_queue_setup(), rte_event_dequeue_burst(), RTE_EVENT_OP_RELEASE */