From patchwork Wed Sep 12 16:16:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 44620 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 41BBE4CA1; Wed, 12 Sep 2018 18:16:20 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 6765F4C9F for ; Wed, 12 Sep 2018 18:16:17 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Sep 2018 09:16:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,365,1531810800"; d="scan'208";a="73717343" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.187]) by orsmga006.jf.intel.com with ESMTP; 12 Sep 2018 09:16:16 -0700 From: Harry van Haaren To: dev@dpdk.org Cc: jerin.jacob@caviumnetworks.com, matias.elo@nokia.com, Harry van Haaren Date: Wed, 12 Sep 2018 17:16:14 +0100 Message-Id: <20180912161616.42306-1-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.17.1 Subject: [dpdk-dev] [PATCH 1/3] event: add function for reading unlink in progress X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit introduces a new function in the eventdev API, which allows applications to read the number of unlink requests in progress on a particular port of an eventdev instance. This information allows applications to verify when no more packets from a particular queue (or any queue) will arrive at a port. The application could decide to stop polling, or put the core into a sleep state if it wishes, as it is ensured that no new packets will arrive at a particular port anymore if all queues are unlinked. Suggested-by: Matias Elo Signed-off-by: Harry van Haaren Acked-by: Jerin Jacob dev_ops->port_unlinks_in_progress, 0); + + return (*dev->dev_ops->port_unlinks_in_progress)(dev, + dev->data->ports[port_id]); +} + int rte_event_port_links_get(uint8_t dev_id, uint8_t port_id, uint8_t queues[], uint8_t priorities[]) diff --git a/lib/librte_eventdev/rte_eventdev.h b/lib/librte_eventdev/rte_eventdev.h index b6fd6ee7f..d07e297bc 100644 --- a/lib/librte_eventdev/rte_eventdev.h +++ b/lib/librte_eventdev/rte_eventdev.h @@ -1656,8 +1656,9 @@ rte_event_port_link(uint8_t dev_id, uint8_t port_id, * event port designated by its *port_id* on the event device designated * by its *dev_id*. * - * The unlink establishment shall disable the event port *port_id* from - * receiving events from the specified event queue *queue_id* + * The unlink call issues an async request to disable the event port *port_id* + * from receiving events from the specified event queue *queue_id*. See + * *rte_event_port_unlinks_in_progress* to poll for completed unlinks. * * Event queue(s) to event port unlink establishment can be changed at runtime * without re-configuring the device. @@ -1694,6 +1695,29 @@ int rte_event_port_unlink(uint8_t dev_id, uint8_t port_id, uint8_t queues[], uint16_t nb_unlinks); +/** + * Returns the number of unlinks in progress. + * + * This function provides the application with a method to detect when an + * unlink has been completed by the implementation. See *rte_event_port_unlink* + * on how to issue unlink requests. + * + * @param dev_id + * The indentifier of the device. + * + * @param port_id + * Event port identifier to select port to check for unlinks in progress. + * + * @return + * The number of unlinks that are in progress. A return of zero indicates that + * there are no outstanding unlink requests. A positive return value indicates + * the number of unlinks that are in progress, but are not yet complete. + * A negative return value indicates an error, -EINVAL indicates an invalid + * parameter passed for *dev_id* or *port_id*. + */ +int __rte_experimental +rte_event_port_unlinks_in_progress(uint8_t dev_id, uint8_t port_id); + /** * Retrieve the list of source event queues and its associated service priority * linked to the destination event port designated by its *port_id* diff --git a/lib/librte_eventdev/rte_eventdev_pmd.h b/lib/librte_eventdev/rte_eventdev_pmd.h index 3fbb4d2b2..65645730a 100644 --- a/lib/librte_eventdev/rte_eventdev_pmd.h +++ b/lib/librte_eventdev/rte_eventdev_pmd.h @@ -332,6 +332,23 @@ typedef int (*eventdev_port_link_t)(struct rte_eventdev *dev, void *port, typedef int (*eventdev_port_unlink_t)(struct rte_eventdev *dev, void *port, uint8_t queues[], uint16_t nb_unlinks); +/** + * Unlinks in progress. Returns number of unlinks that the PMD is currently + * performing, but have not yet been completed. + * + * @param dev + * Event device pointer + * + * @param port + * Event port pointer + * + * @return + * Returns the number of in-progress unlinks. Zero is returned if none are + * in progress. + */ +typedef int (*eventdev_port_unlinks_in_progress_t)(struct rte_eventdev *dev, + void *port); + /** * Converts nanoseconds to *timeout_ticks* value for rte_event_dequeue() * @@ -815,6 +832,8 @@ struct rte_eventdev_ops { /**< Link event queues to an event port. */ eventdev_port_unlink_t port_unlink; /**< Unlink event queues from an event port. */ + eventdev_port_unlinks_in_progress_t port_unlinks_in_progress; + /**< Unlinks in progress on an event port. */ eventdev_dequeue_timeout_ticks_t timeout_ticks; /**< Converts ns to *timeout_ticks* value for rte_event_dequeue() */ eventdev_dump_t dump; diff --git a/lib/librte_eventdev/rte_eventdev_version.map b/lib/librte_eventdev/rte_eventdev_version.map index 12835e9f2..24e7a45c0 100644 --- a/lib/librte_eventdev/rte_eventdev_version.map +++ b/lib/librte_eventdev/rte_eventdev_version.map @@ -96,6 +96,7 @@ EXPERIMENTAL { rte_event_crypto_adapter_stats_reset; rte_event_crypto_adapter_stop; rte_event_eth_rx_adapter_cb_register; + rte_event_port_unlinks_in_progress; rte_event_timer_adapter_caps_get; rte_event_timer_adapter_create; rte_event_timer_adapter_create_ext; From patchwork Wed Sep 12 16:16:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 44621 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E59AE4CE4; Wed, 12 Sep 2018 18:16:26 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id A850B4CC5 for ; Wed, 12 Sep 2018 18:16:25 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Sep 2018 09:16:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,365,1531810800"; d="scan'208";a="73717366" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.187]) by orsmga006.jf.intel.com with ESMTP; 12 Sep 2018 09:16:23 -0700 From: Harry van Haaren To: dev@dpdk.org Cc: jerin.jacob@caviumnetworks.com, matias.elo@nokia.com, Harry van Haaren Date: Wed, 12 Sep 2018 17:16:15 +0100 Message-Id: <20180912161616.42306-2-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180912161616.42306-1-harry.van.haaren@intel.com> References: <20180912161616.42306-1-harry.van.haaren@intel.com> Subject: [dpdk-dev] [PATCH 2/3] event/sw: implement unlinks in progress function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds a counter to each port, which counts the number of unlinks that have been performed. When the scheduler thread starts its scheduling routine, it "acks" all unlinks that have been requested, and the application is gauranteed that no more events will be scheduled to the port from the unlinked queue. Signed-off-by: Harry van Haaren --- drivers/event/sw/sw_evdev.c | 12 ++++++++++++ drivers/event/sw/sw_evdev.h | 8 ++++++++ drivers/event/sw/sw_evdev_scheduler.c | 7 ++++++- 3 files changed, 26 insertions(+), 1 deletion(-) diff --git a/drivers/event/sw/sw_evdev.c b/drivers/event/sw/sw_evdev.c index a6bb91388..9e1412537 100644 --- a/drivers/event/sw/sw_evdev.c +++ b/drivers/event/sw/sw_evdev.c @@ -113,9 +113,20 @@ sw_port_unlink(struct rte_eventdev *dev, void *port, uint8_t queues[], } } } + + p->unlinks_in_progress += unlinked; + rte_smp_mb(); + return unlinked; } +static int +sw_port_unlinks_in_progress(struct rte_eventdev *dev, void *port) +{ + struct sw_port *p = port; + return p->unlinks_in_progress; +} + static int sw_port_setup(struct rte_eventdev *dev, uint8_t port_id, const struct rte_event_port_conf *conf) @@ -925,6 +936,7 @@ sw_probe(struct rte_vdev_device *vdev) .port_release = sw_port_release, .port_link = sw_port_link, .port_unlink = sw_port_unlink, + .port_unlinks_in_progress = sw_port_unlinks_in_progress, .eth_rx_adapter_caps_get = sw_eth_rx_adapter_caps_get, diff --git a/drivers/event/sw/sw_evdev.h b/drivers/event/sw/sw_evdev.h index d90b96d4b..7c77b2495 100644 --- a/drivers/event/sw/sw_evdev.h +++ b/drivers/event/sw/sw_evdev.h @@ -148,6 +148,14 @@ struct sw_port { /* A numeric ID for the port */ uint8_t id; + /* An atomic counter for when the port has been unlinked, and the + * scheduler has not yet acked this unlink - hence there may still be + * events in the buffers going to the port. When the unlinks in + * progress is read by the scheduler, no more events will be pushed to + * the port - hence the scheduler core can just assign zero. + */ + uint8_t unlinks_in_progress; + int16_t is_directed; /** Takes from a single directed QID */ /** * For loadbalanced we can optimise pulling packets from diff --git a/drivers/event/sw/sw_evdev_scheduler.c b/drivers/event/sw/sw_evdev_scheduler.c index e3a41e02f..9b54d5ce7 100644 --- a/drivers/event/sw/sw_evdev_scheduler.c +++ b/drivers/event/sw/sw_evdev_scheduler.c @@ -517,13 +517,18 @@ sw_event_schedule(struct rte_eventdev *dev) /* Pull from rx_ring for ports */ do { in_pkts = 0; - for (i = 0; i < sw->port_count; i++) + for (i = 0; i < sw->port_count; i++) { + /* ack the unlinks in progress as done */ + if (sw->ports[i].unlinks_in_progress) + sw->ports[i].unlinks_in_progress = 0; + if (sw->ports[i].is_directed) in_pkts += sw_schedule_pull_port_dir(sw, i); else if (sw->ports[i].num_ordered_qids > 0) in_pkts += sw_schedule_pull_port_lb(sw, i); else in_pkts += sw_schedule_pull_port_no_reorder(sw, i); + } /* QID scan for re-ordered */ in_pkts += sw_schedule_reorder(sw, 0, From patchwork Wed Sep 12 16:16:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 44622 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DABE55592; Wed, 12 Sep 2018 18:16:31 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id E01945398 for ; Wed, 12 Sep 2018 18:16:30 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Sep 2018 09:16:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,365,1531810800"; d="scan'208";a="73717382" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.187]) by orsmga006.jf.intel.com with ESMTP; 12 Sep 2018 09:16:29 -0700 From: Harry van Haaren To: dev@dpdk.org Cc: jerin.jacob@caviumnetworks.com, matias.elo@nokia.com, Harry van Haaren Date: Wed, 12 Sep 2018 17:16:16 +0100 Message-Id: <20180912161616.42306-3-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180912161616.42306-1-harry.van.haaren@intel.com> References: <20180912161616.42306-1-harry.van.haaren@intel.com> Subject: [dpdk-dev] [PATCH 3/3] event/sw: add unit test for unlinks in progress X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds a unit test that checks the behaviour of the unlinks_in_progress() function, ensuring that the returned values are the number of unlinks requested, until the scheduler runs and "acks" the requests, after which the count should be zero again. Signed-off-by: Harry van Haaren --- drivers/event/sw/sw_evdev_selftest.c | 76 ++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) diff --git a/drivers/event/sw/sw_evdev_selftest.c b/drivers/event/sw/sw_evdev_selftest.c index c40912db5..374abef7c 100644 --- a/drivers/event/sw/sw_evdev_selftest.c +++ b/drivers/event/sw/sw_evdev_selftest.c @@ -1903,6 +1903,77 @@ qid_priorities(struct test *t) return 0; } +static int +unlink_in_progress(struct test *t) +{ + /* Test unlinking API, in particular that when an unlink request has + * not yet been seen by the scheduler thread, that the + * unlink_in_progress() function returns the number of unlinks. + */ + unsigned int i; + /* Create instance with 1 ports, and 3 qids */ + if (init(t, 3, 1) < 0 || + create_ports(t, 1) < 0) { + printf("%d: Error initializing device\n", __LINE__); + return -1; + } + + for (i = 0; i < 3; i++) { + /* Create QID */ + const struct rte_event_queue_conf conf = { + .schedule_type = RTE_SCHED_TYPE_ATOMIC, + /* increase priority (0 == highest), as we go */ + .priority = RTE_EVENT_DEV_PRIORITY_NORMAL - i, + .nb_atomic_flows = 1024, + .nb_atomic_order_sequences = 1024, + }; + + if (rte_event_queue_setup(evdev, i, &conf) < 0) { + printf("%d: error creating qid %d\n", __LINE__, i); + return -1; + } + t->qid[i] = i; + } + t->nb_qids = i; + /* map all QIDs to port */ + rte_event_port_link(evdev, t->port[0], NULL, NULL, 0); + + if (rte_event_dev_start(evdev) < 0) { + printf("%d: Error with start call\n", __LINE__); + return -1; + } + + /* unlink all ports to have outstanding unlink requests */ + int ret = rte_event_port_unlink(evdev, t->port[0], NULL, 0); + if (ret < 0) { + printf("%d: Failed to unlink queues\n", __LINE__); + return -1; + } + + /* get active unlinks here, expect 3 */ + int unlinks_in_progress = + rte_event_port_unlinks_in_progress(evdev, t->port[0]); + if (unlinks_in_progress != 3) { + printf("%d: Expected num unlinks in progress == 3, got %d\n", + __LINE__, unlinks_in_progress); + return -1; + } + + /* run scheduler service on this thread to ack the unlinks */ + rte_service_run_iter_on_app_lcore(t->service_id, 1); + + /* active unlinks expected as 0 as scheduler thread has acked */ + unlinks_in_progress = + rte_event_port_unlinks_in_progress(evdev, t->port[0]); + if (unlinks_in_progress != 0) { + printf("%d: Expected num unlinks in progress == 0, got %d\n", + __LINE__, unlinks_in_progress); + } + + cleanup(t); + return 0; +} + static int load_balancing(struct test *t) { @@ -3260,6 +3331,11 @@ test_sw_eventdev(void) printf("ERROR - QID Priority test FAILED.\n"); goto test_fail; } + ret = unlink_in_progress(t); + if (ret != 0) { + printf("ERROR - Unlink in progress test FAILED.\n"); + goto test_fail; + } printf("*** Running Ordered Reconfigure test...\n"); ret = ordered_reconfigure(t); if (ret != 0) {