From patchwork Thu Sep 29 15:26:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sevincer, Abdullah" X-Patchwork-Id: 117147 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA63BA034C; Thu, 29 Sep 2022 17:31:23 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A6A2E41143; Thu, 29 Sep 2022 17:31:18 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id 95C5340395 for ; Thu, 29 Sep 2022 17:31:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1664465476; x=1696001476; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VkabtjjukNabY6kku20RObaxlU9mXIoYFE3pxHo0LkA=; b=PK9oQ0l6phE563ACMcZKUzuS7CWmWE5DY9+62XM4NId2AwUyNBl6KrwV kH3XWeecCLxxmYDCMDt/3UsGdUYx8gTpQHz7EN59FVavnI2A4BETEDYSg x22V9PVrPuRdupEnk3zcUzHc398S2G3fKPtgCsgJ7qGc9dFw/eVSTYRMC KJOCO6Ef2wTlPjkuxxT3p0QgU/Yacb4ELDrvXRrf8CG0HzECLUvKJ5YGm WBf4RkT6pPBpu7MrSaYx8RuneJZNqzcDliKQ4BGoRzIoXbHgjRBYnCpwk QoQgAL0fs9mCPu7HkUhCUvrQ4fZLtAweUEWf1o8lkUuZ4ntHjX3kcelKh A==; X-IronPort-AV: E=McAfee;i="6500,9779,10485"; a="289092521" X-IronPort-AV: E=Sophos;i="5.93,355,1654585200"; d="scan'208";a="289092521" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2022 08:26:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10485"; a="797596657" X-IronPort-AV: E=Sophos;i="5.93,355,1654585200"; d="scan'208";a="797596657" Received: from txanpdk02.an.intel.com ([10.123.117.76]) by orsmga005.jf.intel.com with ESMTP; 29 Sep 2022 08:26:29 -0700 From: Abdullah Sevincer To: dev@dpdk.org Cc: jerinj@marvell.com, Abdullah Sevincer Subject: [PATCH v11 2/3] event/dlb2: add fence bypass option for producer ports Date: Thu, 29 Sep 2022 10:26:25 -0500 Message-Id: <20220929152626.1719144-2-abdullah.sevincer@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220929152626.1719144-1-abdullah.sevincer@intel.com> References: <20220820005957.2986689-2-timothy.mcdaniel@intel.com> <20220929152626.1719144-1-abdullah.sevincer@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org If producer thread is only acting as a bridge between NIC and DLB, then performance can be greatly improved by bypassing the fence instruction. DLB enqueue API calls memory fence once per enqueue burst. If prodcuer thread is just reading from NIC and sending to DLB without updating the read buffers or buffer headers OR producer is not writing to data structures with dependencies on the enqueue write order, then fencing can be safely disabled. Signed-off-by: Abdullah Sevincer --- drivers/event/dlb2/dlb2.c | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/drivers/event/dlb2/dlb2.c b/drivers/event/dlb2/dlb2.c index 6a9db4b642..4dd1d55ddc 100644 --- a/drivers/event/dlb2/dlb2.c +++ b/drivers/event/dlb2/dlb2.c @@ -35,6 +35,16 @@ #include "dlb2_iface.h" #include "dlb2_inline_fns.h" +/* + * Bypass memory fencing instructions when port is of Producer type. + * This should be enabled very carefully with understanding that producer + * is not doing any writes which need fencing. The movdir64 instruction used to + * enqueue events to DLB is a weakly-ordered instruction and movdir64 write + * to DLB can go ahead of relevant application writes like updates to buffers + * being sent with event + */ +#define DLB2_BYPASS_FENCE_ON_PP 0 /* 1 == Bypass fence, 0 == do not bypass */ + /* * Resources exposed to eventdev. Some values overridden at runtime using * values returned by the DLB kernel driver. @@ -1985,21 +1995,15 @@ dlb2_eventdev_port_setup(struct rte_eventdev *dev, sw_credit_quanta = dlb2->sw_credit_quanta; hw_credit_quanta = dlb2->hw_credit_quanta; + ev_port->qm_port.is_producer = false; ev_port->qm_port.is_directed = port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_SINGLE_LINK; - /* - * Validate credit config before creating port - */ - - /* Default for worker ports */ - sw_credit_quanta = dlb2->sw_credit_quanta; - hw_credit_quanta = dlb2->hw_credit_quanta; - if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_HINT_PRODUCER) { /* Producer type ports. Mostly enqueue */ sw_credit_quanta = DLB2_SW_CREDIT_P_QUANTA_DEFAULT; hw_credit_quanta = DLB2_SW_CREDIT_P_BATCH_SZ; + ev_port->qm_port.is_producer = true; } if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_HINT_CONSUMER) { /* Consumer type ports. Mostly dequeue */ @@ -2009,6 +2013,10 @@ dlb2_eventdev_port_setup(struct rte_eventdev *dev, ev_port->credit_update_quanta = sw_credit_quanta; ev_port->qm_port.hw_credit_quanta = hw_credit_quanta; + /* + * Validate credit config before creating port + */ + if (port_conf->enqueue_depth > sw_credit_quanta || port_conf->enqueue_depth > hw_credit_quanta) { DLB2_LOG_ERR("Invalid port config. Enqueue depth %d must be <= credit quanta %d and batch size %d\n", @@ -3073,7 +3081,12 @@ __dlb2_event_enqueue_burst(void *event_port, dlb2_event_build_hcws(qm_port, &events[i], j - pop_offs, sched_types, queue_ids); +#if DLB2_BYPASS_FENCE_ON_PP == 1 + /* Bypass fence instruction for producer ports */ + dlb2_hw_do_enqueue(qm_port, i == 0 && !qm_port->is_producer, port_data); +#else dlb2_hw_do_enqueue(qm_port, i == 0, port_data); +#endif /* Don't include the token pop QE in the enqueue count */ i += j - pop_offs;