From patchwork Sat Jul 2 16:03:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timothy McDaniel X-Patchwork-Id: 113640 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24DFBA0093; Sat, 2 Jul 2022 18:04:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CBEFB42B71; Sat, 2 Jul 2022 18:03:50 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id 5A57540E50 for ; Sat, 2 Jul 2022 18:03:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656777826; x=1688313826; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iPgi5J2UBlJ7Taa4Usg5YEAsg77772/o3ji4TnWZ49k=; b=Nfi8PZR4dvSv5y3GNjGjESD1ZOmp7XZxIKOpGKAGf/cfZtwERtIajbm7 fWRk50/oEEN2NNdFphkZwZ2pq6FSrvgDXIAwcnQZxEHYVzrqfcQ7MR6QO SDXfRM0GDL9jMrrXuCLw/wch3mW3sRArX5CSLr7d62k2pP2IfPxr7Bh/V Igpw4YjbZDZG0ozm1wE0JJgMVWOQuaA5kbx4v0dnehqN1cIi/a8ilKN/G 3fi+7iN1q3X1WNCDa3kzqGmhaAL1Id3U7dL53WK9P5wzkgWe266moA/LO IG38dbonCPVszYgGKsBdmZEPFrFHMahzHOAWTDIEQIvr1iQZG5zQhphpd A==; X-IronPort-AV: E=McAfee;i="6400,9594,10396"; a="271616952" X-IronPort-AV: E=Sophos;i="5.92,240,1650956400"; d="scan'208";a="271616952" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jul 2022 09:03:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,240,1650956400"; d="scan'208";a="648776830" Received: from txanpdk03.an.intel.com ([10.123.117.78]) by fmsmga008.fm.intel.com with ESMTP; 02 Jul 2022 09:03:44 -0700 From: Timothy McDaniel To: jerinj@marvell.com Cc: dev@dpdk.org, timothy.mcdaniel@intel.com Subject: [PATCH v2 4/5] event/dlb2: fix cq depth override credit deadlock Date: Sat, 2 Jul 2022 11:03:39 -0500 Message-Id: <20220702160340.1591058-5-timothy.mcdaniel@intel.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220702160340.1591058-1-timothy.mcdaniel@intel.com> References: <20220629153638.1269743-1-timothy.mcdaniel@intel.com> <20220702160340.1591058-1-timothy.mcdaniel@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This commit fixes a bug, where we could encounter a credit deadlock due to changing the CQ depth. To remedy this situation, the commit reduces the maximum CQ depth from 1024 to 128, and also allows configuring the maximum enqueue depth. Maximum enqueue depth must be tuned to the CQ depth, if the CQ depth is increased. Fixes: 86fe66d45667 ("event/dlb2: allow CQ depths up to 1024") Cc: timothy.mcdaniel@intel.com Signed-off-by: Timothy McDaniel --- drivers/event/dlb2/dlb2.c | 51 +++++++++++++++++++++++++++++++++ drivers/event/dlb2/dlb2_priv.h | 8 +++++- drivers/event/dlb2/pf/dlb2_pf.c | 3 +- 3 files changed, 60 insertions(+), 2 deletions(-) diff --git a/drivers/event/dlb2/dlb2.c b/drivers/event/dlb2/dlb2.c index b50cd8e5ce..8a68c25c93 100644 --- a/drivers/event/dlb2/dlb2.c +++ b/drivers/event/dlb2/dlb2.c @@ -326,6 +326,36 @@ set_max_cq_depth(const char *key __rte_unused, return 0; } +static int +set_max_enq_depth(const char *key __rte_unused, + const char *value, + void *opaque) +{ + int *max_enq_depth = opaque; + int ret; + + if (value == NULL || opaque == NULL) { + DLB2_LOG_ERR("NULL pointer\n"); + return -EINVAL; + } + + ret = dlb2_string_to_int(max_enq_depth, value); + if (ret < 0) + return ret; + + if (*max_enq_depth < DLB2_MIN_ENQ_DEPTH_OVERRIDE || + *max_enq_depth > DLB2_MAX_ENQ_DEPTH_OVERRIDE || + !rte_is_power_of_2(*max_enq_depth)) { + DLB2_LOG_ERR("dlb2: max_enq_depth %d and %d and a power of 2\n", + DLB2_MIN_ENQ_DEPTH_OVERRIDE, + DLB2_MAX_ENQ_DEPTH_OVERRIDE); + return -EINVAL; + } + + return 0; +} + + static int set_max_num_events(const char *key __rte_unused, const char *value, @@ -4514,6 +4544,15 @@ dlb2_primary_eventdev_probe(struct rte_eventdev *dev, evdev_dlb2_default_info.max_event_port_dequeue_depth = dlb2->max_cq_depth; + if (dlb2_args->max_enq_depth != 0) + dlb2->max_enq_depth = dlb2_args->max_enq_depth; + else + dlb2->max_enq_depth = DLB2_DEFAULT_CQ_DEPTH; + + evdev_dlb2_default_info.max_event_port_enqueue_depth = + dlb2->max_enq_depth; + + err = dlb2_iface_open(&dlb2->qm_instance, name); if (err < 0) { DLB2_LOG_ERR("could not open event hardware device, err=%d\n", @@ -4650,6 +4689,7 @@ dlb2_parse_params(const char *params, DLB2_DEPTH_THRESH_ARG, DLB2_VECTOR_OPTS_ENAB_ARG, DLB2_MAX_CQ_DEPTH, + DLB2_MAX_ENQ_DEPTH, DLB2_CQ_WEIGHT, DLB2_PORT_COS, DLB2_COS_BW, @@ -4789,6 +4829,17 @@ dlb2_parse_params(const char *params, return ret; } + ret = rte_kvargs_process(kvlist, + DLB2_MAX_ENQ_DEPTH, + set_max_enq_depth, + &dlb2_args->max_enq_depth); + if (ret != 0) { + DLB2_LOG_ERR("%s: Error parsing vector opts enabled", + name); + rte_kvargs_free(kvlist); + return ret; + } + ret = rte_kvargs_process(kvlist, DLB2_CQ_WEIGHT, set_cq_weight, diff --git a/drivers/event/dlb2/dlb2_priv.h b/drivers/event/dlb2/dlb2_priv.h index 8744efa79d..1edea83a5b 100644 --- a/drivers/event/dlb2/dlb2_priv.h +++ b/drivers/event/dlb2/dlb2_priv.h @@ -29,7 +29,10 @@ #define DLB2_SW_CREDIT_C_QUANTA_DEFAULT 256 /* Consumer */ #define DLB2_DEPTH_THRESH_DEFAULT 256 #define DLB2_MIN_CQ_DEPTH_OVERRIDE 32 -#define DLB2_MAX_CQ_DEPTH_OVERRIDE 1024 +#define DLB2_MAX_CQ_DEPTH_OVERRIDE 128 +#define DLB2_MIN_ENQ_DEPTH_OVERRIDE 32 +#define DLB2_MAX_ENQ_DEPTH_OVERRIDE 1024 + /* command line arg strings */ #define NUMA_NODE_ARG "numa_node" @@ -44,6 +47,7 @@ #define DLB2_DEPTH_THRESH_ARG "default_depth_thresh" #define DLB2_VECTOR_OPTS_ENAB_ARG "vector_opts_enable" #define DLB2_MAX_CQ_DEPTH "max_cq_depth" +#define DLB2_MAX_ENQ_DEPTH "max_enqueue_depth" #define DLB2_CQ_WEIGHT "cq_weight" #define DLB2_PORT_COS "port_cos" #define DLB2_COS_BW "cos_bw" @@ -585,6 +589,7 @@ struct dlb2_eventdev { int num_dir_credits_override; bool vector_opts_enabled; int max_cq_depth; + int max_enq_depth; volatile enum dlb2_run_state run_state; uint16_t num_dir_queues; /* total num of evdev dir queues requested */ union { @@ -660,6 +665,7 @@ struct dlb2_devargs { int default_depth_thresh; bool vector_opts_enabled; int max_cq_depth; + int max_enq_depth; struct dlb2_cq_weight cq_weight; struct dlb2_port_cos port_cos; struct dlb2_cos_bw cos_bw; diff --git a/drivers/event/dlb2/pf/dlb2_pf.c b/drivers/event/dlb2/pf/dlb2_pf.c index 0627f06a6e..086d4a1cc7 100644 --- a/drivers/event/dlb2/pf/dlb2_pf.c +++ b/drivers/event/dlb2/pf/dlb2_pf.c @@ -708,7 +708,8 @@ dlb2_eventdev_pci_init(struct rte_eventdev *eventdev) .sw_credit_quanta = DLB2_SW_CREDIT_QUANTA_DEFAULT, .hw_credit_quanta = DLB2_SW_CREDIT_BATCH_SZ, .default_depth_thresh = DLB2_DEPTH_THRESH_DEFAULT, - .max_cq_depth = DLB2_DEFAULT_CQ_DEPTH + .max_cq_depth = DLB2_DEFAULT_CQ_DEPTH, + .max_enq_depth = DLB2_MAX_ENQUEUE_DEPTH }; struct dlb2_eventdev *dlb2;