From patchwork Wed Feb 23 12:19:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 108160 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 96E37A034C; Wed, 23 Feb 2022 13:22:01 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1912742726; Wed, 23 Feb 2022 13:20:54 +0100 (CET) Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) by mails.dpdk.org (Postfix) with ESMTP id 261BD426FE for ; Wed, 23 Feb 2022 13:20:50 +0100 (CET) Received: by mail-ej1-f44.google.com with SMTP id p9so52009521ejd.6 for ; Wed, 23 Feb 2022 04:20:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2n5HHJon8hGLBAiixQuAOZl6BD+0aVat5XbPEqLE+T8=; b=pt6Ul3SNeCOmct77+bWrCz/TyFVHDFKfsob3F1WpW582Nz+6kbNDGWeJyBa/iWKmBP bUDyQ4CEAlC7MmKvkU+WrE6GJsqo8RWum5e2rrlnANjFjOLra5VCn5RjB/XLLHlXDSxM +lpCH9sbM2Et9kcTrtvCs26nF0FwvLQpGi3Ma4Gdqf5kcBgmsr3QvGUeCWlG5sc+vIGS vl69OVsS4BE0gE08H0yMhjY6I2/048e53rImFvxRd6v7hTUYijnK6nwjOeZYgzTfPwsa j00H5j6yoWANvEYW9FBVMvh+GWKAU4jP6+0+JELcCzm60DnD6/L+y/cwCZ2hVKZuCbEq kh/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2n5HHJon8hGLBAiixQuAOZl6BD+0aVat5XbPEqLE+T8=; b=cP9q1VjLMPz6bf6eoE9WzmBKZX8j9vlgkSbVudh9p3n2mCkk4weVOyqkUwhjDRRrfS A/9vJcF7AGwSob7ciVzfxAWWm6SUEoYXRY/3qpadM0j9eXyy4kJ3f4IDP1I62vkm0S9A qRleR3Ghb4ysD9E/2FagNH4v5SmdpljlNHue+V2OF4wioRdGQsF6wfK9Vfn0Ox7F4YoF /IYxQABX/Cymg8n3HYWztP/wrhK0Ti1WEKfREeM6/cv6FsAocj0iaI64qlt/1kKoI6q6 1Iu7fooFh7OD3qprsI6sZkUAHMWjTWAdxZlpqfkLD0vpgPXcqk+b6TpqQruNs4NzTKc9 zc/w== X-Gm-Message-State: AOAM530m9Yt3lSb1C+xzYfQsppTueFwMvH1PJCNWrq20ulASKrIzvFE6 Z1CigMhDM1MAa2iq80KuzslafB79bEnqYQ== X-Google-Smtp-Source: ABdhPJzPdKEO750PbfVguZnS+86+6c+69ELOr1St7qdQAmr3XpItCGtN/sj4hqvyXhEjMvth6WeExQ== X-Received: by 2002:a17:907:2bc4:b0:6cc:e4b7:dcf8 with SMTP id gv4-20020a1709072bc400b006cce4b7dcf8mr22695919ejc.596.1645618849503; Wed, 23 Feb 2022 04:20:49 -0800 (PST) Received: from DESKTOP-U5LNN3J.localdomain (89-79-181-52.dynamic.chello.pl. [89.79.181.52]) by smtp.gmail.com with ESMTPSA id v12sm11629224edr.8.2022.02.23.04.20.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 04:20:48 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: ferruh.yigit@intel.com, shaibran@amazon.com, upstream@semihalf.com, Michal Krawczyk , Dawid Gorecki Subject: [PATCH v3 16/21] net/ena: make Tx completion timeout configurable Date: Wed, 23 Feb 2022 13:19:39 +0100 Message-Id: <20220223121944.24156-17-mk@semihalf.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223121944.24156-1-mk@semihalf.com> References: <20220222181146.28882-1-mk@semihalf.com> <20220223121944.24156-1-mk@semihalf.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The default missing Tx completion timeout was set to 5 seconds. In order to provide users with the interface to control this timeout to adjust it with the application's watchdog, the device argument for controlling this value was added. The parameter is called 'miss_txc_to' and can be modified using the devargs interface: ./app -a ,miss_txc_to=UINT_NUMBER This parameter accepts values from 0 to 60 and indicates number of seconds after which the Tx packet will be considered as missing. HW hints for the Tx completions timeout were removed to do not overwrite parameter from the user. Also specyfing default Tx completion timeout value was moved from the configuration to init phase in order to simplify default value assignment. Signed-off-by: Michal Krawczyk Reviewed-by: Dawid Gorecki Reviewed-by: Shai Brandes --- doc/guides/nics/ena.rst | 9 ++++ doc/guides/rel_notes/release_22_03.rst | 1 + drivers/net/ena/ena_ethdev.c | 66 ++++++++++++++++++++------ drivers/net/ena/ena_ethdev.h | 1 + 4 files changed, 62 insertions(+), 15 deletions(-) diff --git a/doc/guides/nics/ena.rst b/doc/guides/nics/ena.rst index 85c790e80f..3d780acac9 100644 --- a/doc/guides/nics/ena.rst +++ b/doc/guides/nics/ena.rst @@ -87,6 +87,15 @@ Configuration information effect only if the device also supports large LLQ headers. Otherwise, the default value will be used. + * **miss_txc_to** (default 5) + + Number of seconds after which the Tx packet will be considered missing. + If the missing packets number will exceed dynamically calculated threshold, + the driver will trigger the device reset which should be handled by the + application. Checking for missing Tx completions happens in the driver's + timer service. Setting this parameter to 0 disables this feature. Maximum + allowed value is 60 seconds. + **ENA Configuration Parameters** * **Number of Queues** diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst index 1746abddbb..e04567ed5b 100644 --- a/doc/guides/rel_notes/release_22_03.rst +++ b/doc/guides/rel_notes/release_22_03.rst @@ -116,6 +116,7 @@ New Features * Added ENA admin queue support for the MP applications. * Added free Tx mbuf on demand feature support. * Added ``rte_eth_xstats_get_names_by_id`` API support. + * Added ``miss_txc_to`` device argument for setting the Tx completion timeout. * **Updated Cisco enic driver.** diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 3cdeba9fad..0ca4269522 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -62,6 +62,10 @@ struct ena_stats { /* Device arguments */ #define ENA_DEVARG_LARGE_LLQ_HDR "large_llq_hdr" +/* Timeout in seconds after which a single uncompleted Tx packet should be + * considered as a missing. + */ +#define ENA_DEVARG_MISS_TXC_TO "miss_txc_to" /* * Each rte_memzone should have unique name. @@ -2132,6 +2136,8 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) snprintf(adapter->name, ENA_NAME_MAX_LEN, "ena_%d", adapter->id_number); + adapter->missing_tx_completion_to = ENA_TX_TIMEOUT; + rc = ena_parse_devargs(adapter, pci_dev->device.devargs); if (rc != 0) { PMD_INIT_LOG(CRIT, "Failed to parse devargs\n"); @@ -2307,7 +2313,6 @@ static int ena_dev_configure(struct rte_eth_dev *dev) adapter->missing_tx_completion_budget = RTE_MIN(ENA_MONITORED_TX_QUEUES, dev->data->nb_tx_queues); - adapter->missing_tx_completion_to = ENA_TX_TIMEOUT; /* To avoid detection of the spurious Tx completion timeout due to * application not calling the Tx cleanup function, set timeout for the * Tx queue which should be half of the missing completion timeout for a @@ -2830,20 +2835,6 @@ static void ena_update_hints(struct ena_adapter *adapter, adapter->ena_dev.mmio_read.reg_read_to = hints->mmio_read_timeout * 1000; - if (hints->missing_tx_completion_timeout) { - if (hints->missing_tx_completion_timeout == - ENA_HW_HINTS_NO_TIMEOUT) { - adapter->missing_tx_completion_to = - ENA_HW_HINTS_NO_TIMEOUT; - } else { - /* Convert from msecs to ticks */ - adapter->missing_tx_completion_to = rte_get_timer_hz() * - hints->missing_tx_completion_timeout / 1000; - adapter->tx_cleanup_stall_delay = - adapter->missing_tx_completion_to / 2; - } - } - if (hints->driver_watchdog_timeout) { if (hints->driver_watchdog_timeout == ENA_HW_HINTS_NO_TIMEOUT) adapter->keep_alive_timeout = ENA_HW_HINTS_NO_TIMEOUT; @@ -3396,6 +3387,45 @@ static int ena_xstats_get_by_id(struct rte_eth_dev *dev, return valid; } +static int ena_process_uint_devarg(const char *key, + const char *value, + void *opaque) +{ + struct ena_adapter *adapter = opaque; + char *str_end; + uint64_t uint_value; + + uint_value = strtoull(value, &str_end, 10); + if (value == str_end) { + PMD_INIT_LOG(ERR, + "Invalid value for key '%s'. Only uint values are accepted.\n", + key); + return -EINVAL; + } + + if (strcmp(key, ENA_DEVARG_MISS_TXC_TO) == 0) { + if (uint_value > ENA_MAX_TX_TIMEOUT_SECONDS) { + PMD_INIT_LOG(ERR, + "Tx timeout too high: %" PRIu64 " sec. Maximum allowed: %d sec.\n", + uint_value, ENA_MAX_TX_TIMEOUT_SECONDS); + return -EINVAL; + } else if (uint_value == 0) { + PMD_INIT_LOG(INFO, + "Check for missing Tx completions has been disabled.\n"); + adapter->missing_tx_completion_to = + ENA_HW_HINTS_NO_TIMEOUT; + } else { + PMD_INIT_LOG(INFO, + "Tx packet completion timeout set to %" PRIu64 " seconds.\n", + uint_value); + adapter->missing_tx_completion_to = + uint_value * rte_get_timer_hz(); + } + } + + return 0; +} + static int ena_process_bool_devarg(const char *key, const char *value, void *opaque) @@ -3427,6 +3457,7 @@ static int ena_parse_devargs(struct ena_adapter *adapter, { static const char * const allowed_args[] = { ENA_DEVARG_LARGE_LLQ_HDR, + ENA_DEVARG_MISS_TXC_TO, NULL, }; struct rte_kvargs *kvlist; @@ -3444,7 +3475,12 @@ static int ena_parse_devargs(struct ena_adapter *adapter, rc = rte_kvargs_process(kvlist, ENA_DEVARG_LARGE_LLQ_HDR, ena_process_bool_devarg, adapter); + if (rc != 0) + goto exit; + rc = rte_kvargs_process(kvlist, ENA_DEVARG_MISS_TXC_TO, + ena_process_uint_devarg, adapter); +exit: rte_kvargs_free(kvlist); return rc; diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index ca3e5ed691..c0094b03ee 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -40,6 +40,7 @@ #define ENA_DEVICE_KALIVE_TIMEOUT (ENA_WD_TIMEOUT_SEC * rte_get_timer_hz()) #define ENA_TX_TIMEOUT (5 * rte_get_timer_hz()) +#define ENA_MAX_TX_TIMEOUT_SECONDS 60 #define ENA_MONITORED_TX_QUEUES 3 #define ENA_DEFAULT_MISSING_COMP 256U