From patchwork Tue Feb 22 16:06:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 107992 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 71378A0352; Tue, 22 Feb 2022 17:08:46 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0DCEE41C2E; Tue, 22 Feb 2022 17:07:16 +0100 (CET) Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by mails.dpdk.org (Postfix) with ESMTP id 3306F411DF for ; Tue, 22 Feb 2022 17:07:13 +0100 (CET) Received: by mail-ej1-f52.google.com with SMTP id r13so21335714ejd.5 for ; Tue, 22 Feb 2022 08:07:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oCWqJTcYdWnvA0jll4GvmUtKZbrObkrt69QJcxg4sj8=; b=71RUBrUd6J90dgpL9mEfkJsqD5eAUF+7l1WjGN0HmXMBgSd064O46bQKAMVwtjTCf+ AxFQ3qo1Y7p44ZosBBFkyU+VBa2iFfQgQxZOxB6CCKCUURZUPlvuz7Ypz9EfWtE47nyi mRqELMr4QyGh7z7oSxSFbxi21PeC7LpCGy05C6qVv6TU/dTdU9DjiSm6wMMGTS+goU69 nQlo7Vm0shLyur1LC6sH5hV/He5nVyOb1oGVNArhpd6dJmjB+ZfJGH2RXJBlHQ/OhonO x4Ja+t5feynNZxMkcRzA/qnRaIb0MEiESPAhEPZlulb5LhkVW7eRloQfZfNaY0rFZPF5 tEtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oCWqJTcYdWnvA0jll4GvmUtKZbrObkrt69QJcxg4sj8=; b=dxnHUWFLikdF0RK8bHOS0K5/cIAkAiRWa7BSIM7iZLvx2ZX9ThQ8dKIFNGv7fCcr72 3CAckE8Tl5N7vwxlGpgNYbD/DJftHF08dlMtHkzqf+4HNN2S1vv1tlLckWqr9ZiWLQ9q VGAXheIpigFZdw24E5/u8OIEniqRxiBqhXF51+IrWkkDBo687D6UJIWrSmCGXbvaWYUN bmXgMV+GJHTxBT8RWfUMxgTUtPu7B16boD3a3fUUI60RC6zCld6FcQoD0lm6u/2BzWky C9p4KCX7Ow1e4dVLLAcJFiiAznlyXI52lYPUCnJN4Wk4M3wBD19wDdvPkKgp737QZ8hA jHnQ== X-Gm-Message-State: AOAM530CkpGdhy+wBvL2mJEcvZFQSz1aLL806Ye8rX4piuK5tebnccFH Y3l6NXUmBY6RgRa6Rsf+ILurzsMzO3vOcg== X-Google-Smtp-Source: ABdhPJxJztcRitKwv7T0W0xAcc6D6CWo70Vcp8gyh8htGib4lliG7pycSA8CXJWuYS5oV9VsvNN3cw== X-Received: by 2002:a17:906:6d12:b0:6cd:6970:2d4c with SMTP id m18-20020a1709066d1200b006cd69702d4cmr19888009ejr.323.1645546032419; Tue, 22 Feb 2022 08:07:12 -0800 (PST) Received: from DESKTOP-U5LNN3J.localdomain (89-79-181-52.dynamic.chello.pl. [89.79.181.52]) by smtp.gmail.com with ESMTPSA id x6sm10013477edv.109.2022.02.22.08.07.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Feb 2022 08:07:11 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: shaibran@amazon.com, upstream@semihalf.com, Michal Krawczyk , Dawid Gorecki Subject: [PATCH 16/21] net/ena: make Tx completion timeout configurable Date: Tue, 22 Feb 2022 17:06:29 +0100 Message-Id: <20220222160634.24489-17-mk@semihalf.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220222160634.24489-1-mk@semihalf.com> References: <20220222160634.24489-1-mk@semihalf.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The default missing Tx completion timeout was set to 5 seconds. In order to provide users with the interface to control this timeout to adjust it with the application's watchdog, the device argument for controlling this value was added. The parameter is called 'miss_txc_to' and can be modified using the devargs interface: ./app -a ,miss_txc_to=UINT_NUMBER This parameter accepts values from 0 to 60 and indicates number of seconds after which the Tx packet will be considered as missing. HW hints for the Tx completions timeout were removed to do not overwrite parameter from the user. Also specyfing default Tx completion timeout value was moved from the configuration to init phase in order to simplify default value assignment. Signed-off-by: Michal Krawczyk Reviewed-by: Dawid Gorecki Reviewed-by: Shai Brandes --- doc/guides/nics/ena.rst | 9 ++++ doc/guides/rel_notes/release_22_03.rst | 1 + drivers/net/ena/ena_ethdev.c | 66 ++++++++++++++++++++------ drivers/net/ena/ena_ethdev.h | 1 + 4 files changed, 62 insertions(+), 15 deletions(-) diff --git a/doc/guides/nics/ena.rst b/doc/guides/nics/ena.rst index 85c790e80f..3d780acac9 100644 --- a/doc/guides/nics/ena.rst +++ b/doc/guides/nics/ena.rst @@ -87,6 +87,15 @@ Configuration information effect only if the device also supports large LLQ headers. Otherwise, the default value will be used. + * **miss_txc_to** (default 5) + + Number of seconds after which the Tx packet will be considered missing. + If the missing packets number will exceed dynamically calculated threshold, + the driver will trigger the device reset which should be handled by the + application. Checking for missing Tx completions happens in the driver's + timer service. Setting this parameter to 0 disables this feature. Maximum + allowed value is 60 seconds. + **ENA Configuration Parameters** * **Number of Queues** diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst index 1b87b9c174..d0d2824371 100644 --- a/doc/guides/rel_notes/release_22_03.rst +++ b/doc/guides/rel_notes/release_22_03.rst @@ -116,6 +116,7 @@ New Features * Added ENA admin queue support for the MP applications. * Added free Tx mbuf on demand feature support. * Added ``rte_eth_xstats_get_names_by_id`` API support. + * Added ``miss_txc_to`` device argument for setting the Tx completion timeout. * **Updated Cisco enic driver.** diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index d413111b08..4bb2b195b1 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -62,6 +62,10 @@ struct ena_stats { /* Device arguments */ #define ENA_DEVARG_LARGE_LLQ_HDR "large_llq_hdr" +/* Timeout in seconds after which a single uncompleted Tx packet should be + * considered as a missing. + */ +#define ENA_DEVARG_MISS_TXC_TO "miss_txc_to" /* * Each rte_memzone should have unique name. @@ -2132,6 +2136,8 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) snprintf(adapter->name, ENA_NAME_MAX_LEN, "ena_%d", adapter->id_number); + adapter->missing_tx_completion_to = ENA_TX_TIMEOUT; + rc = ena_parse_devargs(adapter, pci_dev->device.devargs); if (rc != 0) { PMD_INIT_LOG(CRIT, "Failed to parse devargs\n"); @@ -2307,7 +2313,6 @@ static int ena_dev_configure(struct rte_eth_dev *dev) adapter->missing_tx_completion_budget = RTE_MIN(ENA_MONITORED_TX_QUEUES, dev->data->nb_tx_queues); - adapter->missing_tx_completion_to = ENA_TX_TIMEOUT; /* To avoid detection of the spurious Tx completion timeout due to * application not calling the Tx cleanup function, set timeout for the * Tx queue which should be half of the missing completion timeout for a @@ -2830,20 +2835,6 @@ static void ena_update_hints(struct ena_adapter *adapter, adapter->ena_dev.mmio_read.reg_read_to = hints->mmio_read_timeout * 1000; - if (hints->missing_tx_completion_timeout) { - if (hints->missing_tx_completion_timeout == - ENA_HW_HINTS_NO_TIMEOUT) { - adapter->missing_tx_completion_to = - ENA_HW_HINTS_NO_TIMEOUT; - } else { - /* Convert from msecs to ticks */ - adapter->missing_tx_completion_to = rte_get_timer_hz() * - hints->missing_tx_completion_timeout / 1000; - adapter->tx_cleanup_stall_delay = - adapter->missing_tx_completion_to / 2; - } - } - if (hints->driver_watchdog_timeout) { if (hints->driver_watchdog_timeout == ENA_HW_HINTS_NO_TIMEOUT) adapter->keep_alive_timeout = ENA_HW_HINTS_NO_TIMEOUT; @@ -3396,6 +3387,45 @@ static int ena_xstats_get_by_id(struct rte_eth_dev *dev, return valid; } +static int ena_process_uint_devarg(const char *key, + const char *value, + void *opaque) +{ + struct ena_adapter *adapter = opaque; + char *str_end; + uint64_t uint_value; + + uint_value = strtoull(value, &str_end, 10); + if (value == str_end) { + PMD_INIT_LOG(ERR, + "Invalid value for key '%s'. Only uint values are accepted.\n", + key); + return -EINVAL; + } + + if (strcmp(key, ENA_DEVARG_MISS_TXC_TO) == 0) { + if (uint_value > ENA_MAX_TX_TIMEOUT_SECONDS) { + PMD_INIT_LOG(ERR, + "Tx timeout too high: %" PRIu64 " sec. Maximum allowed: %d sec.\n", + uint_value, ENA_MAX_TX_TIMEOUT_SECONDS); + return -EINVAL; + } else if (uint_value == 0) { + PMD_INIT_LOG(INFO, + "Check for missing Tx completions has been disabled.\n"); + adapter->missing_tx_completion_to = + ENA_HW_HINTS_NO_TIMEOUT; + } else { + PMD_INIT_LOG(INFO, + "Tx packet completion timeout set to %" PRIu64 " seconds.\n", + uint_value); + adapter->missing_tx_completion_to = + uint_value * rte_get_timer_hz(); + } + } + + return 0; +} + static int ena_process_bool_devarg(const char *key, const char *value, void *opaque) @@ -3427,6 +3457,7 @@ static int ena_parse_devargs(struct ena_adapter *adapter, { static const char * const allowed_args[] = { ENA_DEVARG_LARGE_LLQ_HDR, + ENA_DEVARG_MISS_TXC_TO, NULL, }; struct rte_kvargs *kvlist; @@ -3444,7 +3475,12 @@ static int ena_parse_devargs(struct ena_adapter *adapter, rc = rte_kvargs_process(kvlist, ENA_DEVARG_LARGE_LLQ_HDR, ena_process_bool_devarg, adapter); + if (rc != 0) + goto exit; + rc = rte_kvargs_process(kvlist, ENA_DEVARG_MISS_TXC_TO, + ena_process_uint_devarg, adapter); +exit: rte_kvargs_free(kvlist); return rc; diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index ca3e5ed691..c0094b03ee 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -40,6 +40,7 @@ #define ENA_DEVICE_KALIVE_TIMEOUT (ENA_WD_TIMEOUT_SEC * rte_get_timer_hz()) #define ENA_TX_TIMEOUT (5 * rte_get_timer_hz()) +#define ENA_MAX_TX_TIMEOUT_SECONDS 60 #define ENA_MONITORED_TX_QUEUES 3 #define ENA_DEFAULT_MISSING_COMP 256U