From patchwork Tue Feb 22 18:11:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 108024 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ECC17A034E; Tue, 22 Feb 2022 19:13:54 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6B80641C26; Tue, 22 Feb 2022 19:12:38 +0100 (CET) Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) by mails.dpdk.org (Postfix) with ESMTP id 1E6454116A for ; Tue, 22 Feb 2022 19:12:36 +0100 (CET) Received: by mail-ej1-f45.google.com with SMTP id qx21so45342366ejb.13 for ; Tue, 22 Feb 2022 10:12:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oCWqJTcYdWnvA0jll4GvmUtKZbrObkrt69QJcxg4sj8=; b=qV2DVhWjyKRYAN/dt4BZwndxFWQee512nCLQwyd3oe3N/UACvD81K7kfUaD13zL1tv SRHY36PCxIUJLIXs39cuB5MvJJNJmirbYQdoSjqwtfkKw3zY3R806/ongxgKx9pn2byY ZOSdnu7ReWcH51bil6XktnivlNSOHOytBTcDZOks/LavsDSueL22aHysBM15ta4SJs7E ZoPxSvscoKqtXoF8y6n2C2QrHVanxlzShSYdoGIoa/Zpnz+FfqZz8OB7BptbF/ZOHJXq 3StTyKvvVGimgGLbx0OXkJcQS4BWGiq4GlILEYM+ANInAjMCwnceJH7DfHNLk9ofAQ1y 42Pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oCWqJTcYdWnvA0jll4GvmUtKZbrObkrt69QJcxg4sj8=; b=D0/83i0vhbxGA3JcC6KB5Wtd/SNJSSYQYlvpDK/X9SNTRWejhtAnhBG35xyN9Aziqv 4Q+IBFkD606plPAv6VUKJEGrquWYH0nfukBTDsdN2Lp8XflAqlRVLQ6ANwgy5hInqsA0 asPWD7OjJp6CCaCSWyTlxYbGUOqO8vRnhzazI2L5sN+GhOfMXeidTFCGKjaUYz0gcaH4 hTrKOcGTKKOfdFdlisH9uH5nKfz4wyH/Y0jEUFIEzcApWCXjQqVDqnTZp5/UtvQ3leXb c/pNO4Al990icADubDQ/kkzkn+gG1EG0I6Tn4yu13vx/OwhkfGRdLDzxK425+g33jzF9 IjVg== X-Gm-Message-State: AOAM5314YYGb7Pvdmsm9mAG+nDlf8rvWPcr03+focylRCb5bVT2HgiBC bBMQ25whkQagtZUFgEG9aDEdKf3fO9QuYQ== X-Google-Smtp-Source: ABdhPJx3Nj9xSKv8rIL03TgqefzU1dG511IE9cqTrtvbp1VoquggXO86sU+yIXOKn4W6VGWa44x1Mw== X-Received: by 2002:a17:906:50b:b0:6d0:9ebc:b9df with SMTP id j11-20020a170906050b00b006d09ebcb9dfmr16328470eja.67.1645553554421; Tue, 22 Feb 2022 10:12:34 -0800 (PST) Received: from DESKTOP-U5LNN3J.localdomain (89-79-181-52.dynamic.chello.pl. [89.79.181.52]) by smtp.gmail.com with ESMTPSA id y21sm10610153eda.38.2022.02.22.10.12.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Feb 2022 10:12:33 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: shaibran@amazon.com, upstream@semihalf.com, Michal Krawczyk , Dawid Gorecki Subject: [PATCH v2 16/21] net/ena: make Tx completion timeout configurable Date: Tue, 22 Feb 2022 19:11:41 +0100 Message-Id: <20220222181146.28882-17-mk@semihalf.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220222181146.28882-1-mk@semihalf.com> References: <20220222160634.24489-1-mk@semihalf.com> <20220222181146.28882-1-mk@semihalf.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The default missing Tx completion timeout was set to 5 seconds. In order to provide users with the interface to control this timeout to adjust it with the application's watchdog, the device argument for controlling this value was added. The parameter is called 'miss_txc_to' and can be modified using the devargs interface: ./app -a ,miss_txc_to=UINT_NUMBER This parameter accepts values from 0 to 60 and indicates number of seconds after which the Tx packet will be considered as missing. HW hints for the Tx completions timeout were removed to do not overwrite parameter from the user. Also specyfing default Tx completion timeout value was moved from the configuration to init phase in order to simplify default value assignment. Signed-off-by: Michal Krawczyk Reviewed-by: Dawid Gorecki Reviewed-by: Shai Brandes --- doc/guides/nics/ena.rst | 9 ++++ doc/guides/rel_notes/release_22_03.rst | 1 + drivers/net/ena/ena_ethdev.c | 66 ++++++++++++++++++++------ drivers/net/ena/ena_ethdev.h | 1 + 4 files changed, 62 insertions(+), 15 deletions(-) diff --git a/doc/guides/nics/ena.rst b/doc/guides/nics/ena.rst index 85c790e80f..3d780acac9 100644 --- a/doc/guides/nics/ena.rst +++ b/doc/guides/nics/ena.rst @@ -87,6 +87,15 @@ Configuration information effect only if the device also supports large LLQ headers. Otherwise, the default value will be used. + * **miss_txc_to** (default 5) + + Number of seconds after which the Tx packet will be considered missing. + If the missing packets number will exceed dynamically calculated threshold, + the driver will trigger the device reset which should be handled by the + application. Checking for missing Tx completions happens in the driver's + timer service. Setting this parameter to 0 disables this feature. Maximum + allowed value is 60 seconds. + **ENA Configuration Parameters** * **Number of Queues** diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst index 1b87b9c174..d0d2824371 100644 --- a/doc/guides/rel_notes/release_22_03.rst +++ b/doc/guides/rel_notes/release_22_03.rst @@ -116,6 +116,7 @@ New Features * Added ENA admin queue support for the MP applications. * Added free Tx mbuf on demand feature support. * Added ``rte_eth_xstats_get_names_by_id`` API support. + * Added ``miss_txc_to`` device argument for setting the Tx completion timeout. * **Updated Cisco enic driver.** diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index d413111b08..4bb2b195b1 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -62,6 +62,10 @@ struct ena_stats { /* Device arguments */ #define ENA_DEVARG_LARGE_LLQ_HDR "large_llq_hdr" +/* Timeout in seconds after which a single uncompleted Tx packet should be + * considered as a missing. + */ +#define ENA_DEVARG_MISS_TXC_TO "miss_txc_to" /* * Each rte_memzone should have unique name. @@ -2132,6 +2136,8 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) snprintf(adapter->name, ENA_NAME_MAX_LEN, "ena_%d", adapter->id_number); + adapter->missing_tx_completion_to = ENA_TX_TIMEOUT; + rc = ena_parse_devargs(adapter, pci_dev->device.devargs); if (rc != 0) { PMD_INIT_LOG(CRIT, "Failed to parse devargs\n"); @@ -2307,7 +2313,6 @@ static int ena_dev_configure(struct rte_eth_dev *dev) adapter->missing_tx_completion_budget = RTE_MIN(ENA_MONITORED_TX_QUEUES, dev->data->nb_tx_queues); - adapter->missing_tx_completion_to = ENA_TX_TIMEOUT; /* To avoid detection of the spurious Tx completion timeout due to * application not calling the Tx cleanup function, set timeout for the * Tx queue which should be half of the missing completion timeout for a @@ -2830,20 +2835,6 @@ static void ena_update_hints(struct ena_adapter *adapter, adapter->ena_dev.mmio_read.reg_read_to = hints->mmio_read_timeout * 1000; - if (hints->missing_tx_completion_timeout) { - if (hints->missing_tx_completion_timeout == - ENA_HW_HINTS_NO_TIMEOUT) { - adapter->missing_tx_completion_to = - ENA_HW_HINTS_NO_TIMEOUT; - } else { - /* Convert from msecs to ticks */ - adapter->missing_tx_completion_to = rte_get_timer_hz() * - hints->missing_tx_completion_timeout / 1000; - adapter->tx_cleanup_stall_delay = - adapter->missing_tx_completion_to / 2; - } - } - if (hints->driver_watchdog_timeout) { if (hints->driver_watchdog_timeout == ENA_HW_HINTS_NO_TIMEOUT) adapter->keep_alive_timeout = ENA_HW_HINTS_NO_TIMEOUT; @@ -3396,6 +3387,45 @@ static int ena_xstats_get_by_id(struct rte_eth_dev *dev, return valid; } +static int ena_process_uint_devarg(const char *key, + const char *value, + void *opaque) +{ + struct ena_adapter *adapter = opaque; + char *str_end; + uint64_t uint_value; + + uint_value = strtoull(value, &str_end, 10); + if (value == str_end) { + PMD_INIT_LOG(ERR, + "Invalid value for key '%s'. Only uint values are accepted.\n", + key); + return -EINVAL; + } + + if (strcmp(key, ENA_DEVARG_MISS_TXC_TO) == 0) { + if (uint_value > ENA_MAX_TX_TIMEOUT_SECONDS) { + PMD_INIT_LOG(ERR, + "Tx timeout too high: %" PRIu64 " sec. Maximum allowed: %d sec.\n", + uint_value, ENA_MAX_TX_TIMEOUT_SECONDS); + return -EINVAL; + } else if (uint_value == 0) { + PMD_INIT_LOG(INFO, + "Check for missing Tx completions has been disabled.\n"); + adapter->missing_tx_completion_to = + ENA_HW_HINTS_NO_TIMEOUT; + } else { + PMD_INIT_LOG(INFO, + "Tx packet completion timeout set to %" PRIu64 " seconds.\n", + uint_value); + adapter->missing_tx_completion_to = + uint_value * rte_get_timer_hz(); + } + } + + return 0; +} + static int ena_process_bool_devarg(const char *key, const char *value, void *opaque) @@ -3427,6 +3457,7 @@ static int ena_parse_devargs(struct ena_adapter *adapter, { static const char * const allowed_args[] = { ENA_DEVARG_LARGE_LLQ_HDR, + ENA_DEVARG_MISS_TXC_TO, NULL, }; struct rte_kvargs *kvlist; @@ -3444,7 +3475,12 @@ static int ena_parse_devargs(struct ena_adapter *adapter, rc = rte_kvargs_process(kvlist, ENA_DEVARG_LARGE_LLQ_HDR, ena_process_bool_devarg, adapter); + if (rc != 0) + goto exit; + rc = rte_kvargs_process(kvlist, ENA_DEVARG_MISS_TXC_TO, + ena_process_uint_devarg, adapter); +exit: rte_kvargs_free(kvlist); return rc; diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index ca3e5ed691..c0094b03ee 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -40,6 +40,7 @@ #define ENA_DEVICE_KALIVE_TIMEOUT (ENA_WD_TIMEOUT_SEC * rte_get_timer_hz()) #define ENA_TX_TIMEOUT (5 * rte_get_timer_hz()) +#define ENA_MAX_TX_TIMEOUT_SECONDS 60 #define ENA_MONITORED_TX_QUEUES 3 #define ENA_DEFAULT_MISSING_COMP 256U