From patchwork Tue Jun 7 16:43:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 112486 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6EA1A054D; Tue, 7 Jun 2022 18:44:14 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5755C42685; Tue, 7 Jun 2022 18:44:12 +0200 (CEST) Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by mails.dpdk.org (Postfix) with ESMTP id 174E540156 for ; Tue, 7 Jun 2022 18:44:10 +0200 (CEST) Received: by mail-ej1-f53.google.com with SMTP id s12so29159544ejx.3 for ; Tue, 07 Jun 2022 09:44:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WBBbzGo9JucMC1Sfg60ZcdeQzuYwPxkhJ2lnaBOip68=; b=SyCSnVCeTY8LeiQrY4eE68+ysR7JTtAr4wF+cWE3uiq6N3l3LXCXTbRSxUZQrViss5 F7tk9v1jSXZ3RzjvFAk88IBR36KTCYaG1mTOsg1NGV5yamiJBgrjqycvFPy32JNKTnMM jqa8CWjWXKulXyzrIJts6vaNsd25r3LCV19+d1YbQkoxdC4R6cB36P/ARt5iXwjTZDGZ m6Qan167oQAp+TCZvIZFnM8uYqb8+30X0u1Z3uuWOZl5qNVyc0M8ffI4xI/EZNVEpIEx 2Qrpr+AEV34epx/QfqaNjG7KU6Vu80MlZr+RY6YATKqLmlJI7rDsZKtKAage2KdVOhRi 3bcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WBBbzGo9JucMC1Sfg60ZcdeQzuYwPxkhJ2lnaBOip68=; b=Y2crNuNwrJGDCb/1v7xgR8osD/0hYN5W4mSvvNIdo/qJxBN0KoNR5WZDheZHzAdh+3 HjUl98fotIQ8vbtV8O/XDqy1tlP136CT0yG98TSVVL57OzMM37TOuF1inXzmi/kilTu3 NI7RyoWW788ZdMUs72dk9XZtoTtrLyaSkcL8n/xuOh6WO6UobHEP3M6WqIY0117XlL+/ T8uPRLFbl8kU4nZr6AyTOGHafXy3lSgb/NJGABB5+KWZJK9Xz5HV5NQAVtRxIQeka5xi 1sgIxc0v+Ype4Ge5eS3Jx8glHT8bO8GeZ3ZUCVmtrDFg8I90+6Kf61x3B4U7BPXF0jNL QArA== X-Gm-Message-State: AOAM53147iytrcnZrDxlFL9gZCWqjIhu8YOyitSygH9b4Onlb0/mTAF2 DJ1FK+I3U2dgbIuKdpaeyu2R8Q== X-Google-Smtp-Source: ABdhPJzuV3tXFt/tDo85U7Kj98/2h5De5EvZfeRWwNtuCaYBjW3hnHmM70zsBvHC6U91H7eau6OKPA== X-Received: by 2002:a17:907:ea9:b0:710:9003:9b33 with SMTP id ho41-20020a1709070ea900b0071090039b33mr16812499ejc.175.1654620249777; Tue, 07 Jun 2022 09:44:09 -0700 (PDT) Received: from DESKTOP-U5LNN3J.localdomain ([83.142.187.86]) by smtp.gmail.com with ESMTPSA id m3-20020aa7c483000000b0042de29d8fc0sm10625901edq.94.2022.06.07.09.44.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jun 2022 09:44:09 -0700 (PDT) From: Michal Krawczyk To: ferruh.yigit@intel.com Cc: shaibran@amazon.com, upstream@semihalf.com, mw@semihalf.com, dev@dpdk.org, Dawid Gorecki , Michal Krawczyk , Amit Bernstein Subject: [PATCH 1/4] net/ena: add fast mbuf free support Date: Tue, 7 Jun 2022 18:43:38 +0200 Message-Id: <20220607164341.19088-2-mk@semihalf.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220607164341.19088-1-mk@semihalf.com> References: <20220607164341.19088-1-mk@semihalf.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Dawid Gorecki Add support for RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE offload. It can be enabled if all the mbufs for a given queue belong to the same mempool and their reference count is equal to 1. Signed-off-by: Dawid Gorecki Reviewed-by: Michal Krawczyk Reviewed-by: Shai Brandes Reviewed-by: Amit Bernstein --- doc/guides/nics/features/ena.ini | 1 + doc/guides/rel_notes/release_22_07.rst | 7 ++++ drivers/net/ena/ena_ethdev.c | 49 ++++++++++++++++++++++++-- 3 files changed, 55 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/features/ena.ini b/doc/guides/nics/features/ena.ini index 59c1ae85fa..1fe7a71e3d 100644 --- a/doc/guides/nics/features/ena.ini +++ b/doc/guides/nics/features/ena.ini @@ -7,6 +7,7 @@ Link status = Y Link status event = Y Rx interrupt = Y +Fast mbuf free = Y Free Tx mbuf on demand = Y MTU update = Y Scattered Rx = Y diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst index d46f773df0..73f566e5fc 100644 --- a/doc/guides/rel_notes/release_22_07.rst +++ b/doc/guides/rel_notes/release_22_07.rst @@ -87,6 +87,13 @@ New Features Added an API which can get the device type of vDPA device. + +* **Updated Amazon ena driver.** + + The new driver version (v2.7.0) includes: + + * Added fast mbuf free feature support. + * **Updated Intel iavf driver.** * Added Tx QoS queue rate limitation support. diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 68768cab70..68a4478410 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -36,6 +36,12 @@ #define ENA_MIN_RING_DESC 128 +/* + * We should try to keep ENA_CLEANUP_BUF_SIZE lower than + * RTE_MEMPOOL_CACHE_MAX_SIZE, so we can fit this in mempool local cache. + */ +#define ENA_CLEANUP_BUF_SIZE 256 + #define ENA_PTYPE_HAS_HASH (RTE_PTYPE_L4_TCP | RTE_PTYPE_L4_UDP) struct ena_stats { @@ -2402,6 +2408,8 @@ static uint64_t ena_get_tx_port_offloads(struct ena_adapter *adapter) port_offloads |= RTE_ETH_TX_OFFLOAD_MULTI_SEGS; + port_offloads |= RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE; + return port_offloads; } @@ -2414,9 +2422,12 @@ static uint64_t ena_get_rx_queue_offloads(struct ena_adapter *adapter) static uint64_t ena_get_tx_queue_offloads(struct ena_adapter *adapter) { + uint64_t queue_offloads = 0; RTE_SET_USED(adapter); - return 0; + queue_offloads |= RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE; + + return queue_offloads; } static int ena_infos_get(struct rte_eth_dev *dev, @@ -3001,13 +3012,38 @@ static int ena_xmit_mbuf(struct ena_ring *tx_ring, struct rte_mbuf *mbuf) return 0; } +static __rte_always_inline size_t +ena_tx_cleanup_mbuf_fast(struct rte_mbuf **mbufs_to_clean, + struct rte_mbuf *mbuf, + size_t mbuf_cnt, + size_t buf_size) +{ + struct rte_mbuf *m_next; + + while (mbuf != NULL) { + m_next = mbuf->next; + mbufs_to_clean[mbuf_cnt++] = mbuf; + if (mbuf_cnt == buf_size) { + rte_mempool_put_bulk(mbufs_to_clean[0]->pool, (void **)mbufs_to_clean, + (unsigned int)mbuf_cnt); + mbuf_cnt = 0; + } + mbuf = m_next; + } + + return mbuf_cnt; +} + static int ena_tx_cleanup(void *txp, uint32_t free_pkt_cnt) { + struct rte_mbuf *mbufs_to_clean[ENA_CLEANUP_BUF_SIZE]; struct ena_ring *tx_ring = (struct ena_ring *)txp; + size_t mbuf_cnt = 0; unsigned int total_tx_descs = 0; unsigned int total_tx_pkts = 0; uint16_t cleanup_budget; uint16_t next_to_clean = tx_ring->next_to_clean; + bool fast_free = tx_ring->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE; /* * If free_pkt_cnt is equal to 0, it means that the user requested @@ -3032,7 +3068,12 @@ static int ena_tx_cleanup(void *txp, uint32_t free_pkt_cnt) tx_info->timestamp = 0; mbuf = tx_info->mbuf; - rte_pktmbuf_free(mbuf); + if (fast_free) { + mbuf_cnt = ena_tx_cleanup_mbuf_fast(mbufs_to_clean, mbuf, mbuf_cnt, + ENA_CLEANUP_BUF_SIZE); + } else { + rte_pktmbuf_free(mbuf); + } tx_info->mbuf = NULL; tx_ring->empty_tx_reqs[next_to_clean] = req_id; @@ -3052,6 +3093,10 @@ static int ena_tx_cleanup(void *txp, uint32_t free_pkt_cnt) ena_com_update_dev_comp_head(tx_ring->ena_com_io_cq); } + if (mbuf_cnt != 0) + rte_mempool_put_bulk(mbufs_to_clean[0]->pool, + (void **)mbufs_to_clean, mbuf_cnt); + /* Notify completion handler that full cleanup was performed */ if (free_pkt_cnt == 0 || total_tx_pkts < cleanup_budget) tx_ring->last_cleanup_ticks = rte_get_timer_cycles(); From patchwork Tue Jun 7 16:43:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 112487 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 13610A054D; Tue, 7 Jun 2022 18:44:20 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4634D427F0; Tue, 7 Jun 2022 18:44:13 +0200 (CEST) Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) by mails.dpdk.org (Postfix) with ESMTP id 42B884161A for ; Tue, 7 Jun 2022 18:44:11 +0200 (CEST) Received: by mail-ej1-f41.google.com with SMTP id o7so3078819eja.1 for ; Tue, 07 Jun 2022 09:44:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DSTM9ywHOpziY6J33IN+nQH65Muxl0QwI8bwYQybWMg=; b=SUeNv9NHNPgx0DtC/BweOLn8NMcisSY/altGCLP5JHhS/dFWm3oEZsxhhWEBeVkybc ryH9Z7zaVP32jkjCHs+bePB6YBQQLt7Cyt751vjexkdJynsCUtbuPlc+YnhWj9XH+q0O 8i00Gj8BiQiej5j6ibzlpIu8sbC0MxIOyEFks+qctnErSvP44g6itiSkBMxXgi5ccZw4 qlss/Pnb3f3NYEngtKKcC0ICu38XzW9dif/w5bUQe3jaEErTf1vs74aJD/gWE8nrjltD mUCRDC//hFEVlRpg9sgEndwIF+JlyZGXVTBSU4tDuTb9tp811+QFRfXEwG/mqOsTw75z uQ3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DSTM9ywHOpziY6J33IN+nQH65Muxl0QwI8bwYQybWMg=; b=6E5Te6f9hyEgPCJENfqiAISFyva8H2H1chHRJcXSNh1SMpFSfrPMR05MhLqwnYgNDm Rac82+J6gFLyjyG+yA4DeF1idBv9EfQsQiA7fHZB2a+oZIkHbwjlM/KYKq67ALD2HlDr WV2H+YLgVJPzSiFvBtrB27lOCW3Z9S8HplvaM8gskrsqMRrrkIEl51WgH8dGS2pLtl3j 9C7r9d7+fmqT/bT5563l+L0Zr40jtReFWeSPsDTRGd6pEKl3UDkwtlKapsDrD5CmUKbM ecHYiZeIDsi2iySjQ9jSFAVaymor/EY9rfSzR+x0DPvpSWgScovAi6Jm5Gj6vLWOYGwk ejag== X-Gm-Message-State: AOAM530mhTSfh2ouigAHXmiWe9ucizE7zBE7um0T4/8TbB3kOObuxcDr Fk+m0/YgQAiIV+nlwYvOf7MEgg== X-Google-Smtp-Source: ABdhPJzsmK2A5pqdgRPmlgXPbZzztLBDtZNweSGEy2uBNf7wiKkYIY4njxcKdLUYhmO50NpT5FfqVQ== X-Received: by 2002:a17:907:9606:b0:70a:e140:6329 with SMTP id gb6-20020a170907960600b0070ae1406329mr25755246ejc.471.1654620250852; Tue, 07 Jun 2022 09:44:10 -0700 (PDT) Received: from DESKTOP-U5LNN3J.localdomain ([83.142.187.86]) by smtp.gmail.com with ESMTPSA id m3-20020aa7c483000000b0042de29d8fc0sm10625901edq.94.2022.06.07.09.44.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jun 2022 09:44:10 -0700 (PDT) From: Michal Krawczyk To: ferruh.yigit@intel.com Cc: shaibran@amazon.com, upstream@semihalf.com, mw@semihalf.com, dev@dpdk.org, Dawid Gorecki , Michal Krawczyk , Amit Bernstein Subject: [PATCH 2/4] net/ena: skip MTU verification Date: Tue, 7 Jun 2022 18:43:39 +0200 Message-Id: <20220607164341.19088-3-mk@semihalf.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220607164341.19088-1-mk@semihalf.com> References: <20220607164341.19088-1-mk@semihalf.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Dawid Gorecki Remove MTU verification from ena_mtu_set() and ena_start(). It is done by rte_ethdev already, so there is no reason to repeat it inside the ENA driver. Signed-off-by: Dawid Gorecki Reviewed-by: Michal Krawczyk Reviewed-by: Shai Brandes Reviewed-by: Amit Bernstein --- drivers/net/ena/ena_ethdev.c | 25 ------------------------- 1 file changed, 25 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 68a4478410..efd6dea4c2 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -987,20 +987,6 @@ static int ena_queue_start_all(struct rte_eth_dev *dev, return rc; } -static int ena_check_valid_conf(struct ena_adapter *adapter) -{ - uint32_t mtu = adapter->edev_data->mtu; - - if (mtu > adapter->max_mtu || mtu < ENA_MIN_MTU) { - PMD_INIT_LOG(ERR, - "Unsupported MTU of %d. Max MTU: %d, min MTU: %d\n", - mtu, adapter->max_mtu, ENA_MIN_MTU); - return ENA_COM_UNSUPPORTED; - } - - return 0; -} - static int ena_calc_io_queue_size(struct ena_calc_queue_size_ctx *ctx, bool use_large_llq_hdr) @@ -1165,13 +1151,6 @@ static int ena_mtu_set(struct rte_eth_dev *dev, uint16_t mtu) ena_dev = &adapter->ena_dev; ena_assert_msg(ena_dev != NULL, "Uninitialized device\n"); - if (mtu > adapter->max_mtu || mtu < ENA_MIN_MTU) { - PMD_DRV_LOG(ERR, - "Invalid MTU setting. New MTU: %d, max MTU: %d, min MTU: %d\n", - mtu, adapter->max_mtu, ENA_MIN_MTU); - return -EINVAL; - } - rc = ENA_PROXY(adapter, ena_com_set_dev_mtu, ena_dev, mtu); if (rc) PMD_DRV_LOG(ERR, "Could not set MTU: %d\n", mtu); @@ -1193,10 +1172,6 @@ static int ena_start(struct rte_eth_dev *dev) return -EPERM; } - rc = ena_check_valid_conf(adapter); - if (rc) - return rc; - rc = ena_setup_rx_intr(dev); if (rc) return rc; From patchwork Tue Jun 7 16:43:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 112488 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0085DA054D; Tue, 7 Jun 2022 18:44:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3194042802; Tue, 7 Jun 2022 18:44:14 +0200 (CEST) Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) by mails.dpdk.org (Postfix) with ESMTP id 84D15427EC for ; Tue, 7 Jun 2022 18:44:12 +0200 (CEST) Received: by mail-ej1-f45.google.com with SMTP id fu3so34882348ejc.7 for ; Tue, 07 Jun 2022 09:44:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aQBGzTN6eJsPIWzjCn+wWuTaRf3XZAi9z/FM4jSoiEU=; b=UU3Qu1kMISWf8lYxDPi+7fADpnAFa7y949OTHldl49sysJiR4buCQMmRH5OggCd0pB 4IpJlAeMIILYAOwfbKy1QGE+1UCtBjJotLtKSqj9NiSqDo4S0FSWvbgC5c8lTwH8zsa1 jWoDRodqSRYbQormY03z6VB57drFBp6hRcfuocE9A5j8kN2fqzGhSX6wVotR6fOMlJdB Wb1PoihghFHREF1cvUFiWT932qxMILgUgVLZBmeuEhH2Y246F061AixAn/keqyWRvLid nQT+3RADX2dqp+yNo5/YJy+KBsSFk8D+tTQHgJOSSsevGTUe/Eiq3MX8lA22WohEENr7 JtLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aQBGzTN6eJsPIWzjCn+wWuTaRf3XZAi9z/FM4jSoiEU=; b=yfLoOQVNBwy0Zx3+gIPpp+NOF6x2HhqAWtd0Wpte4zBC9njXcIqEeZFZkaWF37BeuN za0GDxbKo0UxSlhboZP1QXQUuM1a+5v8GBp2fWdc7VY8i0sf+70dzRJEklWtcXYzOQKf CV6laKXtexrDt7VmCyz+N70X8rBo7BsnLHwM8y5zfUokCKG/NWjDGEXIIbZCvqOQJCVN MwIbTEDNc+9cr5Z1RJY93yqmFPB9RI7F1TCsxAf2A/o+pnejbzPKkUfibvDwIGe0xlaB XMVFGb5dnKgK3H6sirrqVCE707zJYf+PE2pguoqSYyFmQgV6VXFeNT6WOi7UsCZdchPl +u5A== X-Gm-Message-State: AOAM530YctjLFMb7XryiweCmQH4iHxuGiOE4xjg0zqp32qnItN0P/15e MFMLlQJKUaCF+Wxda+z5vEKZIQ== X-Google-Smtp-Source: ABdhPJzlgZ/mRAS4qfLlv4njnvaUIs/8tyf/SfGZi/PTqZ4srP09uVBACLY/YYWHnRyPIPcHsRzUQw== X-Received: by 2002:a17:906:6a28:b0:711:d032:caa4 with SMTP id qw40-20020a1709066a2800b00711d032caa4mr10375661ejc.80.1654620252167; Tue, 07 Jun 2022 09:44:12 -0700 (PDT) Received: from DESKTOP-U5LNN3J.localdomain ([83.142.187.86]) by smtp.gmail.com with ESMTPSA id m3-20020aa7c483000000b0042de29d8fc0sm10625901edq.94.2022.06.07.09.44.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jun 2022 09:44:11 -0700 (PDT) From: Michal Krawczyk To: ferruh.yigit@intel.com Cc: shaibran@amazon.com, upstream@semihalf.com, mw@semihalf.com, dev@dpdk.org, Michal Krawczyk , Dawid Gorecki , Amit Bernstein Subject: [PATCH 3/4] net/ena: add an option to disable LLQ Date: Tue, 7 Jun 2022 18:43:40 +0200 Message-Id: <20220607164341.19088-4-mk@semihalf.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220607164341.19088-1-mk@semihalf.com> References: <20220607164341.19088-1-mk@semihalf.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The PMD attempts to enable the LLQ (Low Latency Queue) whenever it's possible. The LLQ requires the user to enable the Write Combining for the supported igb_uio/vfio-pci modules. The vfio-pci module officially doesn't support the WC. Moreover, in some Linux distributions, it can be built into the kernel, so any modifications to the vfio-pci module require a full rebuild of the kernel. This can make the configuration process much harder and for some users, that are not interested in the great network performance for their setups, it may be redundant. These users requested to be able to turn off LLQ to avoid the hassle of such a setup. It's generally not recommended to disable the LLQ, as it won't result in the performance improvement and on the 6th generation AWS instances the lack of LLQ can have a huge negative impact on hardware performance. The device argument which controls the LLQ is called 'enable_llq` and by default, it's set to 1 (which means that the LLQ is enabled). Setting it to 0 disables the LLQ. This commit also adds the explicit initialization of the devarg for the 'use_large_llq_hdr'. The PMD_REGISTER_PARAM_STRING() call for the ENA was updated with all the available devargs (including ENA_DEVARG_MISS_TXC_TO, which wasn't added previously). Signed-off-by: Michal Krawczyk Reviewed-by: Dawid Gorecki Reviewed-by: Shai Brandes Reviewed-by: Amit Bernstein --- doc/guides/nics/ena.rst | 9 ++++++++ doc/guides/rel_notes/release_22_07.rst | 2 ++ drivers/net/ena/ena_ethdev.c | 29 +++++++++++++++++++++++++- drivers/net/ena/ena_ethdev.h | 1 + 4 files changed, 40 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/ena.rst b/doc/guides/nics/ena.rst index 3d780acac9..df5343e4ae 100644 --- a/doc/guides/nics/ena.rst +++ b/doc/guides/nics/ena.rst @@ -96,6 +96,15 @@ Configuration information timer service. Setting this parameter to 0 disables this feature. Maximum allowed value is 60 seconds. + * **enable_llq** (default 1) + + Determines whenever the driver should use the LLQ (if it's available) or + not. + + **NOTE: On the 6th generation AWS instances disabling LLQ may lead to a + huge performance degradation. In general disabling LLQ is highly not + recommended!** + **ENA Configuration Parameters** * **Number of Queues** diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst index 73f566e5fc..45d0012f2d 100644 --- a/doc/guides/rel_notes/release_22_07.rst +++ b/doc/guides/rel_notes/release_22_07.rst @@ -93,6 +93,8 @@ New Features The new driver version (v2.7.0) includes: * Added fast mbuf free feature support. + * Added ``enable_llq`` device argument for controlling the PMD LLQ + (Low Latency Queue) mode. * **Updated Intel iavf driver.** diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index efd6dea4c2..d748b33e51 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -72,6 +72,12 @@ struct ena_stats { * considered as a missing. */ #define ENA_DEVARG_MISS_TXC_TO "miss_txc_to" +/* + * Controls whether LLQ should be used (if available). Enabled by default. + * NOTE: It's highly not recommended to disable the LLQ, as it may lead to a + * huge performance degradation on 6th generation AWS instances. + */ +#define ENA_DEVARG_ENABLE_LLQ "enable_llq" /* * Each rte_memzone should have unique name. @@ -1932,6 +1938,14 @@ ena_set_queues_placement_policy(struct ena_adapter *adapter, int rc; u32 llq_feature_mask; + if (!adapter->enable_llq) { + PMD_DRV_LOG(WARNING, + "NOTE: LLQ has been disabled as per user's request. " + "This may lead to a huge performance degradation!\n"); + ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + return 0; + } + llq_feature_mask = 1 << ENA_ADMIN_LLQ; if (!(ena_dev->supported_features & llq_feature_mask)) { PMD_DRV_LOG(INFO, @@ -2127,7 +2141,10 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) snprintf(adapter->name, ENA_NAME_MAX_LEN, "ena_%d", adapter->id_number); + /* Assign default devargs values */ adapter->missing_tx_completion_to = ENA_TX_TIMEOUT; + adapter->enable_llq = true; + adapter->use_large_llq_hdr = false; rc = ena_parse_devargs(adapter, pci_dev->device.devargs); if (rc != 0) { @@ -3478,6 +3495,8 @@ static int ena_process_bool_devarg(const char *key, /* Now, assign it to the proper adapter field. */ if (strcmp(key, ENA_DEVARG_LARGE_LLQ_HDR) == 0) adapter->use_large_llq_hdr = bool_value; + else if (strcmp(key, ENA_DEVARG_ENABLE_LLQ) == 0) + adapter->enable_llq = bool_value; return 0; } @@ -3488,6 +3507,7 @@ static int ena_parse_devargs(struct ena_adapter *adapter, static const char * const allowed_args[] = { ENA_DEVARG_LARGE_LLQ_HDR, ENA_DEVARG_MISS_TXC_TO, + ENA_DEVARG_ENABLE_LLQ, NULL, }; struct rte_kvargs *kvlist; @@ -3509,6 +3529,10 @@ static int ena_parse_devargs(struct ena_adapter *adapter, goto exit; rc = rte_kvargs_process(kvlist, ENA_DEVARG_MISS_TXC_TO, ena_process_uint_devarg, adapter); + if (rc != 0) + goto exit; + rc = rte_kvargs_process(kvlist, ENA_DEVARG_ENABLE_LLQ, + ena_process_bool_devarg, adapter); exit: rte_kvargs_free(kvlist); @@ -3727,7 +3751,10 @@ static struct rte_pci_driver rte_ena_pmd = { RTE_PMD_REGISTER_PCI(net_ena, rte_ena_pmd); RTE_PMD_REGISTER_PCI_TABLE(net_ena, pci_id_ena_map); RTE_PMD_REGISTER_KMOD_DEP(net_ena, "* igb_uio | uio_pci_generic | vfio-pci"); -RTE_PMD_REGISTER_PARAM_STRING(net_ena, ENA_DEVARG_LARGE_LLQ_HDR "=<0|1>"); +RTE_PMD_REGISTER_PARAM_STRING(net_ena, + ENA_DEVARG_LARGE_LLQ_HDR "=<0|1> " + ENA_DEVARG_ENABLE_LLQ "=<0|1> " + ENA_DEVARG_MISS_TXC_TO "="); RTE_LOG_REGISTER_SUFFIX(ena_logtype_init, init, NOTICE); RTE_LOG_REGISTER_SUFFIX(ena_logtype_driver, driver, NOTICE); #ifdef RTE_ETHDEV_DEBUG_RX diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index c0094b03ee..0db0918b43 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -303,6 +303,7 @@ struct ena_adapter { bool trigger_reset; + bool enable_llq; bool use_large_llq_hdr; uint32_t last_tx_comp_qid; From patchwork Tue Jun 7 16:43:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 112489 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D7580A054D; Tue, 7 Jun 2022 18:44:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 226194282C; Tue, 7 Jun 2022 18:44:16 +0200 (CEST) Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by mails.dpdk.org (Postfix) with ESMTP id 851054281A for ; Tue, 7 Jun 2022 18:44:14 +0200 (CEST) Received: by mail-ej1-f54.google.com with SMTP id kq6so23340051ejb.11 for ; Tue, 07 Jun 2022 09:44:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xypx0uZL2YbtY+svh7E4uvdn6y7MahPg/ihutsDBzP0=; b=JCsPs+mivlTvhgJl92VUwbkbJGrBNn2uud4F9C26/btou15sxdXbyYawFwxLvLsJ0t G+ceU8R1yP3yvCqVlIo1ctuXYOhDiUhzfd/G4iy4J9xuA9ELd6Y6mtQkj2qq+w2LcTnI qBWxfvQJ2/6j0tyweGRfXqhUdS5Qjiv+3KukJRAV3NaW6Kpg2nGmXSmFVUaVt40oTDLV XWUO5EIxjmfYFvLDZggt0qRokz95m8Lu3TFVoRfCEnkPkru+d1T8Smwn2r5jLHZE0EZC qbQz3Bw4IIwFH0sOnKgz7K5nSBE2V5sMvJYH6y1I+nWncQBFpyIN4KusFh4QQRFJb26t lc9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xypx0uZL2YbtY+svh7E4uvdn6y7MahPg/ihutsDBzP0=; b=AU5HxViMOBHXKxtwg8dEonZmfKQ4rNlwd4FLl+Ob5zUW1j4ew6gyF9PR5fANIMbLvo eWTyLNI8E1E4/Zv2iox6FhzDxxBenSJ+jlUDcjc7IYsCj5ez+tUkOesz1AsrH7uktPst ekilm+tYfl1AAHmFRhV/C+w3on28l5J8ncifbceDKw6bIX1T/gyBOShSLB6x+7lrQdTg zQW6hLsY11i6RsezyYZdY9qSRDWt9JeftsjjINoCZEkkR+sw2du8Ml7kCyS3YL74rUsM 7uHXY/JR9SRMaKmDIGvMS6wJtAcUpXZsIQmdxWy3qMgpf5fYNfoyZ71KHClpMCEDESWq pAkA== X-Gm-Message-State: AOAM530XA1pPHS6WN+6o4SApoyEUEmZemO5SqHZOEqfgT5iBehmadqan 7ik0pUANPnHig6LsLPs6zfvDL1MOM0zBwrjR X-Google-Smtp-Source: ABdhPJyUVtajY10M3R+48PN/arubn1rv45k3eCitkeinx80/kzso+nRnv2LuTXB51ZAKbpnG5QeZZA== X-Received: by 2002:a17:907:1b0c:b0:6fe:25bf:b3e5 with SMTP id mp12-20020a1709071b0c00b006fe25bfb3e5mr27972409ejc.689.1654620253268; Tue, 07 Jun 2022 09:44:13 -0700 (PDT) Received: from DESKTOP-U5LNN3J.localdomain ([83.142.187.86]) by smtp.gmail.com with ESMTPSA id m3-20020aa7c483000000b0042de29d8fc0sm10625901edq.94.2022.06.07.09.44.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jun 2022 09:44:12 -0700 (PDT) From: Michal Krawczyk To: ferruh.yigit@intel.com Cc: shaibran@amazon.com, upstream@semihalf.com, mw@semihalf.com, dev@dpdk.org, Michal Krawczyk Subject: [PATCH 4/4] net/ena: update version to 2.7.0 Date: Tue, 7 Jun 2022 18:43:41 +0200 Message-Id: <20220607164341.19088-5-mk@semihalf.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220607164341.19088-1-mk@semihalf.com> References: <20220607164341.19088-1-mk@semihalf.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This release contains changes listed below. - Fast mbuf free feature support. - Device argument to disable the LLQ. - Simplification of the MTU verification. Signed-off-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index d748b33e51..3e88bcda6c 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -21,7 +21,7 @@ #include #define DRV_MODULE_VER_MAJOR 2 -#define DRV_MODULE_VER_MINOR 6 +#define DRV_MODULE_VER_MINOR 7 #define DRV_MODULE_VER_SUBMINOR 0 #define __MERGE_64B_H_L(h, l) (((uint64_t)h << 32) | l)