Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/96241/?format=api
{ "id": 96241, "url": "https://patches.dpdk.org/api/patches/96241/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/20210723102454.12206-5-mk@semihalf.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20210723102454.12206-5-mk@semihalf.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20210723102454.12206-5-mk@semihalf.com", "date": "2021-07-23T10:24:52", "name": "[v4,4/6] net/ena: add support for Rx interrupts", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": true, "hash": "6a56ab10097af42a781cbe0f0de1a74ee91e9903", "submitter": { "id": 786, "url": "https://patches.dpdk.org/api/people/786/?format=api", "name": "Michal Krawczyk", "email": "mk@semihalf.com" }, "delegate": null, "mbox": "https://patches.dpdk.org/project/dpdk/patch/20210723102454.12206-5-mk@semihalf.com/mbox/", "series": [ { "id": 17968, "url": "https://patches.dpdk.org/api/series/17968/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/list/?series=17968", "date": "2021-07-23T10:24:48", "name": "net/ena: v2.4.0 driver update", "version": 4, "mbox": "https://patches.dpdk.org/series/17968/mbox/" } ], "comments": "https://patches.dpdk.org/api/patches/96241/comments/", "check": "warning", "checks": "https://patches.dpdk.org/api/patches/96241/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 5AB21A0C46;\n\tFri, 23 Jul 2021 12:27:05 +0200 (CEST)", "from [217.70.189.124] (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 37798410E7;\n\tFri, 23 Jul 2021 12:26:46 +0200 (CEST)", "from mail-ej1-f45.google.com (mail-ej1-f45.google.com\n [209.85.218.45]) by mails.dpdk.org (Postfix) with ESMTP id 7B3F1410DA\n for <dev@dpdk.org>; Fri, 23 Jul 2021 12:26:42 +0200 (CEST)", "by mail-ej1-f45.google.com with SMTP id hq13so2924150ejc.7\n for <dev@dpdk.org>; Fri, 23 Jul 2021 03:26:42 -0700 (PDT)", "from DESKTOP-U5LNN3J.localdomain\n (093105178068.dynamic-ww-01.vectranet.pl. [93.105.178.68])\n by smtp.gmail.com with ESMTPSA id jp26sm10506142ejb.28.2021.07.23.03.26.40\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Fri, 23 Jul 2021 03:26:41 -0700 (PDT)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=semihalf-com.20150623.gappssmtp.com; s=20150623;\n h=from:to:cc:subject:date:message-id:in-reply-to:references\n :mime-version:content-transfer-encoding;\n bh=Q+p4Sq1xHLXc2eJkxjnjJY+1N/ycTjXn5qKi5LmCwzE=;\n b=mgN6mNCYR6rmiXhE2EkW8a5Y4dIu/ZNzDPMLbXzuok7OYDXJOf+gJ1IJ6b644ecnOq\n RMI5EZiVaKhChWzzeQMWtw89zhNa8muOuGYEivAumitWImahC6sGy/8rgVZZssuu9LKh\n Z41sz46nrfh0q+WqRebthZ5NbzGsgnViBbu623isBEn1RkZBVnb8pI+mLPPxVeI/KKWn\n Cz+CLIuISKebOUBVKf74GCr6LxqSqVk2Bm93HB2ncKLffTTxmfni9uztnyS9wmmy7g60\n wvLOFQtqg4ISISlP6Hed5N6qmWDYUxLqRRCNiQoLkOK18jnYkPs4aVM6ShmyAHKkDKwP\n L50w==", "X-Google-DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20161025;\n h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n :references:mime-version:content-transfer-encoding;\n bh=Q+p4Sq1xHLXc2eJkxjnjJY+1N/ycTjXn5qKi5LmCwzE=;\n b=SPxTiYPtdvy7h859cBmTVa2uuNtA/zTyTxXExlsKL1q2jvbzO7bsS9a/B5rlbvqXa5\n ZWTnyi0bhzUPtb3rdqGZEsWvQAfHZLUJLLZe7Hc0Y6KHDRknNYk8a1KbZUMz2VKPlafi\n 7wTrGRYDiwnL19i1a/4abi00pHrKU0ndzeGCj+8Hli6nTghEtNkn7iRkaj1psb0R9aYs\n y8QvpxnCOFHggjqSJ4Aw8N4US6WsE1rJ2Fkpd1igEUo4MnR60c2bUi28u/wYto1iXk5p\n 739qX5psN4rw0SvAYE5qm49n3i4j//T0NM+DbsphoV5gUc2+TPO5vWfS6a+TFR1BMnkm\n d80w==", "X-Gm-Message-State": "AOAM531z4fc+FP/lamLXIWO3GAHfDQt+fj2X7UZnKfKqQy16J4BPIprn\n /zKump+NG0B2hhf1YGstVNk3tnbCi0tHqEM7", "X-Google-Smtp-Source": "\n ABdhPJyRRYVAscH6P4Vugnu6mAn7r2O6Xy6Cium3nDDuH6xLGATgdqNNsLcCkLQ+3O2ZE+4US1gv/A==", "X-Received": "by 2002:a17:906:c087:: with SMTP id\n f7mr3863576ejz.487.1627036001875;\n Fri, 23 Jul 2021 03:26:41 -0700 (PDT)", "From": "Michal Krawczyk <mk@semihalf.com>", "To": "dev@dpdk.org", "Cc": "ndagan@amazon.com, shaibran@amazon.com, upstream@semihalf.com,\n Michal Krawczyk <mk@semihalf.com>, Artur Rojek <ar@semihalf.com>,\n Igor Chauskin <igorch@amazon.com>, Shay Agroskin <shayagr@amazon.com>", "Date": "Fri, 23 Jul 2021 12:24:52 +0200", "Message-Id": "<20210723102454.12206-5-mk@semihalf.com>", "X-Mailer": "git-send-email 2.25.1", "In-Reply-To": "<20210723102454.12206-1-mk@semihalf.com>", "References": "<20210714104320.4096-1-mk@semihalf.com>\n <20210723102454.12206-1-mk@semihalf.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "Subject": "[dpdk-dev] [PATCH v4 4/6] net/ena: add support for Rx interrupts", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "In order to support asynchronous Rx in the applications, the driver has\nto configure the event file descriptors and configure the HW.\n\nThis patch configures appropriate data structures for the rte_ethdev\nlayer, adds .rx_queue_intr_enable and .rx_queue_intr_disable API\nhandlers, and configures IO queues to work in the interrupt mode, if it\nwas requested by the application.\n\nSigned-off-by: Michal Krawczyk <mk@semihalf.com>\nReviewed-by: Artur Rojek <ar@semihalf.com>\nReviewed-by: Igor Chauskin <igorch@amazon.com>\nReviewed-by: Shai Brandes <shaibran@amazon.com>\nReviewed-by: Shay Agroskin <shayagr@amazon.com>\n---\n doc/guides/nics/ena.rst | 12 ++\n doc/guides/nics/features/ena.ini | 1 +\n doc/guides/rel_notes/release_21_08.rst | 7 ++\n drivers/net/ena/ena_ethdev.c | 146 +++++++++++++++++++++++--\n 4 files changed, 154 insertions(+), 12 deletions(-)", "diff": "diff --git a/doc/guides/nics/ena.rst b/doc/guides/nics/ena.rst\nindex 0f1f63f722..63951098ea 100644\n--- a/doc/guides/nics/ena.rst\n+++ b/doc/guides/nics/ena.rst\n@@ -141,6 +141,7 @@ Supported features\n * LSC event notification\n * Watchdog (requires handling of timers in the application)\n * Device reset upon failure\n+* Rx interrupts\n \n Prerequisites\n -------------\n@@ -180,6 +181,17 @@ At this point the system should be ready to run DPDK applications. Once the\n application runs to completion, the ENA can be detached from attached module if\n necessary.\n \n+**Rx interrupts support**\n+\n+ENA PMD supports Rx interrupts, which can be used to wake up lcores waiting for\n+input. Please note that it won't work with ``igb_uio``, so to use this feature,\n+the ``vfio-pci`` should be used.\n+\n+ENA handles admin interrupts and AENQ notifications on separate interrupt.\n+There is possibility that there won't be enough event file descriptors to\n+handle both admin and Rx interrupts. In that situation the Rx interrupt request\n+will fail.\n+\n **Note about usage on \\*.metal instances**\n \n On AWS, the metal instances are supporting IOMMU for both arm64 and x86_64\ndiff --git a/doc/guides/nics/features/ena.ini b/doc/guides/nics/features/ena.ini\nindex 2595ff53f9..3976bbbda6 100644\n--- a/doc/guides/nics/features/ena.ini\n+++ b/doc/guides/nics/features/ena.ini\n@@ -6,6 +6,7 @@\n [Features]\n Link status = Y\n Link status event = Y\n+Rx interrupt = Y\n MTU update = Y\n Jumbo frame = Y\n Scattered Rx = Y\ndiff --git a/doc/guides/rel_notes/release_21_08.rst b/doc/guides/rel_notes/release_21_08.rst\nindex 247e672f02..616e2cdea9 100644\n--- a/doc/guides/rel_notes/release_21_08.rst\n+++ b/doc/guides/rel_notes/release_21_08.rst\n@@ -95,6 +95,13 @@ New Features\n Added a new PMD driver for Wangxun 1 Gigabit Ethernet NICs.\n See the :doc:`../nics/ngbe` for more details.\n \n+* **Updated Amazon ENA PMD.**\n+\n+ The new driver version (v2.4.0) introduced bug fixes and improvements,\n+ including:\n+\n+ * Added Rx interrupt support.\n+\n * **Added support for Marvell CNXK crypto driver.**\n \n * Added cnxk crypto PMD which provides support for an integrated\ndiff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c\nindex 67cd91046a..72f9887797 100644\n--- a/drivers/net/ena/ena_ethdev.c\n+++ b/drivers/net/ena/ena_ethdev.c\n@@ -213,11 +213,11 @@ static void ena_rx_queue_release_bufs(struct ena_ring *ring);\n static void ena_tx_queue_release_bufs(struct ena_ring *ring);\n static int ena_link_update(struct rte_eth_dev *dev,\n \t\t\t int wait_to_complete);\n-static int ena_create_io_queue(struct ena_ring *ring);\n+static int ena_create_io_queue(struct rte_eth_dev *dev, struct ena_ring *ring);\n static void ena_queue_stop(struct ena_ring *ring);\n static void ena_queue_stop_all(struct rte_eth_dev *dev,\n \t\t\t enum ena_ring_type ring_type);\n-static int ena_queue_start(struct ena_ring *ring);\n+static int ena_queue_start(struct rte_eth_dev *dev, struct ena_ring *ring);\n static int ena_queue_start_all(struct rte_eth_dev *dev,\n \t\t\t enum ena_ring_type ring_type);\n static void ena_stats_restart(struct rte_eth_dev *dev);\n@@ -249,6 +249,11 @@ static int ena_process_bool_devarg(const char *key,\n static int ena_parse_devargs(struct ena_adapter *adapter,\n \t\t\t struct rte_devargs *devargs);\n static int ena_copy_eni_stats(struct ena_adapter *adapter);\n+static int ena_setup_rx_intr(struct rte_eth_dev *dev);\n+static int ena_rx_queue_intr_enable(struct rte_eth_dev *dev,\n+\t\t\t\t uint16_t queue_id);\n+static int ena_rx_queue_intr_disable(struct rte_eth_dev *dev,\n+\t\t\t\t uint16_t queue_id);\n \n static const struct eth_dev_ops ena_dev_ops = {\n \t.dev_configure = ena_dev_configure,\n@@ -269,6 +274,8 @@ static const struct eth_dev_ops ena_dev_ops = {\n \t.dev_reset = ena_dev_reset,\n \t.reta_update = ena_rss_reta_update,\n \t.reta_query = ena_rss_reta_query,\n+\t.rx_queue_intr_enable = ena_rx_queue_intr_enable,\n+\t.rx_queue_intr_disable = ena_rx_queue_intr_disable,\n };\n \n void ena_rss_key_fill(void *key, size_t size)\n@@ -829,7 +836,7 @@ static int ena_queue_start_all(struct rte_eth_dev *dev,\n \t\t\t\t\t\"Inconsistent state of Tx queues\\n\");\n \t\t\t}\n \n-\t\t\trc = ena_queue_start(&queues[i]);\n+\t\t\trc = ena_queue_start(dev, &queues[i]);\n \n \t\t\tif (rc) {\n \t\t\t\tPMD_INIT_LOG(ERR,\n@@ -1074,6 +1081,10 @@ static int ena_start(struct rte_eth_dev *dev)\n \tif (rc)\n \t\treturn rc;\n \n+\trc = ena_setup_rx_intr(dev);\n+\tif (rc)\n+\t\treturn rc;\n+\n \trc = ena_queue_start_all(dev, ENA_RING_TYPE_RX);\n \tif (rc)\n \t\treturn rc;\n@@ -1114,6 +1125,8 @@ static int ena_stop(struct rte_eth_dev *dev)\n {\n \tstruct ena_adapter *adapter = dev->data->dev_private;\n \tstruct ena_com_dev *ena_dev = &adapter->ena_dev;\n+\tstruct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);\n+\tstruct rte_intr_handle *intr_handle = &pci_dev->intr_handle;\n \tint rc;\n \n \t/* Cannot free memory in secondary process */\n@@ -1132,6 +1145,16 @@ static int ena_stop(struct rte_eth_dev *dev)\n \t\t\tPMD_DRV_LOG(ERR, \"Device reset failed, rc: %d\\n\", rc);\n \t}\n \n+\trte_intr_disable(intr_handle);\n+\n+\trte_intr_efd_disable(intr_handle);\n+\tif (intr_handle->intr_vec != NULL) {\n+\t\trte_free(intr_handle->intr_vec);\n+\t\tintr_handle->intr_vec = NULL;\n+\t}\n+\n+\trte_intr_enable(intr_handle);\n+\n \t++adapter->dev_stats.dev_stop;\n \tadapter->state = ENA_ADAPTER_STATE_STOPPED;\n \tdev->data->dev_started = 0;\n@@ -1139,10 +1162,12 @@ static int ena_stop(struct rte_eth_dev *dev)\n \treturn 0;\n }\n \n-static int ena_create_io_queue(struct ena_ring *ring)\n+static int ena_create_io_queue(struct rte_eth_dev *dev, struct ena_ring *ring)\n {\n-\tstruct ena_adapter *adapter;\n-\tstruct ena_com_dev *ena_dev;\n+\tstruct ena_adapter *adapter = ring->adapter;\n+\tstruct ena_com_dev *ena_dev = &adapter->ena_dev;\n+\tstruct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);\n+\tstruct rte_intr_handle *intr_handle = &pci_dev->intr_handle;\n \tstruct ena_com_create_io_ctx ctx =\n \t\t/* policy set to _HOST just to satisfy icc compiler */\n \t\t{ ENA_ADMIN_PLACEMENT_POLICY_HOST,\n@@ -1151,9 +1176,7 @@ static int ena_create_io_queue(struct ena_ring *ring)\n \tunsigned int i;\n \tint rc;\n \n-\tadapter = ring->adapter;\n-\tena_dev = &adapter->ena_dev;\n-\n+\tctx.msix_vector = -1;\n \tif (ring->type == ENA_RING_TYPE_TX) {\n \t\tena_qid = ENA_IO_TXQ_IDX(ring->id);\n \t\tctx.direction = ENA_COM_IO_QUEUE_DIRECTION_TX;\n@@ -1163,12 +1186,13 @@ static int ena_create_io_queue(struct ena_ring *ring)\n \t} else {\n \t\tena_qid = ENA_IO_RXQ_IDX(ring->id);\n \t\tctx.direction = ENA_COM_IO_QUEUE_DIRECTION_RX;\n+\t\tif (rte_intr_dp_is_en(intr_handle))\n+\t\t\tctx.msix_vector = intr_handle->intr_vec[ring->id];\n \t\tfor (i = 0; i < ring->ring_size; i++)\n \t\t\tring->empty_rx_reqs[i] = i;\n \t}\n \tctx.queue_size = ring->ring_size;\n \tctx.qid = ena_qid;\n-\tctx.msix_vector = -1; /* interrupts not used */\n \tctx.numa_node = ring->numa_socket_id;\n \n \trc = ena_com_create_io_queue(ena_dev, &ctx);\n@@ -1193,6 +1217,10 @@ static int ena_create_io_queue(struct ena_ring *ring)\n \tif (ring->type == ENA_RING_TYPE_TX)\n \t\tena_com_update_numa_node(ring->ena_com_io_cq, ctx.numa_node);\n \n+\t/* Start with Rx interrupts being masked. */\n+\tif (ring->type == ENA_RING_TYPE_RX && rte_intr_dp_is_en(intr_handle))\n+\t\tena_rx_queue_intr_disable(dev, ring->id);\n+\n \treturn 0;\n }\n \n@@ -1229,14 +1257,14 @@ static void ena_queue_stop_all(struct rte_eth_dev *dev,\n \t\t\tena_queue_stop(&queues[i]);\n }\n \n-static int ena_queue_start(struct ena_ring *ring)\n+static int ena_queue_start(struct rte_eth_dev *dev, struct ena_ring *ring)\n {\n \tint rc, bufs_num;\n \n \tena_assert_msg(ring->configured == 1,\n \t\t \"Trying to start unconfigured queue\\n\");\n \n-\trc = ena_create_io_queue(ring);\n+\trc = ena_create_io_queue(dev, ring);\n \tif (rc) {\n \t\tPMD_INIT_LOG(ERR, \"Failed to create IO queue\\n\");\n \t\treturn rc;\n@@ -2944,6 +2972,100 @@ static int ena_parse_devargs(struct ena_adapter *adapter,\n \treturn rc;\n }\n \n+static int ena_setup_rx_intr(struct rte_eth_dev *dev)\n+{\n+\tstruct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);\n+\tstruct rte_intr_handle *intr_handle = &pci_dev->intr_handle;\n+\tint rc;\n+\tuint16_t vectors_nb, i;\n+\tbool rx_intr_requested = dev->data->dev_conf.intr_conf.rxq;\n+\n+\tif (!rx_intr_requested)\n+\t\treturn 0;\n+\n+\tif (!rte_intr_cap_multiple(intr_handle)) {\n+\t\tPMD_DRV_LOG(ERR,\n+\t\t\t\"Rx interrupt requested, but it isn't supported by the PCI driver\\n\");\n+\t\treturn -ENOTSUP;\n+\t}\n+\n+\t/* Disable interrupt mapping before the configuration starts. */\n+\trte_intr_disable(intr_handle);\n+\n+\t/* Verify if there are enough vectors available. */\n+\tvectors_nb = dev->data->nb_rx_queues;\n+\tif (vectors_nb > RTE_MAX_RXTX_INTR_VEC_ID) {\n+\t\tPMD_DRV_LOG(ERR,\n+\t\t\t\"Too many Rx interrupts requested, maximum number: %d\\n\",\n+\t\t\tRTE_MAX_RXTX_INTR_VEC_ID);\n+\t\trc = -ENOTSUP;\n+\t\tgoto enable_intr;\n+\t}\n+\n+\tintr_handle->intr_vec =\trte_zmalloc(\"intr_vec\",\n+\t\tdev->data->nb_rx_queues * sizeof(*intr_handle->intr_vec), 0);\n+\tif (intr_handle->intr_vec == NULL) {\n+\t\tPMD_DRV_LOG(ERR,\n+\t\t\t\"Failed to allocate interrupt vector for %d queues\\n\",\n+\t\t\tdev->data->nb_rx_queues);\n+\t\trc = -ENOMEM;\n+\t\tgoto enable_intr;\n+\t}\n+\n+\trc = rte_intr_efd_enable(intr_handle, vectors_nb);\n+\tif (rc != 0)\n+\t\tgoto free_intr_vec;\n+\n+\tif (!rte_intr_allow_others(intr_handle)) {\n+\t\tPMD_DRV_LOG(ERR,\n+\t\t\t\"Not enough interrupts available to use both ENA Admin and Rx interrupts\\n\");\n+\t\tgoto disable_intr_efd;\n+\t}\n+\n+\tfor (i = 0; i < vectors_nb; ++i)\n+\t\tintr_handle->intr_vec[i] = RTE_INTR_VEC_RXTX_OFFSET + i;\n+\n+\trte_intr_enable(intr_handle);\n+\treturn 0;\n+\n+disable_intr_efd:\n+\trte_intr_efd_disable(intr_handle);\n+free_intr_vec:\n+\trte_free(intr_handle->intr_vec);\n+\tintr_handle->intr_vec = NULL;\n+enable_intr:\n+\trte_intr_enable(intr_handle);\n+\treturn rc;\n+}\n+\n+static void ena_rx_queue_intr_set(struct rte_eth_dev *dev,\n+\t\t\t\t uint16_t queue_id,\n+\t\t\t\t bool unmask)\n+{\n+\tstruct ena_adapter *adapter = dev->data->dev_private;\n+\tstruct ena_ring *rxq = &adapter->rx_ring[queue_id];\n+\tstruct ena_eth_io_intr_reg intr_reg;\n+\n+\tena_com_update_intr_reg(&intr_reg, 0, 0, unmask);\n+\tena_com_unmask_intr(rxq->ena_com_io_cq, &intr_reg);\n+}\n+\n+static int ena_rx_queue_intr_enable(struct rte_eth_dev *dev,\n+\t\t\t\t uint16_t queue_id)\n+{\n+\tena_rx_queue_intr_set(dev, queue_id, true);\n+\n+\treturn 0;\n+}\n+\n+static int ena_rx_queue_intr_disable(struct rte_eth_dev *dev,\n+\t\t\t\t uint16_t queue_id)\n+{\n+\tena_rx_queue_intr_set(dev, queue_id, false);\n+\n+\treturn 0;\n+}\n+\n /*********************************************************************\n * PMD configuration\n *********************************************************************/\n", "prefixes": [ "v4", "4/6" ] }