From patchwork Mon Mar 7 20:05:07 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Medala X-Patchwork-Id: 11152 X-Patchwork-Delegate: bruce.richardson@intel.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 3B1AE2E81; Mon, 7 Mar 2016 21:05:00 +0100 (CET) Received: from mail-lb0-f170.google.com (mail-lb0-f170.google.com [209.85.217.170]) by dpdk.org (Postfix) with ESMTP id 0BA2C2BAC for ; Mon, 7 Mar 2016 21:04:57 +0100 (CET) Received: by mail-lb0-f170.google.com with SMTP id bc4so143855789lbc.2 for ; Mon, 07 Mar 2016 12:04:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JGxdvGzlZelNEyfSiatWXulVQfbTBXz0f4OYVB5N9FE=; b=zuZ7TIFHL1uiCy5CtlMMKNa0/W4wmxEsBxw9x8INmX66J00G9vOu7499kJbB4lGL8i 3HLRz5IWJq0VJi9cfzvtoakIsdaxTPto1hDkELt1+UOffRqyXBmuO3oslNe+amOAsr2X Tfh6B/JgcvdnrCSf8iu7VfguraMkhbt4s/V/1v0taqVTT0LYtOtaPicdwbt6ZuQDbC/w mnDtR55MonfVfedj5fYR7D9/IEtZkh33U85nboauK0B96Np++64IASkh6oSgA3l5dDXG vKqRR65413SGdacXGnFL0qgaJdUxerqQfvBl5+/ZGQugc6TfLu+rgF5/PFbE2w2yVxZw ZcIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JGxdvGzlZelNEyfSiatWXulVQfbTBXz0f4OYVB5N9FE=; b=k0nopXgP7KZl1DQdSEZNrcmyqaTFP4rG8WHolwYu2/eOOS6NyRMfJ6DJ0Oq8fgCCNY W+0b11qfzKWiPwUSR33Vvd+I6eE+BD18pXLfqYaAEah08H1CdaWwaWD1ASTEDqidOzhk BKqTLmRyPL5GpIHpmx/Xbi0v4L/XP8vrLY1Akn/rQoBOorYinG23vd3DiQC+kHbkuXrB tB+8xksINvze6b5kDAmdUV0SoR/bjAa9aGKYwGmR2n+dQt5UB+OgeKkB3zBUeQnGrDGx HpA36z8aYt3tcD07YTWaL863xJ4CVRG5HMFYXzOzdFHyvQdXVxzCibpf60Zhogvm9LOi 0gmw== X-Gm-Message-State: AD7BkJLmMzyoO0534y98oetBRxorUYX3y01AKEZCk1a2/aszLIFy5A6HqBqJb2fTLhQOXA== X-Received: by 10.112.12.66 with SMTP id w2mr8537721lbb.97.1457381096775; Mon, 07 Mar 2016 12:04:56 -0800 (PST) Received: from anpa-dpdk-2.lab.semihalf.com ([80.82.22.190]) by smtp.gmail.com with ESMTPSA id h8sm3031137lbd.48.2016.03.07.12.04.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 07 Mar 2016 12:04:55 -0800 (PST) From: Jan Medala To: dev@dpdk.org Date: Mon, 7 Mar 2016 21:05:07 +0100 Message-Id: <1457381110-20382-2-git-send-email-jan@semihalf.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1457381110-20382-1-git-send-email-jan@semihalf.com> References: <1457381110-20382-1-git-send-email-jan@semihalf.com> Cc: Evgeny Schemeilin , matua@amazon.com Subject: [dpdk-dev] [PATCH v5 1/4] ena: Amazon ENA documentation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Evgeny Schemeilin Signed-off-by: Alexander Matushevsky Signed-off-by: Jan Medala Signed-off-by: Jakub Palider --- MAINTAINERS | 8 ++ doc/guides/nics/ena.rst | 252 ++++++++++++++++++++++++++++++++++++++++++++++ doc/guides/nics/index.rst | 1 + 3 files changed, 261 insertions(+) create mode 100644 doc/guides/nics/ena.rst diff --git a/MAINTAINERS b/MAINTAINERS index 628bc05..d6a726d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -261,6 +261,14 @@ Linux AF_PACKET M: John W. Linville F: drivers/net/af_packet/ +Amazon ena +M: Jan Medala +M: Jakub Palider +M: Netanel Belgazal +M: Evgeny Schemeilin +F: drivers/net/ena/ +F: doc/guides/nics/ena.rst + Chelsio cxgbe M: Rahul Lakkireddy F: drivers/net/cxgbe/ diff --git a/doc/guides/nics/ena.rst b/doc/guides/nics/ena.rst new file mode 100644 index 0000000..fede492 --- /dev/null +++ b/doc/guides/nics/ena.rst @@ -0,0 +1,252 @@ +.. BSD LICENSE + + Copyright (c) 2015-2016 Amazon.com, Inc. or its affiliates. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Amazon.com, Inc. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +ENA Poll Mode Driver +==================== + +The ENA PMD is a DPDK poll-mode driver for the Amazon Elastic +Network Adapter (ENA) family. + +Overview +-------- + +The ENA driver exposes a lightweight management interface with a +minimal set of memory mapped registers and an extendable command set +through an Admin Queue. + +The driver supports a wide range of ENA adapters, is link-speed +independent (i.e., the same driver is used for 10GbE, 25GbE, 40GbE, +etc.), and it negotiates and supports an extendable feature set. + +ENA adapters allow high speed and low overhead Ethernet traffic +processing by providing a dedicated Tx/Rx queue pair per CPU core. + +The ENA driver supports industry standard TCP/IP offload features such +as checksum offload and TCP transmit segmentation offload (TSO). + +Receive-side scaling (RSS) is supported for multi-core scaling. + +Some of the ENA devices support a working mode called Low-latency +Queue (LLQ), which saves several more microseconds. + +Management Interface +-------------------- + +ENA management interface is exposed by means of: + +* Device Registers +* Admin Queue (AQ) and Admin Completion Queue (ACQ) + +ENA device memory-mapped PCIe space for registers (MMIO registers) +are accessed only during driver initialization and are not involved +in further normal device operation. + +AQ is used for submitting management commands, and the +results/responses are reported asynchronously through ACQ. + +ENA introduces a very small set of management commands with room for +vendor-specific extensions. Most of the management operations are +framed in a generic Get/Set feature command. + +The following admin queue commands are supported: + +* Create I/O submission queue +* Create I/O completion queue +* Destroy I/O submission queue +* Destroy I/O completion queue +* Get feature +* Set feature +* Get statistics + +Refer to ``ena_admin_defs.h`` for the list of supported Get/Set Feature +properties. + +Data Path Interface +------------------- + +I/O operations are based on Tx and Rx Submission Queues (Tx SQ and Rx +SQ correspondingly). Each SQ has a completion queue (CQ) associated +with it. + +The SQs and CQs are implemented as descriptor rings in contiguous +physical memory. + +Refer to ``ena_eth_io_defs.h`` for the detailed structure of the descriptor + +The driver supports multi-queue for both Tx and Rx. + +Configuration information +------------------------- + +**DPDK Configuration Parameters** + + The following configuration options are available for the ENA PMD: + + * **CONFIG_RTE_LIBRTE_ENA_PMD** (default y): Enables or disables inclusion + of the ENA PMD driver in the DPDK compilation. + + + * **CONFIG_RTE_LIBRTE_ENA_DEBUG_INIT** (default y): Enables or disables debug + logging of device initialization within the ENA PMD driver. + + * **CONFIG_RTE_LIBRTE_ENA_DEBUG_RX** (default n): Enables or disables debug + logging of RX logic within the ENA PMD driver. + + * **CONFIG_RTE_LIBRTE_ENA_DEBUG_TX** (default n): Enables or disables debug + logging of TX logic within the ENA PMD driver. + + * **CONFIG_RTE_LIBRTE_ENA_COM_DEBUG** (default n): Enables or disables debug + logging of low level tx/rx logic in ena_com(base) within the ENA PMD driver. + +**ENA Configuration Parameters** + + * **Number of Queues** + + This is the requested number of queues upon initialization, however, the actual + number of receive and transmit queues to be created will be the minimum between + the maximal number supported by the device and number of queues requested. + + * **Size of Queues** + + This is the requested size of receive/transmit queues, while the actual size + will be the minimum between the requested size and the maximal receive/transmit + supported by the device. + +Building DPDK +------------- + +See the :ref:`DPDK Getting Started Guide for Linux ` for +instructions on how to build DPDK. + +By default the ENA PMD library will be built into the DPDK library. + +For configuring and using UIO and VFIO frameworks, please also refer :ref:`the +documentation that comes with DPDK suite `. + +Supported ENA adapters +---------------------- + +Current ENA PMD supports the following ENA adapters including: + +* ``1d0f:ec20`` - ENA VF +* ``1d0f:ec21`` - ENA VF with LLQ support + +Supported Operating Systems +--------------------------- + +Any Linux distribution fulfilling the conditions described in ``System Requirements`` +section of :ref:`the DPDK documentation ` or refer to *DPDK Release Notes*. + +Supported features +------------------ + +* Jumbo frames up to 9K +* Port Hardware Statistics +* IPv4/TCP/UDP checksum offload +* TSO offload +* Multiple receive and transmit queues +* RSS +* Low Latency Queue for Tx + +Unsupported features +-------------------- + +The features supported by the device and not yet supported by this PMD include: + +* Asynchronous Event Notification Queue (AENQ) + +Prerequisites +------------- + +#. Prepare the system as recommended by DPDK suite. This includes environment + variables, hugepages configuration, tool-chains and configuration + +#. Insert igb_uio kernel module using the command 'modprobe igb_uio' + +#. Bind the intended ENA device to igb_uio module + + +At this point the system should be ready to run DPDK applications. Once the +application runs to completion, the ENA can be detached from igb_uio if necessary. + +Usage example +------------- + +This section demonstrates how to launch **testpmd** with Amazon ENA +devices managed by librte_pmd_ena. + +#. Load the kernel modules: + + .. code-block:: console + + modprobe uio + insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko + + .. note:: + + Currently Amazon ENA PMD driver depends on igb_uio user space I/O kernel module + +#. Mount and request huge pages: + + .. code-block:: console + + mount -t hugetlbfs nodev /mnt/hugepages + echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + +#. Bind UIO driver to ENA device (using provided by DPDK binding tool): + + .. code-block:: console + + ./tools/dpdk_nic_bind.py --bind=igb_uio 0000:02:00.1 + +#. Start testpmd with basic parameters: + + .. code-block:: console + + ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -- -i + + Example output: + + .. code-block:: console + + [...] + EAL: PCI device 0000:02:00.1 on NUMA socket -1 + EAL: probe driver: 1d0f:ec20 rte_ena_pmd + EAL: PCI memory mapped at 0x7f9b6c400000 + PMD: eth_ena_dev_init(): Initializing 0:2:0.1 + Interactive-mode selected + Configuring Port 0 (socket 0) + Port 0: 00:00:00:11:00:01 + Checking link statuses... + Port 0 Link Up - speed 10000 Mbps - full-duplex + Done + testpmd> + diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 8618114..694b95b 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -39,6 +39,7 @@ Network Interface Controller Drivers bnx2x cxgbe e1000em + ena enic fm10k ixgbe