Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/3583/?format=api
{ "id": 3583, "url": "https://patches.dpdk.org/api/patches/3583/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/1424492227-27229-5-git-send-email-adrien.mazarguil@6wind.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<1424492227-27229-5-git-send-email-adrien.mazarguil@6wind.com>", "list_archive_url": "https://inbox.dpdk.org/dev/1424492227-27229-5-git-send-email-adrien.mazarguil@6wind.com", "date": "2015-02-21T04:17:07", "name": "[dpdk-dev,v2,4/4] doc: add librte_pmd_mlx4 documentation", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "98200a079c7ac128d0854ddb845647e5a149b3cb", "submitter": { "id": 165, "url": "https://patches.dpdk.org/api/people/165/?format=api", "name": "Adrien Mazarguil", "email": "adrien.mazarguil@6wind.com" }, "delegate": null, "mbox": "https://patches.dpdk.org/project/dpdk/patch/1424492227-27229-5-git-send-email-adrien.mazarguil@6wind.com/mbox/", "series": [], "comments": "https://patches.dpdk.org/api/patches/3583/comments/", "check": "pending", "checks": "https://patches.dpdk.org/api/patches/3583/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 4ED58B82D;\n\tSat, 21 Feb 2015 05:17:40 +0100 (CET)", "from mail-wg0-f43.google.com (mail-wg0-f43.google.com\n\t[74.125.82.43]) by dpdk.org (Postfix) with ESMTP id 53B865953\n\tfor <dev@dpdk.org>; Sat, 21 Feb 2015 05:17:38 +0100 (CET)", "by mail-wg0-f43.google.com with SMTP id z12so16561246wgg.2\n\tfor <dev@dpdk.org>; Fri, 20 Feb 2015 20:17:37 -0800 (PST)", "from 6wind.com (6wind.net2.nerim.net. [213.41.180.237])\n\tby mx.google.com with ESMTPSA id\n\tw16sm5238462wia.15.2015.02.20.20.17.35\n\t(version=TLSv1.2 cipher=RC4-SHA bits=128/128);\n\tFri, 20 Feb 2015 20:17:36 -0800 (PST)" ], "X-Google-DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20130820;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references;\n\tbh=e6QcLEsEn83etC95KIf9nmAqkoOLehdjt8mjca817E8=;\n\tb=Du+Q+HMvzqVVZ7yqVFO4jQQzwlYsXNeV3rzXIedQc/41bg88QvvkP6lKZsiqUXWgyP\n\to2BGwUEPFcrRhoOePNcgHl9KEqC6ZNJuMClc5aTU1JdGZpm1LOWkLQQZN+lCgl3aTH6a\n\trFsWAyGDz3R37KbQyNX0lzdlK8mcqBQAzm9Azab2XJB7LZOc4lOqTwqsTEQEiMbpKRHQ\n\tTqk58KwCDtD/bkhN17wQb2vK4YI0GtTJD9GpiX7EWlwBAP3zt1q6TjiLLqasT03DevQe\n\tDSmpFOwiWh9A0f0WrwV/sKcJzGO/DqTLSSDUFf38eZvXe/yzGkBTUqTAqy1DgB660nNQ\n\tg1WQ==", "X-Gm-Message-State": "ALoCoQncwPs9kkP55nUGNmDKlPC6oYzeLD0ilfrjtFfZ0OiPpdXs4bGjNneCih6K0fO7V1VQfrHB", "X-Received": "by 10.180.72.98 with SMTP id c2mr846136wiv.87.1424492257363;\n\tFri, 20 Feb 2015 20:17:37 -0800 (PST)", "From": "Adrien Mazarguil <adrien.mazarguil@6wind.com>", "To": "dev@dpdk.org", "Date": "Sat, 21 Feb 2015 05:17:07 +0100", "Message-Id": "<1424492227-27229-5-git-send-email-adrien.mazarguil@6wind.com>", "X-Mailer": "git-send-email 2.1.0", "In-Reply-To": "<1422544846-10697-1-git-send-email-adrien.mazarguil@6wind.com>", "References": "<1422544846-10697-1-git-send-email-adrien.mazarguil@6wind.com>", "Subject": "[dpdk-dev] [PATCH v2 4/4] doc: add librte_pmd_mlx4 documentation", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "patches and discussions about DPDK <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "This documentation covers implementation details, features and limitations,\nconfiguration, prerequisites and provides a usage example.\n\nSigned-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>\n---\n doc/guides/prog_guide/index.rst | 1 +\n doc/guides/prog_guide/mlx4_poll_mode_drv.rst | 327 +++++++++++++++++++++++++++\n doc/guides/prog_guide/source_org.rst | 1 +\n 3 files changed, 329 insertions(+)\n create mode 100644 doc/guides/prog_guide/mlx4_poll_mode_drv.rst", "diff": "diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst\nindex de69682..87f6b35 100644\n--- a/doc/guides/prog_guide/index.rst\n+++ b/doc/guides/prog_guide/index.rst\n@@ -56,6 +56,7 @@ Programmer's Guide\n intel_dpdk_xen_based_packet_switch_sol\n libpcap_ring_based_poll_mode_drv\n link_bonding_poll_mode_drv_lib\n+ mlx4_poll_mode_drv\n timer_lib\n hash_lib\n lpm_lib\ndiff --git a/doc/guides/prog_guide/mlx4_poll_mode_drv.rst b/doc/guides/prog_guide/mlx4_poll_mode_drv.rst\nnew file mode 100644\nindex 0000000..e2c6b92\n--- /dev/null\n+++ b/doc/guides/prog_guide/mlx4_poll_mode_drv.rst\n@@ -0,0 +1,327 @@\n+.. BSD LICENSE\n+ Copyright(c) 2012-2015 6WIND S.A.\n+ All rights reserved.\n+\n+ Redistribution and use in source and binary forms, with or without\n+ modification, are permitted provided that the following conditions\n+ are met:\n+\n+ * Redistributions of source code must retain the above copyright\n+ notice, this list of conditions and the following disclaimer.\n+ * Redistributions in binary form must reproduce the above copyright\n+ notice, this list of conditions and the following disclaimer in\n+ the documentation and/or other materials provided with the\n+ distribution.\n+ * Neither the name of 6WIND S.A. nor the names of its\n+ contributors may be used to endorse or promote products derived\n+ from this software without specific prior written permission.\n+\n+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+\n+MLX4 poll mode driver library\n+=============================\n+\n+The MLX4 poll mode driver library (**librte_pmd_mlx4**) implements support\n+for **Mellanox ConnectX-3** 10/40 Gbps adapters (EN 40, EN 10, Pro EN 40) as\n+well as their virtual functions (VF) in SR-IOV context.\n+\n+.. note::\n+\n+ Due to external dependencies, this driver is disabled by default. It must\n+ be enabled manually by setting ``CONFIG_RTE_LIBRTE_MLX4_PMD=y`` and\n+ recompiling DPDK.\n+\n+Implementation details\n+----------------------\n+\n+Most Mellanox ConnectX-3 devices provide two ports but expose a single PCI\n+bus address, thus unlike most drivers, librte_pmd_mlx4 registers itself as a\n+PCI driver that allocates one Ethernet device per detected port.\n+\n+For this reason, one cannot white/blacklist a single port without also\n+white/blacklisting the others on the same device.\n+\n+Besides its dependency on libibverbs (that implies libmlx4 and associated\n+kernel support), librte_pmd_mlx4 relies heavily on system calls for control\n+operations such as querying/updating the MTU and flow control parameters.\n+\n+For security reasons and robustness, this driver only deals with virtual\n+memory addresses. The way resources allocations are handled by the kernel\n+combined with hardware specifications that allow it to handle virtual memory\n+addresses directly ensure that DPDK applications cannot access random\n+physical memory (or memory that does not belong to the current process).\n+\n+This capability allows the PMD to coexist with kernel network interfaces\n+which remain functional, although they stop receiving unicast packets as\n+long as they share the same MAC address.\n+\n+Compiling librte_pmd_mlx4 causes DPDK to be linked against libibverbs.\n+\n+Features and limitations\n+------------------------\n+\n+- RSS, also known as RCA, is supported. In this mode the number of\n+ configured RX queues must be a power of two.\n+- VLAN filtering is supported.\n+- Link state information is provided.\n+- Promiscuous mode is supported.\n+- All multicast mode is supported.\n+- Multiple MAC addresses (unicast, multicast) can be configured.\n+- Scattered packets are supported for TX and RX.\n+\n+..\n+\n+- RSS hash key cannot be modified.\n+- Hardware counters are not implemented (they are software counters).\n+- Checksum offloads are not supported yet.\n+\n+Configuration\n+-------------\n+\n+Compilation options\n+~~~~~~~~~~~~~~~~~~~\n+\n+- ``CONFIG_RTE_LIBRTE_MLX4_PMD`` (default **n**)\n+\n+ Toggle compilation of librte_pmd_mlx4 itself.\n+\n+- ``CONFIG_RTE_LIBRTE_MLX4_DEBUG`` (default **n**)\n+\n+ Toggle debugging code and stricter compilation flags. Enabling this option\n+ adds additional run-time checks and debugging messages at the cost of\n+ lower performance.\n+\n+- ``CONFIG_RTE_LIBRTE_MLX4_SGE_WR_N`` (default **4**)\n+\n+ Number of scatter/gather elements (SGEs) per work request (WR). Lowering\n+ this number improves performance but also limits the ability to receive\n+ scattered packets (packets that do not fit a single mbuf). The default\n+ value is a safe tradeoff.\n+\n+- ``CONFIG_RTE_LIBRTE_MLX4_MAX_INLINE`` (default **0**)\n+\n+ Amount of data to be inlined during TX operations. Improves latency but\n+ lowers throughput.\n+\n+- ``CONFIG_RTE_LIBRTE_MLX4_TX_MP_CACHE`` (default **8**)\n+\n+ Maximum number of cached memory pools (MPs) per TX queue. Each MP from\n+ which buffers are to be transmitted must be associated to memory regions\n+ (MRs). This is a slow operation that must be cached.\n+\n+ This value is always 1 for RX queues since they use a single MP.\n+\n+- ``CONFIG_RTE_LIBRTE_MLX4_SOFT_COUNTERS`` (default **1**)\n+\n+ Toggle software counters. No counters are available if this option is\n+ disabled since hardware counters are not supported.\n+\n+- ``CONFIG_RTE_LIBRTE_MLX4_COMPAT_VMWARE`` (default **1**)\n+\n+ Toggle VMware compatibility code. It also requires the environment\n+ variable ``MLX4_COMPAT_VMWARE`` set to a nonzero value at runtime.\n+\n+Environment variables\n+~~~~~~~~~~~~~~~~~~~~~\n+\n+- ``MLX4_INLINE_RECV_SIZE``\n+\n+ A nonzero value enables inline receive for packets up to that size. May\n+ significantly improve performance in some cases but lower it in\n+ others. Requires careful testing.\n+\n+- ``MLX4_COMPAT_VMWARE``\n+\n+ Only supported when compiled with\n+ ``CONFIG_RTE_LIBRTE_MLX4_COMPAT_VMWARE=1``. Adds workarounds to run in\n+ VMware systems that do not support the flows API properly.\n+\n+Run-time configuration\n+~~~~~~~~~~~~~~~~~~~~~~\n+\n+- The only constraint when RSS mode is requested is to make sure the number\n+ of RX queues is a power of two. This is a hardware requirement.\n+\n+- librte_pmd_mlx4 brings kernel network interfaces up during initialization\n+ because it is affected by their state. Forcing them down prevents packets\n+ reception.\n+\n+- **ethtool** operations on related kernel interfaces also affect the PMD.\n+\n+Prerequisites\n+-------------\n+\n+This driver relies on external libraries and kernel drivers for resources\n+allocations and initialization. The following dependencies are not part of\n+DPDK and must be installed separately:\n+\n+- **libibverbs**\n+\n+ User space verbs framework used by librte_pmd_mlx4. This library provides\n+ a generic interface between the kernel and low-level user space drivers\n+ such as libmlx4.\n+\n+ It allows slow and privileged operations (context initialization, hardware\n+ resources allocations) to be managed by the kernel and fast operations to\n+ never leave user space.\n+\n+- **libmlx4**\n+\n+ Low-level user space driver library for Mellanox ConnectX-3 devices,\n+ it is automatically loaded by libibverbs.\n+\n+ This library basically implements send/receive calls to the hardware\n+ queues.\n+\n+- **Kernel modules** (mlnx-ofed-kernel)\n+\n+ They provide the kernel-side verbs API and low level device drivers that\n+ manage actual hardware initialization and resources sharing with user\n+ space processes.\n+\n+ Unlike most other PMDs, these modules must remain loaded and bound to\n+ their devices:\n+\n+ - mlx4_core: hardware driver managing Mellanox ConnectX-3 devices.\n+ - mlx4_en: Ethernet device driver that provides kernel network interfaces.\n+ - mlx4_ib: InifiniBand device driver.\n+ - ib_uverbs: user space driver for verbs (entry point for libibverbs).\n+\n+While these libraries and kernel modules are available on OpenFabrics\n+Aliance's `website <https://www.openfabrics.org/>`_ and provided by package\n+managers on most distributions, this PMD requires Ethernet extensions that\n+may not be supported at the moment (this is a work in progress).\n+\n+`Mellanox OFED\n+<http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers>`_\n+includes the necessary support and should be used in the meantime. For DPDK,\n+only libibverbs, libmlx4 and mlnx-ofed-kernel packages are required from\n+that distribution.\n+\n+.. note::\n+\n+ Both libraries are BSD and GPL licensed. Linux kernel modules are GPL\n+ licensed.\n+\n+Usage example\n+-------------\n+\n+This section demonstrates how to launch **testpmd** with Mellanox ConnectX-3\n+devices managed by librte_pmd_mlx4.\n+\n+#. Load the kernel modules:\n+\n+ .. code-block:: console\n+\n+ modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib\n+\n+ .. note::\n+\n+ User space I/O kernel modules (uio and igb_uio) are not used and do\n+ not have to be loaded.\n+\n+#. Make sure Ethernet interfaces are in working order and linked to kernel\n+ verbs. Related sysfs entries should be present:\n+\n+ .. code-block:: console\n+\n+ ls -d /sys/class/net/*/device/infiniband_verbs/uverbs* | cut -d / -f 5\n+\n+ Example output:\n+\n+ .. code-block:: console\n+\n+ eth2\n+ eth3\n+ eth4\n+ eth5\n+\n+#. Optionally, retrieve their PCI bus addresses for whitelisting:\n+\n+ .. code-block:: console\n+\n+ {\n+ for intf in eth2 eth3 eth4 eth5;\n+ do\n+ (cd \"/sys/class/net/${intf}/device/\" && pwd -P);\n+ done;\n+ } |\n+ sed -n 's,.*/\\(.*\\),-w \\1,p'\n+\n+ Example output:\n+\n+ .. code-block:: console\n+\n+ -w 0000:83:00.0\n+ -w 0000:83:00.0\n+ -w 0000:84:00.0\n+ -w 0000:84:00.0\n+\n+ .. note::\n+\n+ There are only two distinct PCI bus addresses because the Mellanox\n+ ConnectX-3 adapters installed on this system are dual port.\n+\n+#. Request huge pages:\n+\n+ .. code-block:: console\n+\n+ echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/nr_hugepages\n+\n+#. Start testpmd with basic parameters:\n+\n+ .. code-block:: console\n+\n+ testpmd -c 0xff00 -n 4 -w 0000:83:00.0 -w 0000:84:00.0 -- --rxq=2 --txq=2 -i\n+\n+ Example output:\n+\n+ .. code-block:: console\n+\n+ [...]\n+ EAL: PCI device 0000:83:00.0 on NUMA socket 1\n+ EAL: probe driver: 15b3:1007 librte_pmd_mlx4\n+ PMD: librte_pmd_mlx4: PCI information matches, using device \"mlx4_0\" (VF: false)\n+ PMD: librte_pmd_mlx4: 2 port(s) detected\n+ PMD: librte_pmd_mlx4: port 1 MAC address is 00:02:c9:b5:b7:50\n+ PMD: librte_pmd_mlx4: port 2 MAC address is 00:02:c9:b5:b7:51\n+ EAL: PCI device 0000:84:00.0 on NUMA socket 1\n+ EAL: probe driver: 15b3:1007 librte_pmd_mlx4\n+ PMD: librte_pmd_mlx4: PCI information matches, using device \"mlx4_1\" (VF: false)\n+ PMD: librte_pmd_mlx4: 2 port(s) detected\n+ PMD: librte_pmd_mlx4: port 1 MAC address is 00:02:c9:b5:ba:b0\n+ PMD: librte_pmd_mlx4: port 2 MAC address is 00:02:c9:b5:ba:b1\n+ Interactive-mode selected\n+ Configuring Port 0 (socket 0)\n+ PMD: librte_pmd_mlx4: 0x867d60: TX queues number update: 0 -> 2\n+ PMD: librte_pmd_mlx4: 0x867d60: RX queues number update: 0 -> 2\n+ Port 0: 00:02:C9:B5:B7:50\n+ Configuring Port 1 (socket 0)\n+ PMD: librte_pmd_mlx4: 0x867da0: TX queues number update: 0 -> 2\n+ PMD: librte_pmd_mlx4: 0x867da0: RX queues number update: 0 -> 2\n+ Port 1: 00:02:C9:B5:B7:51\n+ Configuring Port 2 (socket 0)\n+ PMD: librte_pmd_mlx4: 0x867de0: TX queues number update: 0 -> 2\n+ PMD: librte_pmd_mlx4: 0x867de0: RX queues number update: 0 -> 2\n+ Port 2: 00:02:C9:B5:BA:B0\n+ Configuring Port 3 (socket 0)\n+ PMD: librte_pmd_mlx4: 0x867e20: TX queues number update: 0 -> 2\n+ PMD: librte_pmd_mlx4: 0x867e20: RX queues number update: 0 -> 2\n+ Port 3: 00:02:C9:B5:BA:B1\n+ Checking link statuses...\n+ Port 0 Link Up - speed 10000 Mbps - full-duplex\n+ Port 1 Link Up - speed 40000 Mbps - full-duplex\n+ Port 2 Link Up - speed 10000 Mbps - full-duplex\n+ Port 3 Link Up - speed 40000 Mbps - full-duplex\n+ Done\n+ testpmd>\ndiff --git a/doc/guides/prog_guide/source_org.rst b/doc/guides/prog_guide/source_org.rst\nindex c8ca54f..c66ad16 100644\n--- a/doc/guides/prog_guide/source_org.rst\n+++ b/doc/guides/prog_guide/source_org.rst\n@@ -83,6 +83,7 @@ The lib directory contains::\n +-- librte_pmd_e1000 # 1GbE poll mode drivers (igb and em)\n +-- librte_pmd_ixgbe # 10GbE poll mode driver\n +-- librte_pmd_i40e # 40GbE poll mode driver\n+ +-- librte_pmd_mlx4 # Mellanox ConnectX-3 poll mode driver\n +-- librte_pmd_pcap # PCAP poll mode driver\n +-- librte_pmd_ring # ring poll mode driver\n +-- librte_pmd_virtio # virtio poll mode driver\n", "prefixes": [ "dpdk-dev", "v2", "4/4" ] }