Patch Detail

HTTP 200 OK
Allow: GET, PUT, PATCH, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "id": 110488,
    "url": "https://patches.dpdk.org/api/patches/110488/?format=api",
    "web_url": "https://patches.dpdk.org/project/dts/patch/20220429073246.19781-1-weix.ling@intel.com/",
    "project": {
        "id": 3,
        "url": "https://patches.dpdk.org/api/projects/3/?format=api",
        "name": "DTS",
        "link_name": "dts",
        "list_id": "dts.dpdk.org",
        "list_email": "dts@dpdk.org",
        "web_url": "",
        "scm_url": "git://dpdk.org/tools/dts",
        "webscm_url": "http://git.dpdk.org/tools/dts/",
        "list_archive_url": "https://inbox.dpdk.org/dts",
        "list_archive_url_format": "https://inbox.dpdk.org/dts/{}",
        "commit_url_format": ""
    },
    "msgid": "<20220429073246.19781-1-weix.ling@intel.com>",
    "list_archive_url": "https://inbox.dpdk.org/dts/20220429073246.19781-1-weix.ling@intel.com",
    "date": "2022-04-29T07:32:46",
    "name": "[V2,1/3] test_plans/vhost_cbdma_test_plan: modify testplan by DPDK command change",
    "commit_ref": null,
    "pull_url": null,
    "state": "superseded",
    "archived": false,
    "hash": "c7343c52dd0d9c3ee7e7ec567c2facd942f6c953",
    "submitter": {
        "id": 1828,
        "url": "https://patches.dpdk.org/api/people/1828/?format=api",
        "name": "Ling, WeiX",
        "email": "weix.ling@intel.com"
    },
    "delegate": null,
    "mbox": "https://patches.dpdk.org/project/dts/patch/20220429073246.19781-1-weix.ling@intel.com/mbox/",
    "series": [
        {
            "id": 22728,
            "url": "https://patches.dpdk.org/api/series/22728/?format=api",
            "web_url": "https://patches.dpdk.org/project/dts/list/?series=22728",
            "date": "2022-04-29T07:32:18",
            "name": "modify vhost_cbdma",
            "version": 2,
            "mbox": "https://patches.dpdk.org/series/22728/mbox/"
        }
    ],
    "comments": "https://patches.dpdk.org/api/patches/110488/comments/",
    "check": "pending",
    "checks": "https://patches.dpdk.org/api/patches/110488/checks/",
    "tags": {},
    "related": [],
    "headers": {
        "Return-Path": "<dts-bounces@dpdk.org>",
        "X-Original-To": "patchwork@inbox.dpdk.org",
        "Delivered-To": "patchwork@inbox.dpdk.org",
        "Received": [
            "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 8D672A034C;\n\tFri, 29 Apr 2022 09:33:06 +0200 (CEST)",
            "from [217.70.189.124] (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 87B0D415D7;\n\tFri, 29 Apr 2022 09:33:06 +0200 (CEST)",
            "from mga02.intel.com (mga02.intel.com [134.134.136.20])\n by mails.dpdk.org (Postfix) with ESMTP id 0343D40E50\n for <dts@dpdk.org>; Fri, 29 Apr 2022 09:33:03 +0200 (CEST)",
            "from fmsmga003.fm.intel.com ([10.253.24.29])\n by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 29 Apr 2022 00:33:02 -0700",
            "from unknown (HELO localhost.localdomain) ([10.239.251.222])\n by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 29 Apr 2022 00:33:01 -0700"
        ],
        "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple;\n d=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n t=1651217584; x=1682753584;\n h=from:to:cc:subject:date:message-id:mime-version:\n content-transfer-encoding;\n bh=DfQ274AMFkjOs2L0v+Pbf+rs3NHZTLdXzNrq1O12yCU=;\n b=Ej0VkgjRtlNkBbiVnJV4fbkS3yaYuSUsbVcG4VCQg1kQjHNLjBqo7AiT\n ow0+i4nEGrTnBp60+IUjGCz2aHWv4ELYIrXGAKvZt47nV7gmjJvIbwnpw\n NRjFNR9y5tEI628p0dxFu5T41s8907+pPrynen0FxXGb66shqJeZ5HG5Q\n sGpOjkA0KgDpzf6WtiBwCIJz2oGzOf5OoFvcIfK67qJS6set7ZsvxRvOg\n 58PUqygKdGA6UWJB++dNgVZcsoNJBtC4gM/pe+y9blm0R7Kp5B7F77UrL\n 42euW2r96l0KQ8rpOnc7ZmXEgklJdCMnq7+f2ecL80iP7ivm7mLt06b0O A==;",
        "X-IronPort-AV": [
            "E=McAfee;i=\"6400,9594,10331\"; a=\"253945739\"",
            "E=Sophos;i=\"5.91,297,1647327600\"; d=\"scan'208\";a=\"253945739\"",
            "E=Sophos;i=\"5.91,297,1647327600\"; d=\"scan'208\";a=\"651635452\""
        ],
        "From": "Wei Ling <weix.ling@intel.com>",
        "To": "dts@dpdk.org",
        "Cc": "Wei Ling <weix.ling@intel.com>",
        "Subject": "[dts][PATCH V2 1/3] test_plans/vhost_cbdma_test_plan: modify testplan\n by DPDK command change",
        "Date": "Fri, 29 Apr 2022 07:32:46 +0000",
        "Message-Id": "<20220429073246.19781-1-weix.ling@intel.com>",
        "X-Mailer": "git-send-email 2.34.1",
        "MIME-Version": "1.0",
        "Content-Type": "text/plain; charset=y",
        "Content-Transfer-Encoding": "8bit",
        "X-BeenThere": "dts@dpdk.org",
        "X-Mailman-Version": "2.1.29",
        "Precedence": "list",
        "List-Id": "test suite reviews and discussions <dts.dpdk.org>",
        "List-Unsubscribe": "<https://mails.dpdk.org/options/dts>,\n <mailto:dts-request@dpdk.org?subject=unsubscribe>",
        "List-Archive": "<http://mails.dpdk.org/archives/dts/>",
        "List-Post": "<mailto:dts@dpdk.org>",
        "List-Help": "<mailto:dts-request@dpdk.org?subject=help>",
        "List-Subscribe": "<https://mails.dpdk.org/listinfo/dts>,\n <mailto:dts-request@dpdk.org?subject=subscribe>",
        "Errors-To": "dts-bounces@dpdk.org"
    },
    "content": "v1:\nAs commit 53d3f4778c(vhost: integrate dmadev in asynchronous data-path),\nmodify vhost_cbdma testplan by DPDK22.03 Lib change.\n\nv2:\nModify the `Description` content in test_plan.\n\nSigned-off-by: Wei Ling <weix.ling@intel.com>\n---\n test_plans/vhost_cbdma_test_plan.rst | 1409 ++++++++++++++++++++------\n 1 file changed, 1113 insertions(+), 296 deletions(-)",
    "diff": "diff --git a/test_plans/vhost_cbdma_test_plan.rst b/test_plans/vhost_cbdma_test_plan.rst\nindex c8f8b8c5..6190819e 100644\n--- a/test_plans/vhost_cbdma_test_plan.rst\n+++ b/test_plans/vhost_cbdma_test_plan.rst\n@@ -1,404 +1,1221 @@\n-.. Copyright (c) <2021>, Intel Corporation\n-    All rights reserved.\n-\n-    Redistribution and use in source and binary forms, with or without\n-    modification, are permitted provided that the following conditions\n-    are met:\n-\n-    - Redistributions of source code must retain the above copyright\n- notice, this list of conditions and the following disclaimer.\n-\n-    - Redistributions in binary form must reproduce the above copyright\n- notice, this list of conditions and the following disclaimer in\n- the documentation and/or other materials provided with the\n- distribution.\n-\n-    - Neither the name of Intel Corporation nor the names of its\n- contributors may be used to endorse or promote products derived\n- from this software without specific prior written permission.\n-\n-    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n-    \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n-    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS\n-    FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n-    COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n-    INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n-    (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\n-    SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)\n-    HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,\n-    STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)\n-    ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED\n-    OF THE POSSIBILITY OF SUCH DAMAGE.\n+.. Copyright (c) <2022>, Intel Corporation\n+   All rights reserved.\n+\n+   Redistribution and use in source and binary forms, with or without\n+   modification, are permitted provided that the following conditions\n+   are met:\n+\n+   - Redistributions of source code must retain the above copyright\n+     notice, this list of conditions and the following disclaimer.\n+\n+   - Redistributions in binary form must reproduce the above copyright\n+     notice, this list of conditions and the following disclaimer in\n+     the documentation and/or other materials provided with the\n+     distribution.\n+\n+   - Neither the name of Intel Corporation nor the names of its\n+     contributors may be used to endorse or promote products derived\n+     from this software without specific prior written permission.\n+\n+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS\n+   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n+   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n+   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n+   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\n+   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)\n+   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,\n+   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)\n+   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED\n+   OF THE POSSIBILITY OF SUCH DAMAGE.\n \n ==========================================================\n DMA-accelerated Tx operations for vhost-user PMD test plan\n ==========================================================\n \n-Overview\n+Description\n+===========\n+\n+This document provides the test plan for testing Vhost asynchronous\n+data path with CBDMA driver in the PVP topology environment with testpmd.\n+\n+CBDMA is a kind of DMA engine, Vhost asynchronous data path leverages DMA devices\n+to offload memory copies from the CPU and it is implemented in an asynchronous way.\n+Linux kernel and DPDK provide CBDMA driver, no matter which driver is used,\n+DPDK DMA library is used in data-path to offload copies to CBDMA, and the only difference is which driver configures CBDMA.\n+It enables applications, like OVS, to save CPU cycles and hide memory copy overhead, thus achieving higher throughput.\n+Vhost doesn't manage DMA devices and applications, like OVS, need to manage and configure CBDMA devices.\n+Applications need to tell vhost what CBDMA devices to use in every data path function call.\n+This design enables the flexibility for applications to dynamically use DMA channels in different\n+function modules, not limited in vhost. In addition, vhost supports M:N mapping between vrings\n+and DMA virtual channels. Specifically, one vring can use multiple different DMA channels\n+and one DMA channel can be shared by multiple vrings at the same time.\n+\n+Note:\n+1. When CBDMA devices are bound to vfio driver, VA mode is the default and recommended.\n+   For PA mode, page by page mapping may exceed IOMMU's max capability, better to use 1G guest hugepage.\n+2. DPDK local patch that about vhost pmd is needed when testing Vhost asynchronous data path with testpmd.\n+\n+For more about dpdk-testpmd sample, please refer to the DPDK docments:\n+https://doc.dpdk.org/guides/testpmd_app_ug/run_app.html\n+For virtio-user vdev parameter, you can refer to the DPDK docments:\n+https://doc.dpdk.org/guides/nics/virtio.html#virtio-paths-selection-and-usage.\n+\n+Prerequisites\n+=============\n+\n+Topology\n --------\n+      Test flow: TG-->NIC-->Vhost-->Virtio-->Vhost-->NIC-->TG\n \n-This feature supports to offload large data movement in vhost enqueue operations\n-from the CPU to the I/OAT(a DMA engine in Intel's processor) device for every queue.\n-In addition, a queue can only use one I/OAT device, and I/OAT devices cannot be shared\n-among vhost ports and queues. That is, an I/OAT device can only be used by one queue at\n-a time. DMA devices(e.g.,CBDMA) used by queues are assigned by users; for a queue without\n-assigning a DMA device, the PMD will leverages librte_vhost to perform vhost enqueue\n-operations. Moreover, users cannot enable I/OAT acceleration for live-migration. Large\n-copies are offloaded from the CPU to the DMA engine in an asynchronous manner. The CPU\n-just submits copy jobs to the DMA engine and without waiting for DMA copy completion;\n-there is no CPU intervention during DMA data transfer. By overlapping CPU\n-computation and DMA copy, we can save precious CPU cycles and improve the overall\n-throughput for vhost-user PMD based applications, like OVS. Due to startup overheads\n-associated with DMA engines, small copies are performed by the CPU.\n-DPDK 21.11 adds vfio support for DMA device in vhost. When DMA devices are bound to\n-vfio driver, VA mode is the default and recommended. For PA mode, page by page mapping\n-may exceed IOMMU's max capability, better to use 1G guest hugepage.\n-\n-We introduce a new vdev parameter to enable DMA acceleration for Tx operations of queues:\n-- dmas: This parameter is used to specify the assigned DMA device of a queue.\n+Hardware\n+--------\n+      Supportted NICs: ALL\n+\n+Software\n+--------\n+      Trex:http://trex-tgn.cisco.com/trex/release/v2.26.tar.gz\n+\n+General set up\n+--------------\n+1. Compile DPDK::\n+\n+      # CC=gcc meson --werror -Denable_kmods=True -Dlibdir=lib -Dexamples=all --default-library=<dpdk build dir>\n+      # ninja -C <dpdk build dir> -j 110\n+      For example：\n+      CC=gcc meson --werror -Denable_kmods=True -Dlibdir=lib -Dexamples=all --default-library=x86_64-native-linuxapp-gcc\n+      ninja -C x86_64-native-linuxapp-gcc -j 110\n+\n+2. Get the PCI device ID and DMA device ID of DUT, for example, 0000:18:00.0 is PCI device ID, 0000:00:04.0, 0000:00:04.1 are DMA device IDs::\n+\n+      <dpdk dir># ./usertools/dpdk-devbind.py -s\n+\n+      Network devices using kernel driver\n+      ===================================\n+      0000:18:00.0 'Device 159b' if=ens785f0 drv=ice unused=vfio-pci\n+\n+      DMA devices using kernel driver\n+      ===============================\n+      0000:00:04.0 'Sky Lake-E CBDMA Registers 2021' drv=ioatdma unused=vfio-pci\n+      0000:00:04.1 'Sky Lake-E CBDMA Registers 2021' drv=ioatdma unused=vfio-pci\n+\n+Test case\n+=========\n+\n+Common steps\n+------------\n+1. Bind 1 NIC port and CBDMA devices to vfio-pci::\n+\n+      <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci <DUT port pci device id>\n+      <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci <DUT port DMA device id>\n+\n+      For example, Bind 1 NIC port and 2 CBDMA devices::\n+      ./usertools/dpdk-devbind.py -b vfio-pci 0000:00:18.0\n+      ./usertools/dpdk-devbind.py -b vfio-pci 0000:00:04.0,0000:00:04.1\n+\n+2. Send imix packets [64,1518] to NIC by traffic generator::\n+\n+      The imix packets include packet size [64, 128, 256, 512, 1024, 1518], and the format of packet is as follows.\n+      +-------------+-------------+-------------+-------------+\n+      | MAC         | MAC         | IPV4        | IPV4        |\n+      | Src address | Dst address | Src address | Dst address |\n+      |-------------|-------------|-------------|-------------|\n+      | Random MAC  | Virtio mac  | Random IP   | Random IP   |\n+      +-------------+-------------+-------------+-------------+\n+      All the packets in this test plan use the Virtio mac: 00:11:22:33:44:10.\n+\n+Test Case 1: PVP split ring all path vhost enqueue operations with 1 to 1 mapping between vrings and CBDMA virtual channels\n+---------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of split ring in each virtio path with 1 core and 1 queue\n+when vhost uses the asynchronous enqueue operations and the mapping between vrings and CBDMA virtual channels is 1:1.\n+Both iova as VA and PA mode have been tested.\n+\n+1. Bind 1 NIC port and 1 CBDMA device to vfio-pci, as common step 1.\n+\n+2. Launch vhost by below command::\n+\n+      <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0 \\\n+      --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=1,dmas=[txq0],dma_ring_size=2048' \\\n+      --iova=va -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024 \\\n+      --lcore-dma=[lcore11@0000:00:04.0]\n+      testpmd> set fwd mac\n+      testpmd> start\n+\n+3. Launch virtio-user with inorder mergeable path::\n+\n+      <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+      --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=1 \\\n+      -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+      testpmd> set fwd mac\n+      testpmd> start\n+\n+4. Send imix packets [64,1518] from packet generator as common step2, and then check the throughput can get expected data::\n+\n+      testpmd> show port stats all\n+\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n+\n+      testpmd> stop\n+\n+6. Restart vhost port and send imix pkts again, then check the throuhput can get expected data::\n+\n+      testpmd> start\n+      testpmd> show port stats all\n+\n+7. Relaunch virtio-user with mergeable path, then repeat step 4-6::\n+\n+      <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+      --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=0,queues=1 \\\n+      -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+      testpmd> set fwd mac\n+      testpmd> start\n+\n+8. Relaunch virtio-user with inorder non-mergeable path, then repeat step 4-6::\n+\n+      <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+      --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,queues=1 \\\n+      -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+      testpmd> set fwd mac\n+      testpmd> start\n+\n+9. Relaunch virtio-user with non-mergeable path, then repeat step 4-6::\n+\n+      <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+      --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,queues=1 \\\n+      -- -i --enable-hw-vlan-strip --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+      testpmd> set fwd mac\n+      testpmd> start\n+\n+.. note::\n+\n+    Rx offload(s) are requested when using split ring non-mergeable path. So add the parameter \"--enable-hw-vlan-strip\".\n+\n+10. Relaunch virtio-user with vectorized path, then repeat step 4-6::\n+\n+      <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+      --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,vectorized=1,queues=1 \\\n+      -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+      testpmd> set fwd mac\n+      testpmd> start\n+\n+11. Quit all testpmd and relaunch vhost with iova=pa by below command::\n+\n+      <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0\n+      --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=1,dmas=[txq0],dma_ring_size=2048' \\\n+      --iova=pa -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024 \\\n+      --lcore-dma=[lcore11@0000:00:04.0]\n+      testpmd> set fwd mac\n+      testpmd> start\n+\n+12. Rerun steps 3-11.\n+\n+Test Case 2: PVP split ring all path multi-queues vhost enqueue operations with 1 to 1 mapping between vrings and CBDMA virtual channels\n+----------------------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of split ring in each virtio path with multi-queues\n+when vhost uses the asynchronous enqueue operations and the mapping between vrings and CBDMA virtual channels is 1:1.\n+Both iova as VA and PA mode have been tested.\n+\n+1. Bind 1 NIC port and 8 CBDMA devices to vfio-pci, as common step 1.\n+\n+2. Launch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=8 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.1,lcore13@0000:00:04.2,lcore14@0000:00:04.3,lcore15@0000:00:04.4,lcore16@0000:00:04.5,lcore17@0000:00:04.6,lcore18@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+3. Launch virtio-user with inorder mergeable path::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+4. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n+\n+       testpmd> show port stats all\n+\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n+\n+       testpmd> stop\n+\n+6. Restart vhost port and send imix pkts again, then check the throuhput can get expected data::\n+\n+       testpmd> start\n+       testpmd> show port stats all\n+\n+7. Relaunch virtio-user with mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=0,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+8. Relaunch virtio-user with inorder non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+9. Relaunch virtio-user with non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,queues=8 \\\n+       -- -i --enable-hw-vlan-strip --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+10. Relaunch virtio-user with vectorized path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,vectorized=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+11. Quit all testpmd and relaunch vhost with iova=pa by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=pa -- -i --nb-cores=8 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.1,lcore13@0000:00:04.2,lcore14@0000:00:04.3,lcore15@0000:00:04.4,lcore16@0000:00:04.5,lcore17@0000:00:04.6,lcore18@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+12. Rerun step 7.\n+\n+Test Case 3: PVP split ring all path multi-queues vhost enqueue operations with M to 1 mapping between vrings and CBDMA virtual channels\n+----------------------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of split ring in each virtio path with multi-queues\n+when vhost uses the asynchronous enqueue operations and the mapping between vrings and CBDMA virtual channels is M:1.\n+Both iova as VA and PA mode have been tested.\n+\n+1. Bind 1 NIC port and 8 CBDMA devices to vfio-pci, as common step 1.\n+\n+2. Launch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+3. Launch virtio-user with inorder mergeable path::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+3. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n+\n+       testpmd> show port stats all\n+\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n+\n+       testpmd> stop\n+\n+6. Restart vhost port and send imix pkts again, then check the throught can get expected data::\n+\n+       testpmd> start\n+       testpmd> show port stats all\n+\n+7. Relaunch virtio-user with mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=0,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+8. Relaunch virtio-user with inorder non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+9. Relaunch virtio-user with non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,queues=8 \\\n+       -- -i --enable-hw-vlan-strip --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+10. Relaunch virtio-user with vectorized path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,vectorized=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+11. Quit all testpmd and relaunch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=3 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.0,lcore13@0000:00:04.0]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+12. Rerun steps 4-6.\n+\n+13. Quit all testpmd and relaunch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=8 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.0,lcore13@0000:00:04.0,lcore14@0000:00:04.0,lcore15@0000:00:04.0,lcore16@0000:00:04.0,lcore17@0000:00:04.0,lcore18@0000:00:04.0]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+14. Rerun steps 7.\n+\n+15. Quit all testpmd and relaunch vhost with iova=pa by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=pa -- -i --nb-cores=8 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.0,lcore13@0000:00:04.0,lcore14@0000:00:04.0,lcore15@0000:00:04.0,lcore16@0000:00:04.0,lcore17@0000:00:04.0,lcore18@0000:00:04.0]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+16. Rerun steps 7.\n+\n+Test Case 4: PVP split ring all path vhost enqueue operations with 1 to N mapping between vrings and CBDMA virtual channels\n+---------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of split ring in each virtio path when vhost uses\n+the asynchronous enqueue operations and the mapping between vrings and CBDMA virtual channels is 1:N.\n+Both iova as VA and PA mode have been tested.\n+\n+1. Bind 1 NIC port and 8 CBDMA devices to vfio-pci, as common step 1.\n+\n+2. Launch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=1,dmas=[txq0],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.1,lcore11@0000:00:04.2,lcore11@0000:00:04.3,lcore11@0000:00:04.4,lcore11@0000:00:04.5,lcore11@0000:00:04.6,lcore11@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+3. Launch virtio-user with inorder mergeable path::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+4. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n+\n+       testpmd> show port stats all\n+\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n+\n+       testpmd> stop\n+\n+6. Restart vhost port and send imix pkts again, then check the throught can get expected data::\n+\n+       testpmd> start\n+       testpmd> show port stats all\n+\n+7. Relaunch virtio-user with mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=0,queues=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+8. Relaunch virtio-user with inorder non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,queues=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+9. Relaunch virtio-user with non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,queues=1 \\\n+       -- -i --enable-hw-vlan-strip --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+10. Relaunch virtio-user with vectorized path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,vectorized=1,queues=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+11. Quit all testpmd and relaunch vhost with iova=pa by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=1,dmas=[txq0],dma_ring_size=2048' \\\n+       --iova=pa -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.1,lcore11@0000:00:04.2,lcore11@0000:00:04.3,lcore11@0000:00:04.4,lcore11@0000:00:04.5,lcore11@0000:00:04.6,lcore11@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+12. Rerun steps 9.\n+\n+Test Case 5: PVP split ring all path multi-queues vhost enqueue operations with M to N mapping between vrings and CBDMA virtual channels\n+----------------------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of split ring in each virtio path with multi-queues\n+when vhost uses the asynchronous enqueue operations and the mapping between vrings and CBDMA virtual channels is M:N.\n+Both iova as VA and PA mode have been tested.\n+\n+1. Bind 1 NIC port and 8 CBDMA devices to vfio-pci, as common step 1.\n+\n+2. Launch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.1,lcore11@0000:00:04.2,lcore11@0000:00:04.3,lcore11@0000:00:04.4,lcore11@0000:00:04.5,lcore11@0000:00:04.6,lcore11@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+3. Launch virtio-user with inorder mergeable path::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+4. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n+\n+       testpmd> show port stats all\n+\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n+\n+       testpmd> stop\n+\n+6. Restart vhost port and send imix pkts again, then check the throught can get expected data::\n+\n+       testpmd> start\n+       testpmd> show port stats all\n+\n+7. Relaunch virtio-user with mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=0,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+8. Relaunch virtio-user with inorder non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+9. Relaunch virtio-user with non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,queues=8 \\\n+       -- -i --enable-hw-vlan-strip --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+10. Relaunch virtio-user with vectorized path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,vectorized=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+11. Quit all testpmd and relaunch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.1,lcore11@0000:00:04.2,lcore11@0000:00:04.3,lcore11@0000:00:04.4,lcore11@0000:00:04.5,lcore11@0000:00:04.6,lcore11@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+12. Rerun steps 8.\n+\n+13. Quit all testpmd and relaunch vhost with iova=pa by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=pa -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.1,lcore11@0000:00:04.2,lcore11@0000:00:04.3,lcore11@0000:00:04.4,lcore11@0000:00:04.5,lcore11@0000:00:04.6,lcore11@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+14. Rerun steps 10.\n+\n+Test Case 6: PVP split ring dynamic queue number vhost enqueue operations with M to N mapping between vrings and CBDMA virtual channels\n+---------------------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of split ring when vhost uses the asynchronous enqueue operations\n+and if the vhost-user can work well when the queue number dynamic change. Both iova as VA and PA mode have been tested.\n+Both iova as VA and PA mode have been tested.\n+\n+1. Bind 1 NIC port and 8 CBDMA devices to vfio-pci, as common step 1.\n+\n+2. Launch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,client=1' \\\n+       --iova=va -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+3. Launch virtio-user by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=8,server=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+4. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n+\n+       testpmd> show port stats all\n+\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log.\n+\n+       testpmd> stop\n+\n+6. Restart vhost port and send imix pkts again, then check the throught can get expected data::\n+\n+       testpmd> start\n+       testpmd> show port stats all\n+\n+7. Quit and relaunch vhost with 1:1 mapping between vrings and CBDMA virtual channels, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,client=1,dmas=[txq0;txq1;txq2;txq3]' \\\n+       --iova=va -- -i --nb-cores=4 --txq=4 --rxq=4 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.1,lcore13@0000:00:04.2,lcore14@0000:00:04.3]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+8. Quit and relaunch vhost with M:N(1:N;M<N) mapping between vrings and CBDMA virtual channels, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18  --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,client=1,dmas=[txq0;txq1;txq2;txq3;txq4;txq6;txq7]' \\\n+       --iova=va -- -i --nb-cores=5 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.7,lcore12@0000:00:04.1,lcore12@0000:00:04.2,lcore12@0000:00:04.3,lcore13@0000:00:04.2,lcore13@0000:00:04.3,lcore13@0000:00:04.4,lcore14@0000:00:04.2,lcore14@0000:00:04.3,lcore14@0000:00:04.4,lcore14@0000:00:04.5,lcore15@0000:00:04.0,lcore15@0000:00:04.1,lcore15@0000:00:04.2,lcore15@0000:00:04.3,lcore15@0000:00:04.4,lcore15@0000:00:04.5,lcore15@0000:00:04.6,lcore15@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+9. Quit and relaunch vhost with diff M:N(M:1;M>N) mapping between vrings and CBDMA virtual channels, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,client=1,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6]' \\\n+       --iova=va -- -i --nb-cores=5 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.0,lcore13@0000:00:04.1,lcore13@0000:00:04.2,lcore14@0000:00:04.1,lcore14@0000:00:04.2,lcore15@0000:00:04.1,lcore15@0000:00:04.2]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+10. Quit and relaunch vhost with iova=pa by below command, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,client=1,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7]' \\\n+       --iova=pa -- -i --nb-cores=5 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.0,lcore13@0000:00:04.1,lcore13@0000:00:04.2,lcore14@0000:00:04.1,lcore14@0000:00:04.2,lcore15@0000:00:04.1,lcore15@0000:00:04.2]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+Test Case 7: PVP packed ring all path vhost enqueue operations with 1 to 1 mapping between vrings and CBDMA virtual channels\n+----------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of packed ring in each virtio path with 1 core and 1 queue\n+when vhost uses the asynchronous enqueue operations and the mapping between vrings and CBDMA virtual channels is 1:1.\n+Both iova as VA and PA mode have been tested.\n+\n+1. Bind 1 NIC port and 1 CBDMA device to vfio-pci, as common step 1.\n+\n+2. Launch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost  -a 0000:18:00.0 -a 0000:00:04.0\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=1,dmas=[txq0],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+3. Launch virtio-user with inorder mergeable path::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=1,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+4. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n+\n+       testpmd> show port stats all\n+\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n+\n+       testpmd> stop\n+\n+6. Restart vhost port and send imix pkts again, then check the throught can get expected data::\n+\n+       testpmd> start\n+       testpmd> show port stats all\n+\n+7. Relaunch virtio-user with mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=0,queues=1,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+8. Relaunch virtio-user with inorder non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,queues=1,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+9. Relaunch virtio-user with non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,queues=1,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+10. Relaunch virtio-user with vectorized path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,packed_vq=1,vectorized=1,queues=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+.. note::\n+\n+   If building and running environment support (AVX512 || NEON) && in-order feature is negotiated && Rx mergeable\n+   is not negotiated && TCP_LRO Rx offloading is disabled && vectorized option enabled, packed virtqueue vectorized Rx path will be selected.\n+\n+11. Relaunch virtio-user with vectorized path and ring size is not power of 2, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,packed_vq=1,vectorized=1,queues=1,queue_size=1025 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1025 --rxd=1025\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+12. Quit all testpmd and relaunch vhost with iova=pa by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=1,dmas=[txq0],dma_ring_size=2048' \\\n+       --iova=pa -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+12. Rerun steps 3-6.\n+\n+Test Case 8: PVP packed ring all path multi-queues vhost enqueue operations with 1 to 1 mapping between vrings and CBDMA virtual channels\n+-----------------------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of packed ring in each virtio path with multi-queues\n+when vhost uses the asynchronous enqueue operations and the mapping between vrings and CBDMA virtual channels is 1:1.\n+Both iova as VA and PA mode have been tested.\n+\n+1. Bind 1 NIC port and 8 CBDMA devices to vfio-pci, as common step 1.\n+\n+2. Launch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=8 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.1,lcore13@0000:00:04.2,lcore14@0000:00:04.3,lcore15@0000:00:04.4,lcore16@0000:00:04.5,lcore17@0000:00:04.6,lcore18@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+3. Launch virtio-user with inorder mergeable path::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+4. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n+\n+       testpmd> show port stats all\n+\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n+\n+       testpmd> stop\n+\n+6. Restart vhost port and send imix pkts again, then check the throught can get expected data::\n+\n+       testpmd> start\n+       testpmd> show port stats all\n+\n+7. Relaunch virtio-user with mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=0,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+8. Relaunch virtio-user with inorder non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+9. Relaunch virtio-user with non-mergeable path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+10. Relaunch virtio-user with vectorized path, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,packed_vq=1,vectorized=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+11. Relaunch virtio-user with vectorized path and ring size is not power of 2, then repeat step 4-6::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,packed_vq=1,vectorized=1,queues=8,queue_size=1025 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1025 --rxd=1025\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+12. Quit all testpmd and relaunch vhost with iova=pa by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=pa -- -i --nb-cores=8 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.1,lcore13@0000:00:04.2,lcore14@0000:00:04.3,lcore15@0000:00:04.4,lcore16@0000:00:04.5,lcore17@0000:00:04.6,lcore18@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n+\n+13. Rerun step 7.\n+\n+Test Case 9: PVP packed ring all path multi-queues vhost enqueue operations with M to 1 mapping between vrings and CBDMA virtual channels\n+-----------------------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of packed ring in each virtio path with multi-queues\n+when vhost uses the asynchronous enqueue operations and the mapping between vrings and CBDMA virtual channels is M:1.\n+Both iova as VA and PA mode have been tested.\n+\n+1. Bind 1 NIC port and 1 CBDMA device to vfio-pci, as common step 1.\n+\n+2. Launch vhost by below command::\n+\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-Here is an example:\n-./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c f -n 4 \\\n---vdev 'net_vhost0,iface=/tmp/s0,queues=1,dmas=[txq0@0000:00:04.0] \\\n---iova=va -- -i'\n+3. Launch virtio-user with inorder mergeable path::\n \n-Test Case 1: PVP split ring all path vhost enqueue operations with cbdma\n-========================================================================\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-Packet pipeline:\n-================\n-TG --> NIC --> Vhost --> Virtio--> Vhost --> NIC --> TG\n+4. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n \n-1. Bind 1 CBDMA port and 1 NIC port to vfio-pci, then launch vhost by below command::\n+       testpmd> show port stats all\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --file-prefix=vhost \\\n-    --vdev 'net_vhost0,iface=/tmp/s0,queues=1,dmas=[txq0@0000:00:04.0]' \\\n-    --iova=va -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n \n-2. Launch virtio-user with inorder mergeable path::\n+       testpmd> stop\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=1,in_order=1,queues=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+6. Restart vhost port and send imix pkts again, then check the throught can get expected data::\n \n-3. Send imix packets [64,1518] from packet generator, check the throughput can get expected data, restart vhost port and send imix pkts again, check get same throuhput::\n+       testpmd> start\n+       testpmd> show port stats all\n \n-    testpmd>show port stats all\n-    testpmd>stop\n-    testpmd>start\n-    testpmd>show port stats all\n+7. Relaunch virtio-user with mergeable path, then repeat step 4-6::\n \n-4. Relaunch virtio-user with mergeable path, then repeat step 3::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=0,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=1,in_order=0,queues=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+8. Relaunch virtio-user with inorder non-mergeable path, then repeat step 4-6::\n \n-5. Relaunch virtio-user with inorder non-mergeable path, then repeat step 3::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=0,in_order=1,queues=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+9. Relaunch virtio-user with non-mergeable path, then repeat step 4-6::\n \n-6. Relaunch virtio-user with non-mergeable path, then repeat step 3::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=0,in_order=0,queues=1 \\\n-    -- -i --enable-hw-vlan-strip --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+10. Relaunch virtio-user with vectorized path, then repeat step 4-6::\n \n-7. Relaunch virtio-user with vector_rx path, then repeat step 3::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,packed_vq=1,vectorized=1,queues=8 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=0,in_order=0,vectorized=1,queues=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+11. Relaunch virtio-user with vectorized path and ring size is not power of 2, then repeat step 4-6::\n \n-8. Quit all testpmd and relaunch vhost with iova=pa by below command::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,packed_vq=1,vectorized=1,queues=8,queue_size=1025 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1025 --rxd=1025\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --file-prefix=vhost \\\n-    --vdev 'net_vhost0,iface=/tmp/s0,queues=1,dmas=[txq0@0000:00:04.0]' \\\n-    --iova=pa -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+12. Quit all testpmd and relaunch vhost by below command::\n \n-9. Rerun steps 2-7.\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=3 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.0,lcore13@0000:00:04.0]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-Test Case 2: PVP split ring dynamic queue number vhost enqueue operations with cbdma\n-=====================================================================================\n+13. Rerun steps 3-6.\n \n-1. Bind 8 CBDMA ports and 1 NIC port to vfio-pci, then launch vhost by below command::\n+14. Quit all testpmd and relaunch vhost by below command::\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 28-29 --file-prefix=vhost \\\n-    --vdev 'net_vhost0,iface=/tmp/s0,queues=8,client=1' \\\n-    --iova=va -- -i --nb-cores=1 --txd=1024 --rxd=1024 --txq=8 --rxq=8\n-    >set fwd mac\n-    >start\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=8 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.0,lcore13@0000:00:04.0,lcore14@0000:00:04.0,lcore15@0000:00:04.0,lcore16@0000:00:04.0,lcore17@0000:00:04.0,lcore18@0000:00:04.0]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-2. Launch virtio-user by below command::\n+15. Rerun steps 7.\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 30-31 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=1,in_order=1,queues=8,server=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024 --txq=8 --rxq=8\n-    >set fwd mac\n-    >start\n+16. Quit all testpmd and relaunch vhost with iova=pa by below command::\n \n-3. Send imix packets[64,1518] from packet generator with random ip, check perforamnce can get target.\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 -a 0000:00:04.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=pa -- -i --nb-cores=8 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.0,lcore13@0000:00:04.0,lcore14@0000:00:04.0,lcore15@0000:00:04.0,lcore16@0000:00:04.0,lcore17@0000:00:04.0,lcore18@0000:00:04.0]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-4. Stop vhost port, check vhost RX and TX direction both exist packtes in 8 queues from vhost log.\n+17. Rerun steps 8.\n \n-5. Quit and relaunch vhost with 4 queues w/ cbdma and 4 queues w/o cbdma::\n+Test Case 10: PVP packed ring all path vhost enqueue operations with 1 to N mapping between vrings and CBDMA virtual channels\n+-----------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of packed ring in each virtio path when vhost uses\n+the asynchronous enqueue operations and the mapping between vrings and CBDMA virtual channels is 1:N.\n+Both iova as VA and PA mode have been tested.\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 28-29 --file-prefix=vhost \\\n-    --vdev 'net_vhost0,iface=/tmp/s0,queues=8,client=1,dmas=[txq0@0000:00:04.0;txq1@0000:00:04.1;txq2@0000:00:04.2;txq3@0000:00:04.3]' \\\n-    --iova=va -- -i --nb-cores=1 --txd=1024 --rxd=1024 --txq=8 --rxq=8\n-    >set fwd mac\n-    >start\n+1. Bind 1 NIC port and 8 CBDMA devices to vfio-pci, as common step 1.\n \n-6. Send imix packets[64,1518] from packet generator with random ip, check perforamnce can get target.\n+2. Launch vhost by below command::\n \n-7. Stop vhost port, check vhost RX and TX direction both exist packtes in 8 queues from vhost log.\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=1,dmas=[txq0],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.1,lcore11@0000:00:04.2,lcore11@0000:00:04.3,lcore11@0000:00:04.4,lcore11@0000:00:04.5,lcore11@0000:00:04.6,lcore11@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-8. Quit and relaunch vhost with 8 queues w/ cbdma::\n+3. Launch virtio-user with inorder mergeable path::\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 28-29 --file-prefix=vhost \\\n-    --vdev 'net_vhost0,iface=/tmp/s0,queues=8,client=1,dmas=[txq0@0000:00:04.0;txq1@0000:00:04.1;txq2@0000:00:04.2;txq3@0000:00:04.3;txq4@0000:00:04.4;txq5@0000:00:04.5;txq6@0000:00:04.6;txq7@0000:00:04.7]' \\\n-    --iova=va -- -i --nb-cores=1 --txd=1024 --rxd=1024 --txq=8 --rxq=8\n-    >set fwd mac\n-    >start\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=1,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-9. Send imix packets[64,1518] from packet generator with random ip, check perforamnce can get target.\n+4. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n \n-10. Stop vhost port, check vhost RX and TX direction both exist packtes in 8 queues from vhost log.\n+       testpmd> show port stats all\n \n-11. Quit and relaunch vhost with iova=pa, 6 queues w/ cbdma and 2 queues w/o cbdma::\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n \n-\t./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 28-29 --file-prefix=vhost \\\n-\t--vdev 'net_vhost0,iface=/tmp/s0,queues=8,client=1,dmas=[txq0@0000:00:04.0;txq1@0000:00:04.1;txq2@0000:00:04.2;txq3@0000:00:04.3;txq4@0000:00:04.4;txq5@0000:00:04.5]' \\\n-\t--iova=pa -- -i --nb-cores=1 --txd=1024 --rxd=1024 --txq=8 --rxq=8\n-\t>set fwd mac\n-\t>start\n+       testpmd> stop\n \n-12. Send imix packets[64,1518] from packet generator with random ip, check perforamnce can get target.\n+6. Restart vhost port and send imix pkts again, then check the throught can get expected data::\n \n-13. Stop vhost port, check vhost RX and TX direction both exist packtes in 8 queues from vhost log.\n+       testpmd> start\n+       testpmd> show port stats all\n \n-Test Case 3: PVP packed ring all path vhost enqueue operations with cbdma\n-=========================================================================\n+7. Relaunch virtio-user with mergeable path, then repeat step 4-6::\n \n-Packet pipeline:\n-================\n-TG --> NIC --> Vhost --> Virtio--> Vhost --> NIC --> TG\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=0,queues=1,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-1. Bind 1 CBDMA port and 1 NIC port to vfio-pci, then launch vhost by below command::\n+8. Relaunch virtio-user with inorder non-mergeable path, then repeat step 4-6::\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --file-prefix=vhost \\\n-    --vdev 'net_vhost0,iface=/tmp/s0,queues=1,dmas=[txq0@0000:80:04.0]' \\\n-    --iova=va -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,queues=1,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-2. Launch virtio-user with inorder mergeable path::\n+9. Relaunch virtio-user with non-mergeable path, then repeat step 4-6::\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=1,in_order=1,queues=1,packed_vq=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,queues=1,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-3. Send imix packets [64,1518] from packet generator, check the throughput can get expected data, restart vhost port and send imix pkts again, check get same throuhput::\n+10. Relaunch virtio-user with vectorized path, then repeat step 4-6::\n \n-    testpmd>show port stats all\n-    testpmd>stop\n-    testpmd>start\n-    testpmd>show port stats all\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,packed_vq=1,vectorized=1,queues=1 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-4. Relaunch virtio-user with mergeable path, then repeat step 3::\n+11. Relaunch virtio-user with vectorized path and ring size is not power of 2, then repeat step 3::\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=1,in_order=0,queues=1,packed_vq=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,packed_vq=1,vectorized=1,queues=1,queue_size=1025 \\\n+       -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1025 --rxd=1025\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-5. Relaunch virtio-user with inorder non-mergeable path, then repeat step 3::\n+12. Quit all testpmd and relaunch vhost with iova=pa by below command::\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=0,in_order=1,queues=1,packed_vq=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=1,dmas=[txq0],dma_ring_size=2048' \\\n+       --iova=pa -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.1,lcore11@0000:00:04.2,lcore11@0000:00:04.3,lcore11@0000:00:04.4,lcore11@0000:00:04.5,lcore11@0000:00:04.6,lcore11@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-6. Relaunch virtio-user with non-mergeable path, then repeat step 3::\n+13. Rerun steps 9.\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=0,in_order=0,queues=1,packed_vq=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+Test Case 11: PVP packed ring all path multi-queues vhost enqueue operations with M to N mapping between vrings and CBDMA virtual channels\n+------------------------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of packed ring in each virtio path with multi-queues\n+when vhost uses the asynchronous enqueue operations and the mapping between vrings and CBDMA virtual channels is M:N.\n+Both iova as VA and PA mode have been tested.\n \n-7. Relaunch virtio-user with vectorized path, then repeat step 3::\n+1. Bind 1 NIC port and 8 CBDMA devices to vfio-pci, as common step 1.\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=0,in_order=1,packed_vq=1,vectorized=1,queues=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+2. Launch vhost by below command::\n \n-8. Relaunch virtio-user with vectorized path and ring size is not power of 2, then repeat step 3::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.1,lcore11@0000:00:04.2,lcore11@0000:00:04.3,lcore11@0000:00:04.4,lcore11@0000:00:04.5,lcore11@0000:00:04.6,lcore11@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 5-6 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=0,in_order=1,packed_vq=1,vectorized=1,queues=1,queue_size=1025 \\\n-    -- -i --nb-cores=1 --txd=1025 --rxd=1025\n-    >set fwd mac\n-    >start\n+3. Launch virtio-user with inorder mergeable path::\n \n-9. Quit all testpmd and relaunch vhost with iova=pa by below command::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --file-prefix=vhost \\\n-    --vdev 'net_vhost0,iface=/tmp/s0,queues=1,dmas=[txq0@0000:80:04.0]' \\\n-    --iova=pa -- -i --nb-cores=1 --txd=1024 --rxd=1024\n-    >set fwd mac\n-    >start\n+4. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n \n-10. Rerun steps 2-8.\n+       testpmd> show port stats all\n \n-Test Case 4: PVP packed ring dynamic queue number vhost enqueue operations with cbdma\n-=====================================================================================\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n \n-1. Bind 8 CBDMA ports and 1 NIC port to vfio-pci, then launch vhost by below command::\n+       testpmd> stop\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 28-29 --file-prefix=vhost \\\n-    --vdev 'net_vhost0,iface=/tmp/s0,queues=8,client=1' \\\n-    --iova=va -- -i --nb-cores=1 --txd=1024 --rxd=1024 --txq=8 --rxq=8\n-    >set fwd mac\n-    >start\n+6. Restart vhost port and send imix pkts again, then check the throught can get expected data::\n \n-2. Launch virtio-user by below command::\n+       testpmd> start\n+       testpmd> show port stats all\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 30-31 --no-pci --file-prefix=virtio \\\n-    --vdev=net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/s0,mrg_rxbuf=1,in_order=0,queues=8,server=1,packed_vq=1 \\\n-    -- -i --nb-cores=1 --txd=1024 --rxd=1024 --txq=8 --rxq=8\n-    >set fwd mac\n-    >start\n+7. Relaunch virtio-user with mergeable path, then repeat step 4-6::\n \n-3. Send imix packets from packet generator with random ip, check perforamnce can get target.\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=0,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-4. Stop vhost port, check vhost RX and TX direction both exist packtes in 8 queues from vhost log.\n+8. Relaunch virtio-user with inorder non-mergeable path, then repeat step 4-6::\n \n-5. Quit and relaunch vhost with 4 queues w/ cbdma and 4 queues w/o cbdma::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=1,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 28-29 --file-prefix=vhost \\\n-    --vdev 'net_vhost0,iface=/tmp/s0,queues=8,client=1,dmas=[txq0@0000:80:04.0;txq1@0000:80:04.1;txq2@0000:80:04.2;txq3@0000:80:04.3]' \\\n-    --iova=va -- -i --nb-cores=1 --txd=1024 --rxd=1024 --txq=8 --rxq=8\n-    >set fwd mac\n-    >start\n+9. Relaunch virtio-user with non-mergeable path, then repeat step 4-6::\n \n-6. Send imix packets from packet generator with random ip, check perforamnce can get target.\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,queues=8,packed_vq=1 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-7. Stop vhost port, check vhost RX and TX direction both exist packtes in 4 queues from vhost log.\n+10. Relaunch virtio-user with vectorized path, then repeat step 4-6::\n \n-8. Quit and relaunch vhost with 8 queues w/ cbdma::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,packed_vq=1,vectorized=1,queues=8, \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 28-29 --file-prefix=vhost \\\n-    --vdev 'net_vhost0,iface=/tmp/s0,queues=8,client=1,dmas=[txq0@0000:80:04.0;txq1@0000:80:04.1;txq2@0000:80:04.2;txq3@0000:80:04.3;txq4@0000:80:04.4;txq5@0000:80:04.5;txq6@0000:80:04.6;txq7@0000:80:04.7]' \\\n-    --iova=va -- -i --nb-cores=1 --txd=1024 --rxd=1024 --txq=8 --rxq=8\n-    >set fwd mac\n-    >start\n+11. Relaunch virtio-user with vectorized path and ring size is not power of 2, then repeat step 4-6::\n \n-9. Send imix packets from packet generator with random ip, check perforamnce can get target.\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio --force-max-simd-bitwidth=512 \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=0,in_order=0,packed_vq=1,vectorized=1,queues=8,queue_size=1025 \\\n+       -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1025 --rxd=1025\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-10. Stop vhost port, check vhost RX and TX direction both exist packtes in 8 queues from vhost log.\n+12. Quit all testpmd and relaunch vhost by below command::\n \n-11. Quit and relaunch vhost with iova=pa, 6 queues w/ cbdma and 2 queues w/o cbdma::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=va -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.1,lcore11@0000:00:04.2,lcore11@0000:00:04.3,lcore11@0000:00:04.4,lcore11@0000:00:04.5,lcore11@0000:00:04.6,lcore11@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-\t./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 28-29 --file-prefix=vhost \\\n-\t--vdev 'net_vhost0,iface=/tmp/s0,queues=8,client=1,dmas=[txq0@0000:80:04.0;txq1@0000:80:04.1;txq2@0000:80:04.2;txq3@0000:80:04.3;txq4@0000:80:04.4;txq5@0000:80:04.5]' \\\n-\t--iova=pa -- -i --nb-cores=1 --txd=1024 --rxd=1024 --txq=8 --rxq=8\n-\t>set fwd mac\n-\t>start\n+13. Rerun steps 7.\n \n-12. Send imix packets from packet generator with random ip, check perforamnce can get target.\n+14. Quit all testpmd and relaunch vhost with iova=pa by below command::\n \n-13. Stop vhost port, check vhost RX and TX direction both exist packtes in 8 queues from vhost log.\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7],dma_ring_size=2048' \\\n+       --iova=pa -- -i --nb-cores=1 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.1,lcore11@0000:00:04.2,lcore11@0000:00:04.3,lcore11@0000:00:04.4,lcore11@0000:00:04.5,lcore11@0000:00:04.6,lcore11@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-Test Case 5: loopback split ring large chain packets stress test with cbdma enqueue\n-====================================================================================\n+15. Rerun steps 9.\n \n-Packet pipeline:\n-================\n-Vhost <--> Virtio\n+Test Case 12: PVP packed ring dynamic queue number vhost enqueue operations with M to N mapping between vrings and CBDMA virtual channels\n+-----------------------------------------------------------------------------------------------------------------------------------------\n+This case uses testpmd and Traffic Generator(For example, Trex) to test performance of packed ring when vhost uses the asynchronous enqueue operations\n+and if the vhost-user can work well when the queue number dynamic change. Both iova as VA and PA mode have been tested.\n+Both iova as VA and PA mode have been tested.\n \n-1. Bind 1 CBDMA channel to vfio-pci and launch vhost::\n+1. Bind 1 NIC port and 8 CBDMA devices to vfio-pci, as common step 1.\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-3 -n 4 \\\n-    --vdev 'eth_vhost0,iface=vhost-net0,queues=1,dmas=[txq0@0000:00:04.0]' \\\n-    --iova=va -- -i --nb-cores=1 --mbuf-size=65535\n+2. Launch vhost by below command::\n \n-2. Launch virtio and start testpmd::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost -a 0000:18:00.0 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,client=1' \\\n+        --iova=va -- -i --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 30,31 -n 4  --file-prefix=testpmd0 --no-pci  \\\n-    --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=./vhost-net0,queues=1, \\\n-    mrg_rxbuf=1,in_order=0,vectorized=1,queue_size=2048 \\\n-    -- -i --rxq=1 --txq=1 --txd=2048 --rxd=2048 --nb-cores=1\n-    >start\n+3. Launch virtio-user by below command::\n \n-3. Send large packets from vhost, check virtio can receive packets::\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 2-3 --no-pci --file-prefix=virtio \\\n+       --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost_net0,mrg_rxbuf=1,in_order=1,queues=1,server=1,packed_vq=1 \\\n+       -- -i  --nb-cores=1 --txq=1 --rxq=1 --txd=1024 --rxd=1024\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-    testpmd> vhost enable tx all\n-    testpmd> set txpkts 65535,65535,65535,65535,65535\n-    testpmd> start tx_first 32\n-    testpmd> show port stats all\n+4. Send imix packets [64,1518] from packet generator as common step2, and check the throughput can get expected data::\n \n-4. Quit all testpmd and relaunch vhost with iova=pa::\n+       testpmd> show port stats all\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-3 -n 4 \\\n-    --vdev 'eth_vhost0,iface=vhost-net0,queues=1,dmas=[txq0@0000:00:04.0]' \\\n-    --iova=pa -- -i --nb-cores=1 --mbuf-size=65535\n+5. Stop vhost port, check that there are packets in both directions of RX and TX in each queue from vhost log::\n \n-5. Rerun steps 2-3.\n+       testpmd> stop\n \n-Test Case 6: loopback packed ring large chain packets stress test with cbdma enqueue\n-====================================================================================\n+6. Restart vhost port and send imix pkts again, then check the throught can get expected data::\n \n-Packet pipeline:\n-================\n-Vhost <--> Virtio\n+       testpmd> start\n+       testpmd> show port stats all\n \n-1. Bind 1 CBDMA channel to vfio-pci and launch vhost::\n+7. Quit and relaunch vhost with 1:1 mapping between vrings and CBDMA virtual channels, then repeat step 4-6::\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-3 -n 4 \\\n-    --vdev 'eth_vhost0,iface=vhost-net0,queues=1,dmas=[txq0@0000:00:04.0]' \\\n-    --iova=va -- -i --nb-cores=1 --mbuf-size=65535\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18  --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 \\\n+        --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,client=1,dmas=[txq0;txq1;txq2;txq3]' \\\n+       --iova=va -- -i --nb-cores=4 --txq=4 --rxq=4 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.1,lcore13@0000:00:04.2,lcore14@0000:00:04.3]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-2. Launch virtio and start testpmd::\n+9. Quit and relaunch vhost with M:N(1:N;M<N) mapping between vrings and CBDMA virtual channels, then repeat step 4-6::\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 30,31 -n 4  --file-prefix=testpmd0 --no-pci  \\\n-    --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=./vhost-net0,queues=1, \\\n-    mrg_rxbuf=1,in_order=0,vectorized=1,packed_vq=1,queue_size=2048 \\\n-    -- -i --rxq=1 --txq=1 --txd=2048 --rxd=2048 --nb-cores=1\n-    >start\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,client=1,dmas=[txq0;txq1;txq2;txq3;txq4;txq6;txq7]' \\\n+       --iova=va -- -i --nb-cores=5 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore11@0000:00:04.7,lcore12@0000:00:04.1,lcore12@0000:00:04.2,lcore12@0000:00:04.3,lcore13@0000:00:04.2,lcore13@0000:00:04.3,lcore13@0000:00:04.4,lcore14@0000:00:04.2,lcore14@0000:00:04.3,lcore14@0000:00:04.4,lcore14@0000:00:04.5,lcore15@0000:00:04.0,lcore15@0000:00:04.1,lcore15@0000:00:04.2,lcore15@0000:00:04.3,lcore15@0000:00:04.4,lcore15@0000:00:04.5,lcore15@0000:00:04.6,lcore15@0000:00:04.7]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-3. Send large packets from vhost, check virtio can receive packets::\n+11. Quit and relaunch vhost with diff M:N(M:1;M>N) mapping between vrings and CBDMA virtual channels, then repeat step 4-6::\n \n-    testpmd> vhost enable tx all\n-    testpmd> set txpkts 65535,65535,65535,65535,65535\n-    testpmd> start tx_first 32\n-    testpmd> show port stats all\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,client=1,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6]' \\\n+       --iova=va -- -i --nb-cores=5 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.0,lcore13@0000:00:04.1,lcore13@0000:00:04.2,lcore14@0000:00:04.1,lcore14@0000:00:04.2,lcore15@0000:00:04.1,lcore15@0000:00:04.2]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-4. Quit all testpmd and relaunch vhost with iova=pa::\n+13. Quit and relaunch vhost with iova=pa by below command::\n \n-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-3 -n 4 \\\n-    --vdev 'eth_vhost0,iface=vhost-net0,queues=1,dmas=[txq0@0000:00:04.0]' --iova=pa -- -i --nb-cores=1 --mbuf-size=65535\n+       <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -n 4 -l 10-18 --file-prefix=vhost \\\n+       -a 0000:18:00.0 -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 \\\n+       --vdev 'net_vhost0,iface=/tmp/vhost_net0,queues=8,client=1,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;txq6;txq7]' \\\n+       --iova=pa -- -i --nb-cores=5 --txq=8 --rxq=8 --txd=1024 --rxd=1024 \\\n+       --lcore-dma=[lcore11@0000:00:04.0,lcore12@0000:00:04.0,lcore13@0000:00:04.1,lcore13@0000:00:04.2,lcore14@0000:00:04.1,lcore14@0000:00:04.2,lcore15@0000:00:04.1,lcore15@0000:00:04.2]\n+       testpmd> set fwd mac\n+       testpmd> start\n \n-5. Rerun steps 2-3.\n+14. Rerun step 4-6.\n",
    "prefixes": [
        "V2",
        "1/3"
    ]
}