From patchwork Fri Jul 2 17:26:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Wang, Yinan" X-Patchwork-Id: 95204 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D676FA0A0C; Fri, 2 Jul 2021 10:43:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D25C84136D; Fri, 2 Jul 2021 10:43:00 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id D565B4136A for ; Fri, 2 Jul 2021 10:42:58 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10032"; a="294344090" X-IronPort-AV: E=Sophos;i="5.83,316,1616482800"; d="scan'208";a="294344090" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jul 2021 01:42:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,316,1616482800"; d="scan'208";a="493863566" Received: from dpdk-yinan-ntb1.sh.intel.com ([10.67.119.39]) by fmsmga002.fm.intel.com with ESMTP; 02 Jul 2021 01:42:56 -0700 From: Yinan Wang To: dts@dpdk.org Cc: Yinan Wang Date: Fri, 2 Jul 2021 13:26:35 -0400 Message-Id: <20210702172635.839710-1-yinan.wang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [dts] [PATCH v2] test_plans/vswitch_sample_cbdma: improve test coverage X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Signed-off-by: Yinan Wang --- test_plans/vswitch_sample_cbdma_test_plan.rst | 329 ++++++++++++------ 1 file changed, 227 insertions(+), 102 deletions(-) diff --git a/test_plans/vswitch_sample_cbdma_test_plan.rst b/test_plans/vswitch_sample_cbdma_test_plan.rst index 974c2ee2..8fcf0b0a 100644 --- a/test_plans/vswitch_sample_cbdma_test_plan.rst +++ b/test_plans/vswitch_sample_cbdma_test_plan.rst @@ -1,3 +1,35 @@ +.. Copyright (c) <2021>, Intel Corporation + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + - Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + - Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + + - Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + OF THE POSSIBILITY OF SUCH DAMAGE. + ======================================================== Vswitch sample test with vhost async data path test plan ======================================================== @@ -6,17 +38,10 @@ Description =========== Vswitch sample can leverage IOAT to accelerate vhost async data-path from dpdk 20.11. This plan test -vhost DMA operation callbacks for CBDMAV PMD and vhost async data-path in vhost sample. Also compare -the performance between CPU copy, CBDMA copy and Sync copy. -CPU copy means vhost enqueue w/o cbdma channel; CBDMA copy needs vhost enqueue with cbdma channel -using parameter '-dmas'; Sync copy needs vhost enqueue with cbdma channel, but threshold ( can be -adjusted by change value of f.async_threshold in dpdk code) is larger than forwarding packet length. - -CBDMA performance indicator -=========================== - -(1)CPU copy vs. sync copy delta < 10% for 64B packet size -(2)CBDMA copy vs sync copy delta > 5% for 1518 packet size +vhost DMA operation callbacks for CBDMA PMD and vhost async data-path in vhost sample. +From 20.11 to 21.02, only split ring support cbdma copy with vhost enqueue direction; +from 21.05,packed ring also can support cbdma copy with vhost enqueue direction; +from 21.08, split ring support cbdma copy both vhost enqueue and dequeue direction. Prerequisites ============= @@ -38,175 +63,152 @@ Modify the testpmd code as following:: Test Case1: PVP performance check with CBDMA channel using vhost async driver ============================================================================= -1. Adjust dpdk code as below:: - - --- a/examples/vhost/main.c - +++ b/examples/vhost/main.c - @@ -1343,7 +1343,7 @@ new_device(int vid) - - if (async_vhost_driver) { - f.async_inorder = 1; - - f.async_threshold = 256; - + f.async_threshold = 1518; - return rte_vhost_async_channel_register(vid, VIRTIO_RXQ, - f.intval, &channel_ops); - } - -2. Bind physical port to vfio-pci and CBDMA channel to igb_uio. +1. Bind physical port to vfio-pci and CBDMA channel to igb_uio. -3. On host, launch dpdk-vhost by below command:: +2. On host, launch dpdk-vhost by below command:: ./dpdk-vhost -c 0x1c000000 -n 4 -- \ - -p 0x1 --mergeable 1 --vm2vm 1 --dma-type ioat --stats 1 --socket-file /tmp/vhost-net --dmas [txd0@00:04.0] + -p 0x1 --mergeable 1 --vm2vm 1 --dma-type ioat --stats 1 --socket-file /tmp/vhost-net --dmas [txd0@0000:00:04.0] --client -4. Launch virtio-user with testpmd:: +3. Launch virtio-user with packed ring:: ./dpdk-testpmd -l 29-30 -n 4 --no-pci --file-prefix=testpmd0 \ - --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost-net,queues=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 + --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost-net,queues=1,mrg_rxbuf=0,in_order=1,vectorized=1,packed_vq=1,server=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 -5. Start pkts from virtio-user side to let vswitch know the mac addr:: +4. Start pkts from virtio-user side to let vswitch know the mac addr:: - testpmd>set fwd mac #if use io fwd ,ixia can't receive packets + testpmd>set fwd mac testpmd>start tx_first - testpmd>stop - testpmd>start #start to forward - -6. Inject different length pkts (packets length=64,512,1024,1518) seperately with VLAN_id=1000 and dest_mac=virtio_mac_addresss ( specific in above cmd with 00:11:22:33:44:10) to NIC using packet generator, record performance number. - -7. Adjust vswitch code as below, re-test step 3-6 and record perfromance of different packet length:: - - --- a/examples/vhost/main.c - +++ b/examples/vhost/main.c - @@ -1343,7 +1343,7 @@ new_device(int vid) - - if (async_vhost_driver) { - f.async_inorder = 1; - - f.async_threshold = 256; - + f.async_threshold = 0; - return rte_vhost_async_channel_register(vid, VIRTIO_RXQ, - f.intval, &channel_ops); - } -8. Re-test step 3-6 except to launch dpdk-vhost by below command, record perfromance of different packet length:: +5. Inject pkts (packets length=64...1518) separately with VLAN_id=1000 and dest_mac=virtio_mac_addresss (specific in above cmd with 00:11:22:33:44:10) to NIC using packet generator, record pvp (PG>nic>vswitch>virtio-user>vswitch>nic>PG) performance number can get expected. - ./dpdk-vhost -c 0x1c000000 -n 4 -- -p 0x1 --mergeable 1 --vm2vm 1 --stats 1 --socket-file /tmp/vhost-net +6. Quit and re-launch virtio-user with split ring:: -9. Compare performance, check below two performance indicator:: + ./dpdk-testpmd -l 29-30 -n 4 --no-pci --file-prefix=testpmd0 \ + --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost-net,queues=1,mrg_rxbuf=1,in_order=1,vectorized=1,server=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 - (1)CPU copy vs. sync copy delta < 10% for 64B packet size - (2)CBDMA copy vs sync copy delta > 5% for 1518 packet size +7. Re-test step 4-5, record performance of different packet length. -Test Case2: PV test with multiple CBDMA channels using vhost async driver -========================================================================== +Test Case2: PVP test with two VM and two CBDMA channels using vhost async driver +================================================================================= 1. Bind one physical ports to vfio-pci and two CBDMA channels to igb_uio. 2. On host, launch dpdk-vhost by below command:: ./dpdk-vhost -l 26-28 -n 4 -- \ - -p 0x1 --mergeable 1 --vm2vm 1 --dma-type ioat --stats 1 --socket-file /tmp/vhost-net0 --socket-file /tmp/vhost-net1 --dmas [txd0@00:04.0,txd1@00:04.1] --client + -p 0x1 --mergeable 1 --vm2vm 1 --dma-type ioat --stats 1 --socket-file /tmp/vhost-net0 --socket-file /tmp/vhost-net1 --dmas [txd0@0000:00:04.0,txd1@0000:00:04.1] --client 3. launch two virtio-user ports:: ./dpdk-testpmd -l 29-30 -n 4 --no-pci --file-prefix=testpmd0 \ - --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost-net0,queues=1,server=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 + --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost-net0,queues=1,server=1,mrg_rxbuf=0,in_order=1,vectorized=1,packed_vq=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 ./dpdk-testpmd -l 31-32 -n 4 --no-pci --file-prefix=testpmd1 \ - --vdev=net_virtio_user0,mac=00:11:22:33:44:11,path=/tmp/vhost-net1,queues=1,server=1,mrg_rxbuf=0,in_order=1,vectorized=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 + --vdev=net_virtio_user0,mac=00:11:22:33:44:11,path=/tmp/vhost-net1,queues=1,server=1,mrg_rxbuf=1,in_order=1,vectorized=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 4. Start pkts from two virtio-user side individually to let vswitch know the mac addr:: - testpmd0>start tx_first + testpmd0>set fwd mac + testpmd1>set fwd mac + testpmd1>start tx_first testpmd1>start tx_first -5. Inject 1518B packets with VLAN_id=1000 and dest_mac=virtio_mac_addresss (00:11:22:33:44:10 and 00:11:22:33:44:11) to NIC using packet generator,check two virtio-user ports can receive according packets. +5. Inject IMIX packets (64b...1518b) with VLAN_id=1000 and dest_mac=virtio_mac_addresss (00:11:22:33:44:10 and 00:11:22:33:44:11) to NIC using packet generator,record performance number can get expected from Packet generator rx side. 6. Stop dpdk-vhost side and relaunch it with same cmd as step2. -7.Start pkts from two virtio-user side individually to let vswitch know the mac addr, check two virtio-user ports still can receive according packets and get same throughput as step5:: +7. Start pkts from two virtio-user side individually to let vswitch know the mac addr:: + + testpmd0>stop + testpmd0>start tx_first + testpmd1>stop + testpmd1>start tx_first - testpmd0>stop - testpmd0>start tx_first - testpmd1>stop - testpmd1>start tx_first +8. Inject IMIX packets (64b...1518b) to NIC using packet generator, ensure get same throughput as step5. -Test Case3: VM2VM performance test with two CBDMA channels using vhost async driver -==================================================================================== +Test Case3: VM2VM forwarding test with two CBDMA channels +========================================================= 1.Bind one physical ports to vfio-pci and two CBDMA channels to igb_uio. 2. On host, launch dpdk-vhost by below command:: - ./dpdk-vhost -l 26-28 -n 4 -- -p 0x1 --mergeable 1 --vm2vm 1 --dma-type ioat \ - --socket-file /tmp/vhost-net0 --socket-file /tmp/vhost-net1 --dmas [txd0@00:04.0,txd1@00:04.1] + ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -- -p 0x1 --mergeable 1 --vm2vm 1 --dma-type ioat \ + --socket-file /tmp/vhost-net0 --socket-file /tmp/vhost-net1 --dmas [txd0@0000:00:04.0,txd1@0000:00:04.1] --client 3. Launch virtio-user:: - ./dpdk-testpmd -l 29-30 -n 4 --no-pci --file-prefix=testpmd0 \ - --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost-net0,queues=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 + ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 29-30 -n 4 --no-pci --file-prefix=testpmd0 \ + --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/tmp/vhost-net0,queues=1,server=1,mrg_rxbuf=1,in_order=1,vectorized=1,packed_vq=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 - ./dpdk-testpmd -l 31-32 -n 4 --no-pci --file-prefix=testpmd1 \ - --vdev=net_virtio_user0,mac=00:11:22:33:44:11,path=/tmp/vhost-net0,queues=1,mrg_rxbuf=0,in_order=1,vectorized=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 + ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 31-32 -n 4 --no-pci --file-prefix=testpmd1 \ + --vdev=net_virtio_user0,mac=00:11:22:33:44:11,path=/tmp/vhost-net1,queues=1,server=1,mrg_rxbuf=1,in_order=1,vectorized=1 -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1 -4. Start pkts from two virtio-user sides, record performance number with txpkts=256 and 2000 from testpmd1 seperately:: +4. Loop pkts between two virtio-user sides, record performance number with 64b/2000b/8000b/IMIX pkts can get expected:: testpmd0>set fwd mac testpmd0>start tx_first testpmd0>stop testpmd0>set eth-peer 0 00:11:22:33:44:11 testpmd0>start - testpmd1>set fwd mac testpmd1>set eth-peer 0 00:11:22:33:44:10 + testpmd1>set txpkts 64 + testpmd1>start tx_first + testpmd1>show port stats all + testpmd1>stop testpmd1>set txpkts 2000 testpmd1>start tx_first testpmd1>show port stats all testpmd1>stop - testpmd1>set txpkts 256 + testpmd1>set txpkts 2000,2000,2000,2000 + testpmd1>start tx_first + testpmd1>show port stats all + testpmd1>stop + testpmd1>set txpkts 64,256,2000,64,256,2000 testpmd1>start tx_first testpmd1>show port stats all -5. Re-test step 2-4 except to launch dpdk-vhost by below command, record perfromance:: - - ./dpdk-vhost -l 27-28 -n 4 -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /tmp/vhost-net0 --socket-file /tmp/vhost-net1 +5. Stop dpdk-vhost side and relaunch it with same cmd as step2. -6. Compare perfromance number, when txpkts=2000, first test has better performance; while txpkts=256, the second test has better performance. +6. Rerun step 4. -Test Case4: VM2VM test with 2 vhost device using vhost async driver -======================================================================= +Test Case4: VM2VM test with cbdma channels register/unregister stable check +============================================================================ 1. Bind one physical ports to vfio-pci and two CBDMA channels to igb_uio. 2. On host, launch dpdk-vhost by below command:: - ./dpdk-vhost -l 26-28 -n 4 -- -p 0x1 --mergeable 1 --vm2vm 1 --dma-type ioat \ - --socket-file /tmp/vhost-net0 --socket-file /tmp/vhost-net1 --dmas [txd0@00:04.0,txd1@00:04.1] + ./dpdk-vhost -l 26-28 -n 4 -- -p 0x1 --mergeable 1 --vm2vm 1 --dma-type ioat \ + --socket-file /tmp/vhost-net0 --socket-file /tmp/vhost-net1 --dmas [txd0@0000:00:04.0,txd1@0000:00:04.1] --client -3. Start VM0:: +3. Start VM0 with qemu-5.2.0:: - /home/qemu-install/qemu-4.2.1/bin/qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ + qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \ -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img \ -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \ -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \ -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \ - -chardev socket,id=char0,path=/tmp/vhost-net0 \ + -chardev socket,id=char0,path=/tmp/vhost-net0,server \ -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \ - -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=true,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -vnc :10 + -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=true,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -vnc :10 -4. Start VM1:: +4. Start VM1 with qemu-5.2.0:: - /home/qemu-install/qemu-4.2.1/bin/qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 4 -m 4096 \ + qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 4 -m 4096 \ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \ -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04-2.img \ -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \ -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \ -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \ - -netdev user,id=nttsip1,hostfwd=tcp:10.67.119.61:6003-:22 \ - -chardev socket,id=char0,path=/tmp/vhost-net1 \ + -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \ + -chardev socket,id=char0,path=/tmp/vhost-net1,server \ -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \ - -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=true,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -vnc :12 + -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=true,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -vnc :12 5. Bind virtio port to vfio-pci in both two VMs:: @@ -219,30 +221,153 @@ Test Case4: VM2VM test with 2 vhost device using vhost async driver ./dpdk-testpmd -l 1-2 -n 4 -- -i --rxq=1 --txq=1 --nb-cores=1 --txd=1024 --rxd=1024 -7. Start pkts from two virtio-pmd, record performance number with txpkts=256 and 2000 from testpmd1 seperately:: +7. Loop pkts between two virtio-user sides, record performance number with 64b/2000b/8000b/IMIX pkts can get expected:: testpmd0>set fwd mac testpmd0>start tx_first testpmd0>stop testpmd0>set eth-peer 0 52:54:00:00:00:02 testpmd0>start - testpmd1>set fwd mac testpmd1>set eth-peer 0 52:54:00:00:00:01 + testpmd1>set txpkts 64 + testpmd1>start tx_first + testpmd1>show port stats all + testpmd1>stop testpmd1>set txpkts 2000 testpmd1>start tx_first testpmd1>show port stats all testpmd1>stop - testpmd1>set txpkts 256 + testpmd1>set txpkts 2000,2000,2000,2000 + testpmd1>start tx_first + testpmd1>show port stats all + testpmd1>stop + testpmd1>set txpkts 64,256,2000,64,256,2000 testpmd1>start tx_first testpmd1>show port stats all -8. Inject traffic with VLAN_id=1000 and dest_mac=virtio_mac_addresss (52:54:00:00:00:02 and 52:54:00:00:00:02) to NIC using packet generator, check two virtio-pmd can receive according packets. - -9. Quit two testpmd in two VMs, bind virtio-pmd port to virtio-pci,then bind port back to vfio-pci, rerun below cmd 50 times:: +8. Quit two testpmd in two VMs, bind virtio-pmd port to virtio-pci,then bind port back to vfio-pci, rerun below cmd 50 times:: ./usertools/dpdk-devbind.py -u 00:05.0 ./usertools/dpdk-devbind.py --bind=virtio-pci 00:05.0 ./usertools/dpdk-devbind.py --bind=vfio-pci 00:05.0 -10. Rerun step 6-8，check vhost can stable work. +9. Restart vhost, then rerun step 7，check vhost can stable work and get expected throughput. + +Test Case5: VM2VM split ring test with iperf and reconnect stable check +======================================================================= + +1. Bind one physical ports to vfio-pci and two CBDMA channels to igb_uio. + +2. On host, launch dpdk-vhost by below command:: + + ./dpdk-vhost -l 26-28 -n 4 -- -p 0x1 --mergeable 1 --vm2vm 1 --dma-type ioat \ + --socket-file /tmp/vhost-net0 --socket-file /tmp/vhost-net1 --dmas [txd0@0000:00:04.0,txd1@0000:00:04.1] --client + +3. Start VM0 with qemu-5.2.0:: + + qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ + -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \ + -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img \ + -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \ + -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \ + -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \ + -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \ + -chardev socket,id=char0,path=/tmp/vhost-net0,server \ + -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \ + -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=true,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -vnc :10 + +4. Start VM1 with qemu-5.2.0:: + + qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 4 -m 4096 \ + -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \ + -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04-2.img \ + -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \ + -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \ + -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \ + -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \ + -chardev socket,id=char0,path=/tmp/vhost-net1,server \ + -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \ + -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=true,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -vnc :12 + +5. On VM1, set virtio device IP and run arp protocal:: + + ifconfig ens5 1.1.1.2 + arp -s 1.1.1.8 52:54:00:00:00:02 + +6. On VM2, set virtio device IP and run arp protocal:: + + ifconfig ens5 1.1.1.8 + arp -s 1.1.1.2 52:54:00:00:00:01 + +7. Check the iperf performance between two VMs by below commands:: + + Under VM1, run: `iperf -s -i 1` + Under VM2, run: `iperf -c 1.1.1.2 -i 1 -t 60` + +8. Check iperf throughput can get x Gbits/sec. + +9. Scp 1MB file form VM1 to VM2M, check packets can be forwarding success by scp:: + + Under VM1, run: `scp [xxx] root@1.1.1.8:/` [xxx] is the file name + +10. Relaunch vhost-dpdk, then rerun step 7-9 five times. + +Test Case6: VM2VM packed ring test with iperf and reconnect stable test +======================================================================= + +1. Bind one physical ports to vfio-pci and two CBDMA channels to igb_uio. + +2. On host, launch dpdk-vhost by below command:: + + ./dpdk-vhost -l 26-28 -n 4 -- -p 0x1 --mergeable 1 --vm2vm 1 --dma-type ioat \ + --socket-file /tmp/vhost-net0 --socket-file /tmp/vhost-net1 --dmas [txd0@0000:00:04.0,txd1@0000:00:04.1] + +3. Start VM0 with qemu-5.2.0:: + + qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ + -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \ + -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img \ + -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \ + -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \ + -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \ + -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \ + -chardev socket,id=char0,path=/tmp/vhost-net0 \ + -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \ + -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=true,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -vnc :10 + +4. Start VM1 with qemu-5.2.0:: + + qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 4 -m 4096 \ + -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \ + -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04-2.img \ + -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \ + -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \ + -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \ + -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \ + -chardev socket,id=char0,path=/tmp/vhost-net1 \ + -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \ + -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=true,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -vnc :12 + +5. On VM1, set virtio device IP and run arp protocal:: + + ifconfig ens5 1.1.1.2 + arp -s 1.1.1.8 52:54:00:00:00:02 + +6. On VM2, set virtio device IP and run arp protocal:: + + ifconfig ens5 1.1.1.8 + arp -s 1.1.1.2 52:54:00:00:00:01 + +7. Check the iperf performance between two VMs by below commands:: + + Under VM1, run: `iperf -s -i 1` + Under VM2, run: `iperf -c 1.1.1.2 -i 1 -t 60` + +8. Check iperf throughput can get x Gbits/sec. + +9. Scp 1MB file form VM1 to VM2M, check packets can be forwarding success by scp:: + + Under VM1, run: `scp [xxx] root@1.1.1.8:/` [xxx] is the file name + +10. Rerun step 7-9 five times. \ No newline at end of file