diff mbox series

[V2,1/2] test_plans/vm2vm_virtio_pmd_cbdma_test_plan: modify the dmas parameter by DPDK changed

Message ID 20221122082544.2896525-1-weix.ling@intel.com (mailing list archive)
State Superseded
Headers show
Series modify the dmas parameter by DPDK changed | expand

Commit Message

Wei Ling Nov. 22, 2022, 8:25 a.m. UTC
From DPDK-22.11, the dmas parameter have been changed by the local patch,
so modify the dmas parameter in the testplan.

Signed-off-by: Wei Ling <weix.ling@intel.com>
---
 .../vm2vm_virtio_pmd_cbdma_test_plan.rst      | 574 +++++++++---------
 1 file changed, 292 insertions(+), 282 deletions(-)
diff mbox series

Patch

diff --git a/test_plans/vm2vm_virtio_pmd_cbdma_test_plan.rst b/test_plans/vm2vm_virtio_pmd_cbdma_test_plan.rst
index 480f02b9..74dda7bb 100644
--- a/test_plans/vm2vm_virtio_pmd_cbdma_test_plan.rst
+++ b/test_plans/vm2vm_virtio_pmd_cbdma_test_plan.rst
@@ -17,12 +17,23 @@  This document provides the test plan for testing some basic functions with CBDMA
 1. vm2vm mergeable, non-mergebale path test with virtio 1.0 and virtio1.1 and check virtio-pmd tx chain packets in mergeable path.
 2. dynamic change queue number.
 
-..Note:
-1.For packed virtqueue virtio-net test, need qemu version > 4.2.0 and VM kernel version > 5.1, and packed ring multi-queues not support reconnect in qemu yet.
-2.For split virtqueue virtio-net with multi-queues server mode test, better to use qemu version >= 5.2.0, dut to qemu(v4.2.0~v5.1.0) exist split ring multi-queues reconnection issue.
-3.When DMA devices are bound to vfio driver, VA mode is the default and recommended. For PA mode, page by page mapping may
-exceed IOMMU's max capability, better to use 1G guest hugepage.
-4.DPDK local patch that about vhost pmd is needed when testing Vhost asynchronous data path with testpmd.
+.. note::
+
+   1.For packed virtqueue virtio-net test, need qemu version > 4.2.0 and VM kernel version > 5.1, and packed ring multi-queues not support reconnect in qemu yet.
+   2.For split virtqueue virtio-net with multi-queues server mode test, better to use qemu version >= 5.2.0, dut to qemu(v4.2.0~v5.1.0) exist split ring multi-queues reconnection issue.
+   3.When DMA devices are bound to vfio driver, VA mode is the default and recommended. For PA mode, page by page mapping may
+   exceed IOMMU's max capability, better to use 1G guest hugepage.
+   4.DPDK local patch that about vhost pmd is needed when testing Vhost asynchronous data path with testpmd. In this patch,
+   we enable asynchronous data path for vhostpmd. Asynchronous data path is enabled per tx/rx queue, and users need to specify
+   the DMA device used by the tx/rx queue. Each tx/rx queue only supports to use one DMA device (This is limited by the
+   implementation of vhostpmd), but one DMA device can be shared among multiple tx/rx queues of different vhost PMD ports.
+
+Two PMD parameters are added:
+- dmas:	specify the used DMA device for a tx/rx queue.(Default: no queues enable asynchronous data path)
+- dma-ring-size: DMA ring size.(Default: 4096).
+
+Here is an example:
+--vdev 'eth_vhost0,iface=./s0,dmas=[txq0@0000:00.01.0;rxq0@0000:00.01.1],dma-ring-size=2048'
 
 For more about dpdk-testpmd sample, please refer to the DPDK docments:
 https://doc.dpdk.org/guides/testpmd_app_ug/run_app.html
@@ -37,30 +48,30 @@  Topology
 
 Software
 --------
-    Trex:http://trex-tgn.cisco.com/trex/release/v2.26.tar.gz
+    qemu: https://download.qemu.org/qemu-7.1.0.tar.xz
 
 General set up
 --------------
 1. Compile DPDK::
 
-    # CC=gcc meson --werror -Denable_kmods=True -Dlibdir=lib -Dexamples=all --default-library=static <dpdk build dir>
-    # ninja -C <dpdk build dir> -j 110
-    For example:
-    CC=gcc meson --werror -Denable_kmods=True -Dlibdir=lib -Dexamples=all --default-library=static x86_64-native-linuxapp-gcc
-    ninja -C x86_64-native-linuxapp-gcc -j 110
+       # CC=gcc meson --werror -Denable_kmods=True -Dlibdir=lib -Dexamples=all --default-library=static <dpdk build dir>
+       # ninja -C <dpdk build dir> -j 110
+       For example:
+       CC=gcc meson --werror -Denable_kmods=True -Dlibdir=lib -Dexamples=all --default-library=static x86_64-native-linuxapp-gcc
+       ninja -C x86_64-native-linuxapp-gcc -j 110
 
 2. Get the PCI device ID and DMA device ID of DUT, for example, 0000:18:00.0 is PCI device ID, 0000:00:04.0, 0000:00:04.1 is DMA device ID::
 
-    <dpdk dir># ./usertools/dpdk-devbind.py -s
+       <dpdk dir># ./usertools/dpdk-devbind.py -s
 
-    Network devices using kernel driver
-    ===================================
-    0000:18:00.0 'Device 159b' if=ens785f0 drv=ice unused=vfio-pci
+       Network devices using kernel driver
+       ===================================
+       0000:18:00.0 'Device 159b' if=ens785f0 drv=ice unused=vfio-pci
 
-    DMA devices using kernel driver
-    ===============================
-    0000:00:04.0 'Sky Lake-E CBDMA Registers 2021' drv=ioatdma unused=vfio-pci
-    0000:00:04.1 'Sky Lake-E CBDMA Registers 2021' drv=ioatdma unused=vfio-pci
+       DMA devices using kernel driver
+       ===============================
+       0000:00:04.0 'Sky Lake-E CBDMA Registers 2021' drv=ioatdma unused=vfio-pci
+       0000:00:04.1 'Sky Lake-E CBDMA Registers 2021' drv=ioatdma unused=vfio-pci
 
 Test case
 =========
@@ -69,205 +80,202 @@  Common steps
 ------------
 1. Bind 1 NIC port and CBDMA channels to vfio-pci::
 
-    <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci <DUT port pci device id>
-    <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci <DUT port DMA device id>
+      <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci <DUT port pci device id>
+      <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci <DUT port DMA device id>
 
-    For example, Bind 1 NIC port and 2 CBDMA channels::
-    <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 0000:00:18.0
-    <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 0000:00:04.0 0000:00:04.1
+      For example, Bind 1 NIC port and 2 CBDMA channels::
+      <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 0000:00:18.0
+      <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 0000:00:04.0 0000:00:04.1
 
 2. On VM1 and VM2, bind virtio device(for example,0000:00:05.0) with vfio-pci driver::
 
-    modprobe vfio
-    modprobe vfio-pci
-    echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
-    <dpdk dir># ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
+      modprobe vfio
+      modprobe vfio-pci
+      echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+      <dpdk dir># ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
 
 Test Case 1: VM2VM virtio-pmd split ring mergeable path dynamic queue size with cbdma enable and server mode
 ------------------------------------------------------------------------------------------------------------
-This case tests split ring mergeable path in VM2VM vhost-user/virtio-pmd topology when vhost uses the asynchronous operations with CBDMA channels,
-check that it can work normally after dynamically changing queue number, reconnection has also been tested.
+This case tests split ring mergeable path in VM2VM vhost-user/virtio-pmd topology when vhost uses the asynchronous
+operations with CBDMA channels, check that it can work normally after dynamically changing queue number, reconnection has also been tested.
 
 1. Bind 16 CBDMA channels to vfio-pci, as common step 1.
 
 2. Launch the testpmd with 2 vhost ports below commands::
 
-    <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-5 -n 4 --file-prefix=vhost \
-    -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \
-    -a 0000:80:04.0 -a 0000:80:04.1 -a 0000:80:04.2 -a 0000:80:04.3 -a 0000:80:04.4 -a 0000:80:04.5 -a 0000:80:04.6 -a 0000:80:04.7 \
-    --vdev 'net_vhost0,iface=vhost-net0,client=1,queues=8,dmas=[txq0;txq1;txq2;txq3;rxq0;rxq1;rxq2;rxq3]' \
-    --vdev 'net_vhost1,iface=vhost-net1,client=1,queues=8,dmas=[txq0;txq1;txq2;txq3;rxq0;rxq1;rxq2;rxq3]' \
-    -- -i --nb-cores=4 --txd=1024 --rxd=1024 --rxq=4 --txq=4 \
-    --lcore-dma=[lcore2@0000:00:04.0,lcore2@0000:00:04.1,lcore2@0000:00:04.2,lcore2@0000:00:04.3,lcore3@0000:00:04.4,lcore3@0000:00:04.5,lcore3@0000:00:04.6,lcore3@0000:00:04.7,lcore4@0000:80:04.0,lcore4@0000:80:04.1,lcore4@0000:80:04.2,lcore4@0000:80:04.3,lcore5@0000:80:04.4,lcore5@0000:80:04.5,lcore5@0000:80:04.6,lcore5@0000:80:04.7]
-    testpmd> start
+      <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-5 -n 4 --file-prefix=vhost \
+      -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 \
+      --vdev 'net_vhost0,iface=vhost-net0,client=1,queues=8,dmas=[txq0@0000:00:04.0;txq1@0000:00:04.0;txq2@0000:00:04.1;txq3@0000:00:04.1;rxq0@0000:00:04.0;rxq1@0000:00:04.0;rxq2@0000:00:04.1;rxq3@0000:00:04.1]' \
+      --vdev 'net_vhost1,iface=vhost-net1,client=1,queues=8,dmas=[txq0@0000:00:04.2;txq1@0000:00:04.2;txq2@0000:00:04.3;txq3@0000:00:04.3;rxq0@0000:00:04.2;rxq1@0000:00:04.2;rxq2@0000:00:04.3;rxq3@0000:00:04.3]' \
+      -- -i --nb-cores=4 --txd=1024 --rxd=1024 --rxq=4 --txq=4
+      testpmd> start
 
 3. Launch VM1 and VM2 using qemu::
 
-    taskset -c 6-16 qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 9 -m 4096 \
-    -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
-    -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img  \
-    -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 -device virtio-serial \
-    -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.2 -daemonize \
-    -monitor unix:/tmp/vm1_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
-    -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \
-    -chardev socket,id=char0,path=./vhost-net0,server \
-    -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
-    -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=on,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on -vnc :10
-
-    taskset -c 17-27 qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 9 -m 4096 \
-    -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
-    -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04-2.img  \
-    -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \
-    -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \
-    -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
-    -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \
-    -chardev socket,id=char0,path=./vhost-net1,server \
-    -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
-    -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=on,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on -vnc :12
+      taskset -c 6-16 qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 9 -m 4096 \
+      -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
+      -n uma node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu22-04.img  \
+      -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 -device virtio-serial \
+      -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.2 -daemonize \
+      -monitor unix:/tmp/vm1_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
+      -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \
+      -chardev socket,id=char0,path=./vhost-net0,server \
+      -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
+      -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=on,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on -vnc :10
+
+      taskset -c 17-27 qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 9 -m 4096 \
+      -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
+      -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu22-04-2.img  \
+      -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \
+      -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \
+      -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
+      -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \
+      -chardev socket,id=char0,path=./vhost-net1,server \
+      -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
+      -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=on,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on -vnc :12
 
 4. On VM1 and VM2, bind virtio device with vfio-pci driver::
 
-    modprobe vfio
-    modprobe vfio-pci
-    echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
-    ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
+      modprobe vfio
+      modprobe vfio-pci
+      echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+      ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
 
 5. Launch testpmd in VM1::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024 --max-pkt-len=9600 --rx-offloads=0x00002000
-    testpmd> set fwd mac
-    testpmd> start
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024 --max-pkt-len=9600 --rx-offloads=0x00002000
+      testpmd> set fwd mac
+      testpmd> start
 
-6. Launch testpmd in VM2 and send imix pkts, check imix packets can looped between two VMs for 1 mins and 4 queues (queue0 to queue3) have packets rx/tx::
+6. Launch testpmd in VM2 and send imix packets, check imix packets can loop between two VMs for 1 mins and 4 queues (queue0 to queue3) have packets rx/tx::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024 --max-pkt-len=9600 --rx-offloads=0x00002000
-    testpmd> set fwd mac
-    testpmd> set txpkts 64,256,512,1024,2000,64,256,512,1024,2000
-    testpmd> start tx_first 32
-    testpmd> show port stats all
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024 --max-pkt-len=9600 --rx-offloads=0x00002000
+      testpmd> set fwd mac
+      testpmd> set txpkts 64,256,512,1024,2000,64,256,512,1024,2000
+      testpmd> start tx_first 32
+      testpmd> show port stats all
 
 7. Check vhost use the asynchronous data path(funtion like virtio_dev_rx_async_xxx/virtio_dev_tx_async_xxx)::
 
-    perf top
+      perf top
 
 8. On host, dynamic change queue numbers::
 
-    testpmd> stop
-    testpmd> port stop all
-    testpmd> port config all rxq 8
-    testpmd> port config all txq 8
-    testpmd> port start all
-    testpmd> start
+      testpmd> stop
+      testpmd> port stop all
+      testpmd> port config all rxq 8
+      testpmd> port config all txq 8
+      testpmd> port start all
+      testpmd> start
 
 9. Send packets by testpmd in VM2::
 
-    testpmd> stop
-    testpmd> start tx_first 32
-    testpmd> show port stats all
+      testpmd> stop
+      testpmd> start tx_first 32
+      testpmd> show port stats all
 
-10. Check vhost testpmd RX/TX can work normally, packets can looped between two VMs and both 8 queues can RX/TX traffic.
+10. Check vhost testpmd RX/TX can work normally, packets can loop between two VMs and both 8 queues can RX/TX traffic.
 
 11. Rerun step 7.
 
 12. Relaunch and start vhost side testpmd with 8 queues::
 
-	./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-5 -n 4 --file-prefix=vhost \
-	-a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \
-	-a 0000:80:04.0 -a 0000:80:04.1 -a 0000:80:04.2 -a 0000:80:04.3 -a 0000:80:04.4 -a 0000:80:04.5 -a 0000:80:04.6 -a 0000:80:04.7 \
-	--vdev 'net_vhost0,iface=vhost-net0,client=1,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;rxq2;rxq3;rxq4;rxq5;rxq6;rxq7]' \
-	--vdev 'net_vhost1,iface=vhost-net1,client=1,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;rxq2;rxq3;rxq4;rxq5;rxq6;rxq7]' \
-	-- -i --nb-cores=4 --txd=1024 --rxd=1024 --rxq=8 --txq=8 \
-	--lcore-dma=[lcore2@0000:00:04.0,lcore2@0000:00:04.1,lcore2@0000:00:04.2,lcore2@0000:00:04.3,lcore3@0000:00:04.4,lcore3@0000:00:04.5,lcore3@0000:00:04.6,lcore3@0000:00:04.7,lcore4@0000:80:04.0,lcore4@0000:80:04.1,lcore4@0000:80:04.2,lcore4@0000:80:04.3,lcore5@0000:80:04.4,lcore5@0000:80:04.5,lcore5@0000:80:04.6,lcore5@0000:80:04.7]
-	testpmd> start
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-5 -n 4 --file-prefix=vhost \
+      -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \
+      -a 0000:80:04.0 -a 0000:80:04.1 -a 0000:80:04.2 -a 0000:80:04.3 -a 0000:80:04.4 -a 0000:80:04.5 -a 0000:80:04.6 -a 0000:80:04.7 \
+      --vdev 'net_vhost0,iface=vhost-net0,client=1,queues=8,dmas=[txq0@0000:00:04.0;txq1@0000:00:04.1;txq2@0000:00:04.2;txq3@0000:00:04.3;txq4@0000:00:04.4;txq5@0000:00:04.1;rxq2@0000:00:04.2;rxq3@0000:00:04.3;rxq4@0000:00:04.4;rxq5@0000:00:04.5;rxq6@0000:00:04.6;rxq7@0000:00:04.7]' \
+      --vdev 'net_vhost1,iface=vhost-net1,client=1,queues=8,dmas=[txq0@0000:80:04.0;txq1@0000:80:04.1;txq2@0000:80:04.2;txq3@0000:80:04.3;txq4@0000:80:04.4;txq5@0000:80:04.1;rxq2@0000:80:04.2;rxq3@0000:80:04.3;rxq4@0000:80:04.4;rxq5@0000:80:04.5;rxq6@0000:80:04.6;rxq7@0000:80:04.7]' \
+      -- -i --nb-cores=4 --txd=1024 --rxd=1024 --rxq=8 --txq=8
+      testpmd> set fwd mac
+      testpmd> start
 
-13. Send packets by testpmd in VM2, check imix packets can looped between two VMs for 1 mins and 8 queues all have packets rx/tx::
+13. Send packets by testpmd in VM2, check imix packets can loop between two VMs for 1 mins and 8 queues all have packets rx/tx::
 
-	testpmd> stop
-	testpmd> start tx_first 32
-	testpmd> show port stats all
+      testpmd> stop
+      testpmd> start tx_first 32
+      testpmd> show port stats all
 
-14. Rerun step 12-13 for 5 times.
+14. Rerun step 12-13 for 3 times.
 
 Test Case 2: VM2VM virtio-pmd split ring non-mergeable path dynamic queue size with cbdma enable and server mode
 ----------------------------------------------------------------------------------------------------------------
-This case tests split ring non-mergeable path in VM2VM vhost-user/virtio-pmd topology when vhost uses the asynchronous operations with CBDMA channels,
-check that it can work normally after dynamically changing queue number, reconnection has also been tested.
+This case tests split ring non-mergeable path in VM2VM vhost-user/virtio-pmd topology when vhost uses the asynchronous
+operations with CBDMA channels, check that it can work normally after dynamically changing queue number, reconnection has also been tested.
 
 1. Bind 16 CBDMA channels to vfio-pci, as common step 1.
 
 2. Launch the testpmd with 2 vhost ports below commands::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-5 -n 4 --file-prefix=vhost -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \
-    -a 0000:80:04.0 -a 0000:80:04.1 -a 0000:80:04.2 -a 0000:80:04.3 -a 0000:80:04.4 -a 0000:80:04.5 -a 0000:80:04.6 -a 0000:80:04.7 \
-    --vdev 'net_vhost0,iface=vhost-net0,client=1,queues=8,dmas=[txq0;txq1;txq2;txq3;rxq0;rxq1;rxq2;rxq3]' \
-    --vdev 'net_vhost1,iface=vhost-net1,client=1,queues=8,dmas=[txq0;txq1;txq2;txq3;rxq0;rxq1;rxq2;rxq3]' \
-    -- -i --nb-cores=4 --txd=1024 --rxd=1024 --rxq=4 --txq=4 \
-    --lcore-dma=[lcore2@0000:00:04.0,lcore2@0000:00:04.1,lcore2@0000:00:04.2,lcore2@0000:00:04.3,lcore3@0000:00:04.4,lcore3@0000:00:04.5,lcore3@0000:00:04.6,lcore3@0000:00:04.7,lcore4@0000:80:04.0,lcore4@0000:80:04.1,lcore4@0000:80:04.2,lcore4@0000:80:04.3,lcore5@0000:80:04.4,lcore5@0000:80:04.5,lcore5@0000:80:04.6,lcore5@0000:80:04.7]
-    testpmd> start
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-5 -n 4 --file-prefix=vhost \
+      -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 \
+      --vdev 'net_vhost0,iface=vhost-net0,client=1,queues=8,dmas=[txq0@0000:00:04.0;txq1@0000:00:04.0;txq2@0000:00:04.1;txq3@0000:00:04.1;rxq0@0000:00:04.0;rxq1@0000:00:04.0;rxq2@0000:00:04.1;rxq3@0000:00:04.1]' \
+      --vdev 'net_vhost1,iface=vhost-net1,client=1,queues=8,dmas=[txq0@0000:00:04.0;txq1@0000:00:04.0;txq2@0000:00:04.1;txq3@0000:00:04.1;rxq0@0000:00:04.0;rxq1@0000:00:04.0;rxq2@0000:00:04.1;rxq3@0000:00:04.1]' \
+      -- -i --nb-cores=4 --txd=1024 --rxd=1024 --rxq=4 --txq=4
+      testpmd> start
 
 3. Launch VM1 and VM2 using qemu::
 
-    taskset -c 6-16 qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 9 -m 4096 \
-    -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
-    -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img  \
-    -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 -device virtio-serial \
-    -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.2 -daemonize \
-    -monitor unix:/tmp/vm1_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
-    -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \
-    -chardev socket,id=char0,path=./vhost-net0,server \
-    -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
-    -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=off,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on -vnc :10
-
-    taskset -c 17-27 qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 9 -m 4096 \
-    -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
-    -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04-2.img  \
-    -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \
-    -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \
-    -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
-    -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \
-    -chardev socket,id=char0,path=./vhost-net1,server \
-    -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
-    -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on -vnc :12
+      taskset -c 6-16 qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 9 -m 4096 \
+      -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
+      -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu22-04.img  \
+      -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 -device virtio-serial \
+      -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.2 -daemonize \
+      -monitor unix:/tmp/vm1_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
+      -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \
+      -chardev socket,id=char0,path=./vhost-net0,server \
+      -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
+      -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=off,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on -vnc :10
+
+      taskset -c 17-27 qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 9 -m 4096 \
+      -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
+      -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu22-04-2.img  \
+      -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \
+      -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \
+      -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
+      -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \
+      -chardev socket,id=char0,path=./vhost-net1,server \
+      -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
+      -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on -vnc :12
 
 4. On VM1 and VM2, bind virtio device with vfio-pci driver::
 
-    modprobe vfio
-    modprobe vfio-pci
-    echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
-    ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
+      modprobe vfio
+      modprobe vfio-pci
+      echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+      ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
 
 5. Launch testpmd in VM1::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024
-    testpmd> set fwd mac
-    testpmd> start
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024
+      testpmd> set fwd mac
+      testpmd> start
 
-6. Launch testpmd in VM2 and send imix pkts, check imix packets can looped between two VMs for 1 mins and 4 queues (queue0 to queue3) have packets rx/tx::
+6. Launch testpmd in VM2 and send imix pkts, check imix packets can loop between two VMs for 1 mins and 4 queues (queue0 to queue3) have packets rx/tx::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024
-    testpmd> set fwd mac
-    testpmd> set txpkts 64,256,512
-    testpmd> start tx_first 32
-    testpmd> show port stats all
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024
+      testpmd> set fwd mac
+      testpmd> set txpkts 64,256,512
+      testpmd> start tx_first 32
+      testpmd> show port stats all
 
 7. Check vhost use the asynchronous data path(funtion like virtio_dev_rx_async_xxx/virtio_dev_tx_async_xxx)::
 
-    perf top
+      perf top
 
 8. On VM1 and VM2, dynamic change queue numbers at virtio-pmd side from 8 queues to 4 queues::
 
-    testpmd> stop
-    testpmd> port stop all
-    testpmd> port config all rxq 4
-    testpmd> port config all txq 4
-    testpmd> port start all
-    testpmd> start
+      testpmd> stop
+      testpmd> port stop all
+      testpmd> port config all rxq 4
+      testpmd> port config all txq 4
+      testpmd> port start all
+      testpmd> start
 
-9. Send packets by testpmd in VM2, check Check virtio-pmd RX/TX can work normally and imix packets can looped between two VMs for 1 mins::
+9. Send packets by testpmd in VM2, check Check virtio-pmd RX/TX can work normally and imix packets can loop between two VMs for 1 mins::
 
-    testpmd> stop
-    testpmd> start tx_first 32
-    testpmd> show port stats all
+      testpmd> stop
+      testpmd> start tx_first 32
+      testpmd> show port stats all
 
 10. Rerun step 7.
 
@@ -275,205 +283,207 @@  check that it can work normally after dynamically changing queue number, reconne
 
 Test Case 3: VM2VM virtio-pmd packed ring mergeable path dynamic queue size with cbdma enable and server mode
 -------------------------------------------------------------------------------------------------------------
-This case tests packed ring mergeable path with virtio1.1 and server mode in VM2VM vhost-user/virtio-pmd topology when vhost uses the asynchronous operations with CBDMA channels,check that it can work normally after dynamically changing queue number.
+This case tests packed ring mergeable path with virtio1.1 and server mode in VM2VM vhost-user/virtio-pmd topology
+when vhost uses the asynchronous operations with CBDMA channels, check that it can work normally after dynamically changing queue number.
 
 1. Bind 16 CBDMA channels to vfio-pci, as common step 1.
 
 2. Launch the testpmd with 2 vhost ports below commands::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-5 -n 4 --file-prefix=vhost -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \
-    -a 0000:80:04.0 -a 0000:80:04.1 -a 0000:80:04.2 -a 0000:80:04.3 -a 0000:80:04.4 -a 0000:80:04.5 -a 0000:80:04.6 -a 0000:80:04.7 \
-    --vdev 'net_vhost0,iface=vhost-net0,client=1,queues=8,dmas=[txq0;txq1;txq2;txq3;rxq0;rxq1;rxq2;rxq3]' \
-    --vdev 'net_vhost1,iface=vhost-net1,client=1,queues=8,dmas=[txq0;txq1;txq2;txq3;rxq0;rxq1;rxq2;rxq3]' \
-    -- -i --nb-cores=4 --txd=1024 --rxd=1024 --rxq=4 --txq=4 \
-    --lcore-dma=[lcore2@0000:00:04.0,lcore2@0000:00:04.1,lcore2@0000:00:04.2,lcore2@0000:00:04.3,lcore3@0000:00:04.4,lcore3@0000:00:04.5,lcore3@0000:00:04.6,lcore3@0000:00:04.7,lcore4@0000:80:04.0,lcore4@0000:80:04.1,lcore4@0000:80:04.2,lcore4@0000:80:04.3,lcore5@0000:80:04.4,lcore5@0000:80:04.5,lcore5@0000:80:04.6,lcore5@0000:80:04.7]
-    testpmd> start
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-5 -n 4 --file-prefix=vhost \
+      -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 \
+      --vdev 'net_vhost0,iface=vhost-net0,client=1,queues=8,dmas=[txq0@0000:00:04.0;txq1@0000:00:04.0;txq2@0000:00:04.1;txq3@0000:00:04.1;rxq0@0000:00:04.0;rxq1@0000:00:04.0;rxq2@0000:00:04.1;rxq3@0000:00:04.1]' \
+      --vdev 'net_vhost1,iface=vhost-net1,client=1,queues=8,dmas=[txq0@0000:00:04.0;txq1@0000:00:04.0;txq2@0000:00:04.1;txq3@0000:00:04.1;rxq0@0000:00:04.0;rxq1@0000:00:04.0;rxq2@0000:00:04.1;rxq3@0000:00:04.1]' \
+      -- -i --nb-cores=4 --txd=1024 --rxd=1024 --rxq=4 --txq=4
+      testpmd> set fwd mac
+      testpmd> start
 
 3. Launch VM1 and VM2 using qemu::
 
-    taskset -c 6-16 qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 9 -m 4096 \
-    -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
-    -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img  \
-    -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 -device virtio-serial \
-    -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.2 -daemonize \
-    -monitor unix:/tmp/vm1_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
-    -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \
-    -chardev socket,id=char0,path=./vhost-net0,server \
-    -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
-    -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=on,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on,packed=on -vnc :10
-
-    taskset -c 17-27 qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 9 -m 4096 \
-    -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
-    -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04-2.img  \
-    -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \
-    -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \
-    -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
-    -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \
-    -chardev socket,id=char0,path=./vhost-net1,server \
-    -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
-    -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=on,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on,packed=on -vnc :12
+      taskset -c 6-16 qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 9 -m 4096 \
+      -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
+      -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu22-04.img  \
+      -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 -device virtio-serial \
+      -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.2 -daemonize \
+      -monitor unix:/tmp/vm1_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
+      -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \
+      -chardev socket,id=char0,path=./vhost-net0,server \
+      -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
+      -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=on,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on,packed=on -vnc :10
+
+      taskset -c 17-27 qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 9 -m 4096 \
+      -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
+      -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu22-04-2.img  \
+      -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \
+      -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \
+      -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
+      -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \
+      -chardev socket,id=char0,path=./vhost-net1,server \
+      -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
+      -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=on,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on,packed=on -vnc :12
 
 4. On VM1 and VM2, bind virtio device with vfio-pci driver::
 
-    modprobe vfio
-    modprobe vfio-pci
-    echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
-    ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
+      modprobe vfio
+      modprobe vfio-pci
+      echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+      ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
 
 5. Launch testpmd in VM1::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024 --max-pkt-len=9600 --rx-offloads=0x00002000
-    testpmd> set mac fwd
-    testpmd> start
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024 --max-pkt-len=9600 --rx-offloads=0x00002000
+      testpmd> set mac fwd
+      testpmd> start
 
-6. Launch testpmd in VM2 and send imix pkts, check imix packets can looped between two VMs for 1 mins and 4 queues (queue0 to queue3) have packets rx/tx::
+6. Launch testpmd in VM2 and send imix pkts, check imix packets can loop between two VMs for 1 mins and 4 queues (queue0 to queue3) have packets rx/tx::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024 --max-pkt-len=9600 --rx-offloads=0x00002000
-    testpmd> set mac fwd
-    testpmd> set txpkts 64,256,512,1024,2000,64,256,512,1024,2000
-    testpmd> start tx_first 32
-    testpmd> show port stats all
-    testpmd> stop
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=8 --rxq=8 --txd=1024 --rxd=1024 --max-pkt-len=9600 --rx-offloads=0x00002000
+      testpmd> set mac fwd
+      testpmd> set txpkts 64,256,512,1024,2000,64,256,512,1024,2000
+      testpmd> start tx_first 32
+      testpmd> show port stats all
+      testpmd> stop
 
 7. Quit VM2 and relaunch VM2 with split ring::
 
-    taskset -c 17-27 qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 9 -m 4096 \
-    -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
-    -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04-2.img  \
-    -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \
-    -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \
-    -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
-    -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \
-    -chardev socket,id=char0,path=./vhost-net1,server \
-    -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
-    -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on -vnc :12
+      taskset -c 17-27 qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 9 -m 4096 \
+      -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
+      -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu22-04-2.img  \
+      -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \
+      -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \
+      -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
+      -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \
+      -chardev socket,id=char0,path=./vhost-net1,server \
+      -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
+      -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on -vnc :12
 
-8. Bind virtio device with vfio-pci driver::
+8. On VM2, bind virtio device with vfio-pci driver::
 
-    modprobe vfio
-    modprobe vfio-pci
-    echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
-    <dpdk dir># ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
+      modprobe vfio
+      modprobe vfio-pci
+      echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+      <dpdk dir># ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
 
 9. Launch testpmd in VM2 and send imix pkts from VM2::
 
-	<dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip \
-	--txq=8 --rxq=8 --txd=1024 --rxd=1024 --max-pkt-len=9600 --rx-offloads=0x00002000
-	testpmd> set fwd mac
-	testpmd> set txpkts 64,256,512,1024,2000,64,256,512,1024,2000
+      <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip \
+      --txq=8 --rxq=8 --txd=1024 --rxd=1024 --max-pkt-len=9600 --rx-offloads=0x00002000
+      testpmd> set fwd mac
+      testpmd> set txpkts 64,256,512,1024,2000,64,256,512,1024,2000
 
-10. Check imix packets can looped between two VMs and 4 queues all have packets rx/tx::
+10. Check imix packets can loop between two VMs and 4 queues all have packets rx/tx::
 
-	testpmd> show port stats all
-	testpmd> stop
-	testpmd> start
+      testpmd> show port stats all
+      testpmd> stop
+      testpmd> start
 
 11. Check vhost use the asynchronous data path(funtion like virtio_dev_rx_async_xxx/virtio_dev_tx_async_xxx)::
 
-	perf top
+      perf top
 
 12. On host, dynamic change queue numbers::
 
-	testpmd> stop
-	testpmd> port stop all
-	testpmd> port config all rxq 8
-	testpmd> port config all txq 8
-	testpmd> port start all
-	testpmd> start
+      testpmd> stop
+      testpmd> port stop all
+      testpmd> port config all rxq 8
+      testpmd> port config all txq 8
+      testpmd> port start all
+      testpmd> start
 
 13. Send packets by testpmd in VM2::
 
-	testpmd> stop
-	testpmd> start tx_first 32
-	testpmd> show port stats all
+      testpmd> stop
+      testpmd> start tx_first 32
+      testpmd> show port stats all
 
-14. Check vhost testpmd RX/TX can work normally, packets can looped between two VMs and both 8 queues can RX/TX traffic.
+14. Check vhost testpmd RX/TX can work normally, packets can loop between two VMs and both 8 queues can RX/TX traffic.
 
 15. Rerun step 11.
 
 Test Case 4: VM2VM virtio-pmd packed ring non-mergeable path dynamic queue size with cbdma enable and server mode
 -----------------------------------------------------------------------------------------------------------------
-This case tests packed ring non-mergeable path with virtio1.1 and server mode in VM2VM vhost-user/virtio-pmd topology when vhost uses the asynchronous operations with CBDMA channels,check that it can work normally after dynamically changing queue number.
+This case tests packed ring non-mergeable path with virtio1.1 and server mode in VM2VM vhost-user/virtio-pmd topology
+when vhost uses the asynchronous operations with CBDMA channels,check that it can work normally after dynamically changing queue number.
 
 1. Bind 16 CBDMA channels to vfio-pci, as common step 1.
 
 2. Launch the testpmd with 2 vhost ports below commands::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-5 -n 4 --file-prefix=vhost -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \
-    -a 0000:80:04.0 -a 0000:80:04.1 -a 0000:80:04.2 -a 0000:80:04.3 -a 0000:80:04.4 -a 0000:80:04.5 -a 0000:80:04.6 -a 0000:80:04.7 \
-    --vdev 'net_vhost0,iface=vhost-net0,client=1,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;rxq2;rxq3;rxq4;rxq5;rxq6;rxq7]' \
-    --vdev 'net_vhost1,iface=vhost-net1,client=1,queues=8,dmas=[txq0;txq1;txq2;txq3;txq4;txq5;rxq2;rxq3;rxq4;rxq5;rxq6;rxq7]' \
-    -- -i --nb-cores=4 --txd=1024 --rxd=1024 --rxq=8 --txq=8 \
-    --lcore-dma=[lcore2@0000:00:04.0,lcore2@0000:00:04.1,lcore2@0000:00:04.2,lcore2@0000:00:04.3,lcore3@0000:00:04.4,lcore3@0000:00:04.5,lcore3@0000:00:04.6,lcore3@0000:00:04.7,lcore4@0000:80:04.0,lcore4@0000:80:04.1,lcore4@0000:80:04.2,lcore4@0000:80:04.3,lcore5@0000:80:04.4,lcore5@0000:80:04.5,lcore5@0000:80:04.6,lcore5@0000:80:04.7]
-    testpmd> start
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-5 -n 4 --file-prefix=vhost \
+      -a 0000:00:04.0 -a 0000:00:04.1 -a 0000:00:04.2 -a 0000:00:04.3 -a 0000:00:04.4 -a 0000:00:04.5 -a 0000:00:04.6 -a 0000:00:04.7 \
+      -a 0000:80:04.0 -a 0000:80:04.1 -a 0000:80:04.2 -a 0000:80:04.3 -a 0000:80:04.4 -a 0000:80:04.5 -a 0000:80:04.6 -a 0000:80:04.7 \
+      --vdev 'net_vhost0,iface=vhost-net0,client=1,queues=8,dmas=[txq0@0000:00:04.0;txq1@0000:00:04.1;txq2@0000:00:04.2;txq3@0000:00:04.3;txq4@0000:00:04.4;txq5@0000:00:04.1;rxq2@0000:00:04.2;rxq3@0000:00:04.3;rxq4@0000:00:04.4;rxq5@0000:00:04.5;rxq6@0000:00:04.6;rxq7@0000:00:04.7]' \
+      --vdev 'net_vhost1,iface=vhost-net1,client=1,queues=8,dmas=[txq0@0000:80:04.0;txq1@0000:80:04.1;txq2@0000:80:04.2;txq3@0000:80:04.3;txq4@0000:80:04.4;txq5@0000:80:04.1;rxq2@0000:80:04.2;rxq3@0000:80:04.3;rxq4@0000:80:04.4;rxq5@0000:80:04.5;rxq6@0000:80:04.6;rxq7@0000:80:04.7]' \
+      -- -i --nb-cores=4 --txd=1024 --rxd=1024 --rxq=8 --txq=8
+      testpmd> start
 
 3. Launch VM1 and VM2 using qemu::
 
-    taskset -c 6-16 qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 9 -m 4096 \
-    -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
-    -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img  \
-    -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 -device virtio-serial \
-    -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.2 -daemonize \
-    -monitor unix:/tmp/vm1_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
-    -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \
-    -chardev socket,id=char0,path=./vhost-net0,server \
-    -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
-    -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=off,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on,packed=on -vnc :10
-
-    taskset -c 17-27 qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 9 -m 4096 \
-    -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
-    -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04-2.img  \
-    -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \
-    -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \
-    -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
-    -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \
-    -chardev socket,id=char0,path=./vhost-net1,server \
-    -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
-    -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on,packed=on -vnc :12
+      taskset -c 6-16 qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 9 -m 4096 \
+      -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
+      -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu22-04.img  \
+      -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 -device virtio-serial \
+      -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.2 -daemonize \
+      -monitor unix:/tmp/vm1_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
+      -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \
+      -chardev socket,id=char0,path=./vhost-net0,server \
+      -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
+      -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=off,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on,packed=on -vnc :10
+
+      taskset -c 17-27 qemu-system-x86_64 -name vm2 -enable-kvm -cpu host -smp 9 -m 4096 \
+      -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \
+      -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu22-04-2.img  \
+      -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \
+      -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \
+      -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \
+      -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6003-:22 \
+      -chardev socket,id=char0,path=./vhost-net1,server \
+      -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=8 \
+      -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,mq=on,vectors=40,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,guest_ufo=on,host_ufo=on,packed=on -vnc :12
 
 4. On VM1 and VM2, bind virtio device with vfio-pci driver::
 
-    modprobe vfio
-    modprobe vfio-pci
-    echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
-    ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
+      modprobe vfio
+      modprobe vfio-pci
+      echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+      ./usertools/dpdk-devbind.py --force --bind=vfio-pci 0000:00:05.0
 
 5. Launch testpmd in VM1::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=4 --rxq=4 --txd=1024 --rxd=1024
-    testpmd> set mac fwd
-    testpmd> start
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=4 --rxq=4 --txd=1024 --rxd=1024
+      testpmd> set mac fwd
+      testpmd> start
 
-6. Launch testpmd in VM2 and send imix pkts, check imix packets can looped between two VMs for 1 mins::
+6. Launch testpmd in VM2 and send imix pkts, check imix packets can loop between two VMs for 1 mins::
 
-    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=4 --rxq=4 --txd=1024 --rxd=1024
-    testpmd> set mac fwd
-    testpmd> set txpkts 64,256,512
-    testpmd> start tx_first 32
-    testpmd> show port stats all
+      ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0x3 -n 4 -- -i --tx-offloads=0x00 --enable-hw-vlan-strip --txq=4 --rxq=4 --txd=1024 --rxd=1024
+      testpmd> set mac fwd
+      testpmd> set txpkts 64,256,512
+      testpmd> start tx_first 32
+      testpmd> show port stats all
 
 7. Check vhost use the asynchronous data path(funtion like virtio_dev_rx_async_xxx/virtio_dev_tx_async_xxx)::
 
-    perf top
+      perf top
 
 8. On VM2, stop the testpmd, check that both 4 queues have packets rx/tx::
 
-    testpmd> stop
+      testpmd> stop
 
 9. On VM1 and VM2, dynamic change queue numbers at virtio-pmd side from 4 queues to 8 queues::
 
-    testpmd> stop
-    testpmd> port stop all
-    testpmd> port config all rxq 8
-    testpmd> port config all txq 8
-    testpmd> port start all
-    testpmd> start
+      testpmd> stop
+      testpmd> port stop all
+      testpmd> port config all rxq 8
+      testpmd> port config all txq 8
+      testpmd> port start all
+      testpmd> start
 
-10. Send packets by testpmd in VM2, check Check virtio-pmd RX/TX can work normally and imix packets can looped between two VMs for 1 mins::
+10. Send packets by testpmd in VM2, check Check virtio-pmd RX/TX can work normally and imix packets can loop between two VMs for 1 mins::
 
-	testpmd> stop
-	testpmd> start tx_first 32
-	testpmd> show port stats all
+      testpmd> stop
+      testpmd> start tx_first 32
+      testpmd> show port stats all
 
 11. Rerun step 7.