[v2,11/12] net/mvpp2: document MTR and TM usage

diff --git a/doc/guides/nics/img/mvpp2_tm.png b/doc/guides/nics/img/mvpp2_tm.png
new file mode 100644
index 0000000000000000000000000000000000000000..3c709bab43950c39578a6726ab570008bc722e2f
GIT binary patch
literal 5355
zcmeHL=U-FX(%vBudXXkgk$yk~Q3z53C{4tG2UHXUA}B>dITY!TXatcaBA^CDq^Ssr
zCm^CUrK6#X3B82Sh0sfYyYXJ{?|ttda6jD-N!EJy+B5T;HS^3&;x1U3^KwaW0RX^j
zan{5J03dj9oy`FQzty7x?ZF>NfQ|VXprlP|8r-nDpGKYrfOpB<%<F96{)p>Y8zcaP
zDFOfn2LM|jgqZ?>pyL2Ag8~5ECjfBJFZ1J{`T!v8X<>5uVu;gxmWHpCwopu(RsOlO
zTRdWBT=}-w#kM&8XB8m>H1-e8(tgE_My<kX7hxrLE-DX-`6+*}Q7JkP98}Xj94Uq!
zD-X6@Iz^jfX!t#}Zo&_aS1udq4OD#(?$$iCm@F<93ftY?UYL%RjykomJ{)cSJf<Xj
zXKZY21l;VE%vuru>K7NFETZ_#dTX7dceJvyv!n4WfgP@VK0ZDLsi{Kr==N21k2U?z
zh;Odm{jBgv8E*7eTXazv8@$#ngcrb9yRn0SArZn-XZ+&Qf!{!Mk#1cFrf*RKfJc(~
zC@WdfC@5=8tWlBdZ(#3oP)rV_Y*Y(CZ#_FioYvc`g>mrkV=tTj8z`RPF^1l%^8zR<
zT6(DN;9V@QfY@P{;s^f*9KMUkz<OeS0K{pp+qDG?OsM3M<NB=jB9Pxe`J0nGysvl(
z5LEY%ISg`mlO1k^3gIYKVfhXCUUC<cxFG2Stt}u>A^7TLel(_bQNZI8>u+HAP1|wJ
z7w}3L2I)+K!Xqc8DFAj<rsECAZ=mK<)^!gf?-1S)EO~jdz0f@Q@F3#3L1fr^$o86d
zuQM@X{pa9ZNZcl!jAtjbtQkaXJQNxJ@&v<66d%H=a>YN{<@(BvW>q6N*HmjqQ`NEc
zh1S`Rut$KW-A~TUfg;xlU)L~+hKQd<=76v+epG8j1n#0MBfx6HA{5$e@O8GU`0RSF
z<3ppHe5d-TI08h{s9O+SCyD&p;^trw-!FuAstODG;oWSUUe7AnmB``YR2>n@E*5%>
z8Of38ot0Q==k#Mj6E$v|ygegS8`B2fZpa;ENVQq*i%AqK&#3=1Ba0U@0$dqG=_t7o
zn+p(0)PE>A<A)17a2z-3Abg^N*qN`56(llVH$|>;qKw^7R>FD#yx*o}`^4G8lk9kP
z=W6D#DN7d_r>Mx_<Kz2agxT`FyhnzY5=!X-#d}j7pf>=#uH^KsJfY4clk`rLa1AuA
z(yEfCIPOB3@|Up7Eq1Q)V!Y6!H23*N@J1!`A2GjSqaR@IHoAK@XDY1BDXrTzno5rn
zeshqL@J6ZHog^Jpnm4RRnL5~<4s)ncGrwwCT;-ILyRACDs&Z&{M{YMtl#Muzx7!aj
zht0kjl-xtkQQ2V)fsu~U1eq)GU%=QcgM#5+V~Qj<e7i$_KXG*W96h~eY*@X4(o4k+
ztKTQ|a6(Ewo$Zk-{a5W6U^;BFfB{pJ=B!X>rh9);4`67gt>Vdy3K8)_$0nPE{F|CO
z*+SV<kLRZ?YzN;bPy=6<eXsvJYgJePH09_oQ$jBZqE}ke7G+uHPxSPN$d3j?Xb;5i
z8J|nAq0|e!B#9eWmOWsltbFI*PouIK4`nMsG(1RH=KRW1tH9aou_=ObY;gZJ#oY^d
z7xl;xCuw7vWhe|D83d+U+n!BiPZ)v|zQuODDge#24K^b%GA6^C?B@e*T=Ht&e8iEc
zlk7S6sV12_YA{r{cFlgGl`oacQ~oenfngcegI71rNJGIELqbD^$_kBw=sd4ycM~%#
z%N-OF=X;J>p=^T<0#-TqiNmM*T$9pgGo(Rv-}&^;F(HNv^JAsx?r7m-F-zisCuB$Z
zH#Ugt{j!TR%LOpgR`<RA%fKB7ny#C%$VFIm!)#5oiov<kD)>wYZr%B1r$CKgj<##Y
z1fK6hPFJ@YFskqxtk-e{!0aprZ(!LuH^#<g;Ikq#pp>yGRu%k+DNMl;W_O-`;hnzZ
z9xy!P{2<5f!?8hv@=)@I^=Q|0g3ORGwlDz#8o;t|^|{0zvD8R8D{PA(Px|e*W;|DI
z`hyJ9*HV8p{Q>fbcKbg)Qh}zF_G!F783Q0_;^1{i>|aOmEs$W>RsU-o#R@k!H}Bru
ztY|T=#xZx+hw=64k?)iC=BT9UmQ*FT?daRL;aQoP5zCbt=YxCNbt*d(o`o2!8@mdP
zTtTR*s~2{6cYklJ`S|f!QgSkR&6&QlwIxBB(?Mg&&y~bptr*(BcPbzywV$wy=AAb_
z1eHYHg(YaAc%ic!wJ=H#=8%~yT2t$hNkY)UPg*>0$D#P3+4lHo;6Kg*qW?Sh-}DRZ
zSmWyIsywVo&YN%lmY0{8eLYJxj(JfAhK8(SV`GD}u}xc(BV%JOTU)C|PY-KPOHstc
zF=_8Y?#DA|l1V_5N^mbRWoLJTA;*3&+#eGZ5utBUek<P`1-<y@&=o1Evwz@o1IOs?
zUfh(28O+L%EE3p3pvJXy=6=+jfx@d3*eM&WvB5O}TuRw)R{w1SA3EZxB$1KEKfBeh
zWk7_$Yhk>?o&W{FO92qcgo0)df0N5zs#%HZ0pROPe-^d99RuSCitaE<fq}KN1tLWJ
z?C~H$(fmePV;6Pfgf?pYsC3!PHx<7BC+ztDg#FKF|2H&zt}i!9N1vaZoLuL}A~&u%
zm8<ZKwMkA-OiV=bSy@{b=H}+I>r_-%D=LMwW)(&-HyBtrA3uK~iA35S-dXGSMV5~B
zjvN;^H1{emFHZ&Bx{aNboV}M7FvE|N$H!flARa%fc4mJNg4yNpD@(gO+lpZj`27<W
z(5{wmahIa_J^JpL(ZlgrCUd!+hWi+G;fLYQN;i6^^^twi7V^Zi%X)RU`|y`%GWWJ-
z2_%=a;lf7LwLzxO2?3qhx@$kOX3N?q3#>LWoTEGoL{N1HE<A$4kLAC^^|trWd2kA)
zQJafAdB`lkA_dK-c==q0e%zeHr?CeaU02R{X|D(9W}g&D@7SW(lcXE6{Jtt^qKxfQ
zgNO&z_eBg-T%gxEi=(Y7S(JX_9Mp2aUN4mmAEKP7P`cYFD#2mtv9mlXq8V^7t@Ik0
z*tLq>a9J_ObFQ4^*mw6VJymlyo8#O^Jd_5)IC;TjMpLG2BrH9m5|F$&1x-U82A9-4
z&O!Z`SYF9r?<K#B!xC|HC26PALTdSCPN3AIF^Tt0lWv6TM==>qYW->LD&<3q_TLcx
zGX0>>3=0pcI|Y8_EN@h1A2i%!cI*2hX4dKSf`~_zqD+o$g%@!P_e6$@(Aj(H>p@GD
zOOB3?gFy$Y=(|*s`{4UyI^sD$RSm`)8XDx+-_PdbMf=mXiHIns8088_vo5_kcym6e
zsV_gXmCQ3S(9-oGG9v2}CS&{6lbz~puhbH=8F)B3N;>|gcQz+w;KRuv)ZL878yv)|
zjEaOuIKn&vEQKLms#!=K4=gY7uR#XrLMXbGcc^>d*DCWCCIG&BKE16#U#Yh7b!UjK
z>~w{CAUqI`SUu<~2|CR?jK+c$fr?N-!n5^Rd`?elb89f0vp9Qsn+1(YOnZNBtc&{i
zE<gCZje6=kFo|wWEM}EgUVy+SbRcl6L`BmLxr7{24vCQ$0O~&FPAcqR1sRs|ShBLP
zLb8Muy16tVpFCu-u`=Dg0)8x_3g*6k3SPK27u;IiD+NVkkVwS5M0{I;R7BlpGajeI
zpC0)9SdZRVye>G}+Ma*q1idrYTx~7f)7h(r4?OLSoEHS`!P9P*mVE{%9qr&32U5Z@
zG_`y7OGEa=PAi*bf?>8{bT!9z{!Y~D%y;>}^0u!@;Fq_mosS&pOIns`_bS$}jNFEY
zP8`uvuPs2JRRdo}9eGzb5WUOP@oG@+CEs13=6x~4sla5%8_ripZ2H&Of-}?es&G%{
zwzr0ju4J6j3r;1yT|fMqLRm;<auJ*4X6A{>rH;&*Wk<S^rtD_t2d#x$pB9KI&$^`U
zx8*z0LT9>`-dA#ND7dw@p0?V>BEu<HrfziKO6l>RdC;Bd^Y?W*^UJaqu?bHqo}}vA
zjWo~xl_8@j=ZZ`CTgXzw1|=jqsCc#&ep5Pt%MI!3^~KDv{M*e_?k?xA+sP6vN+x9X
zGGb}6u(-o)p4URF%nI`uYlq-De6GhI(@k6+T%W?q5|v|(y|Z8a)uz}RN3)N#nfX&s
zWHYO$SU=+<51tLNc+vV&d;0L~!nvSSHc6_6j00NrdN{%Lj2I3$7nj0<KTdsp=&Cn%
zCFutXTo#VF@DR!q`fAbStxJT)i!+~Rq&$Odz1qxTke*gIety3q@19@qPB}IZKC$>W
zCb74mw@BY^zIks|)8y@4zk9|Ig@yjrBMRK`kAeB~Rl7Y_rlzm5tT(QuLujGrcu=W|
zeWJvaOpBji;*Y!d-9IwUcj7P<-Q}<AP7APfQqYOx@4z5Kt!^FC^pxVJOsPClODx)p
zDN_NctdfY}+(Yj?d@}AnR;4t@(56i54c9y3Y+vBa@(KC%0Nm5GgA4AjDRtQB8<v&A
zRCgDR*7xRvE9hL+L)SCYfv5f@8aK~-KT&NU-AN2QK1Tg0Tw*z#W9NG(@wQ3d_o?N4
zDfHIctHcxq;RCx}^`1oqvnwD^DQjR7)hTCpSD2a4^Z44ol7rwcQ!X6!znEv~XSrn&
z8cDmUiEugAViD3|cT1uwy>rDVs6T4<^dg)$Kf$6CMb@}XOws91GSPcX<SP$?>l!e3
zZ<5-~9M(@!|E^;wJl)c>K!+qLDZq#fs3ams^-HS0%8iSdzJ@l(`!X;2PK!K850qUn
zk0Gn+Ts?`dw@oj3X$i4ted4(KbUTWh;xRwH^fatgt;iy>&CyY`<Qgu~c(u)pFw%9~
zGCD>{w9Zl^D0Y0|PmYU$EytdRRNX<UPbRqb+(~o_!{ax~63VZbrRG0aoO!%q)4y0j
zvsjbyA=hoDG9FDndy%ps_gT^4L%){NiD)ggoTOPCH)|K4;3P92hNh?2eeu3WupFGx
zd0JtmQ4)|yiuXCJnmc{THNaUho}kv93l;w~%^mKthF3%M#&fOI2Z+h7-`2wWkU%nP
z$c7Gh6I<d^Pu`ZN(7wed&8G;VY@K|yP_{2q9n%~9%bTRF9T-U6W4icjkm)6RvO2n8
z4oal@pdTgof*dt_H&L~vahPQBy5^AvNK7Ag@d&&L?>>>$|6wx(stD*RB3ZO}74JU<
z@DC)2A66<ZzjJ5;J4H8qBL-=j`%aC1sygFlX>WFTDK~4_H{TXGNVpLAQCRDH8U&B`
z?Y4Cq5nae9fn8?2ltcXuaq?9e2l-I426&8-=d6qO?_iNcjc-N3_FKk+fe{4x(Eh6N
zHSB`~IlBcU*-2K)DJ=<Wb9u!2y|$RPg2LS~Z*c6XV`B;DLW0zaK~{VYf2OC4M`(Fk
z%l6-N23u3AsbSf$rMoSk!7uI|zX;7FDQmy)d)OIL`!CMFBJ#Bl<nRtESSUb^nIx*M
z2WZN_%0wr1?zKg@AXRO9zk8UTn(`3LNm@Ilub=7T>sy1&Rwq^bb3}{%8arj}KXy#o
ti<%uDA3wda5=aON0EaxVnuTof6LM+ivt4^5!G9wG3sWnTk~63~{{Y^!D~|vG

literal 0
HcmV?d00001

diff --git a/doc/guides/nics/mvpp2.rst b/doc/guides/nics/mvpp2.rst
index 3b3f8c6..e7f45c3 100644
--- a/doc/guides/nics/mvpp2.rst
+++ b/doc/guides/nics/mvpp2.rst
@@ -56,7 +56,7 @@  Features of the MVPP2 PMD are:
 
 - Speed capabilities
 - Link status
-- Queue start/stop
+- Tx Queue start/stop
 - MTU update
 - Jumbo frame
 - Promiscuous mode
@@ -70,12 +70,13 @@  Features of the MVPP2 PMD are:
 - L4 checksum offload
 - Packet type parsing
 - Basic stats
-- Extended stats
-- QoS
+- :ref:`Extended stats <extstats>`
 - RX flow control
-- TX queue start/stop
 - Scattered TX frames
-
+- :ref:`QoS <qossupport>`
+- :ref:`Flow API <flowapi>`
+- :ref:`Traffic metering and policing <mtrapi>`
+- :ref:`Traffic Management API <tmapi>`
 
 Limitations
 -----------
@@ -89,6 +90,20 @@  Limitations
   functionality. Current workaround is to reset board so that PPv2 has a
   chance to start in a sane state.
 
+- MUSDK architecture does not support changing configuration in run time.
+  All nessesary configurations should be done before first dev_start().
+
+- RX queue start/stop is not supported.
+
+- Current implementation does not support replacement of buffers in the HW buffer pool
+  at run time, so it is responsibility of the application to ensure that MTU does not exceed the configured buffer size.
+
+- Configuring TX flow control currently is not supported.
+
+- In current implementation, mechanism for acknowledging transmitted packets (``tx_done_cleanup``) is not supported.
+
+- Running more than one DPDK-MUSDK application simultaneously is not supported.
+
 
 Prerequisites
 -------------
@@ -139,6 +154,92 @@  The following options can be modified in the ``config`` file.
 
        When MVPP2 PMD is enabled ``CONFIG_RTE_LIBRTE_MVNETA_PMD`` must be disabled
 
+
+Building DPDK
+-------------
+
+Driver needs precompiled MUSDK library during compilation.
+
+.. code-block:: console
+
+   export CROSS_COMPILE=<toolchain>/bin/aarch64-linux-gnu-
+   ./bootstrap
+   ./configure --host=aarch64-linux-gnu
+   make install
+
+MUSDK will be installed to `usr/local` under current directory.
+For the detailed build instructions please consult ``doc/musdk_get_started.txt``.
+
+Before the DPDK build process the environmental variable ``LIBMUSDK_PATH`` with
+the path to the MUSDK installation directory needs to be exported.
+
+For additional instructions regarding DPDK cross compilation please refer to :doc:`Cross compile DPDK for ARM64 <../linux_gsg/cross_build_dpdk_for_arm64>`.
+
+.. code-block:: console
+
+   export LIBMUSDK_PATH=<musdk>/usr/local
+   export CROSS=<toolchain>/bin/aarch64-linux-gnu-
+   export RTE_KERNELDIR=<kernel-dir>
+   export RTE_TARGET=arm64-armv8a-linuxapp-gcc
+
+   make config T=arm64-armv8a-linuxapp-gcc
+   sed -i "s/MVNETA_PMD=y/MVNETA_PMD=n/" build/.config
+   sed -i "s/MVPP2_PMD=n/MVPP2_PMD=y/" build/.config
+   make
+
+Usage Example
+-------------
+
+MVPP2 PMD requires extra out of tree kernel modules to function properly.
+`musdk_cma` sources are part of the MUSDK. Please consult
+``doc/musdk_get_started.txt`` for the detailed build instructions.
+For `mvpp2x_sysfs` please consult ``Documentation/pp22_sysfs.txt`` for the
+detailed build instructions.
+
+.. code-block:: console
+
+   insmod musdk_cma.ko
+   insmod mvpp2x_sysfs.ko
+
+Additionally interfaces used by DPDK application need to be put up:
+
+.. code-block:: console
+
+   ip link set eth0 up
+   ip link set eth2 up
+
+In order to run testpmd example application following command can be used:
+
+.. code-block:: console
+
+   ./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 7 -- \
+     --burst=128 --txd=2048 --rxd=1024 --rxq=2 --txq=2  --nb-cores=2 \
+     -i -a --rss-udp
+
+.. _extstats:
+
+Extended stats
+--------------
+
+MVPP2 PMD supports the following extended statistics:
+
+	- ``rx_bytes``:	number of RX bytes
+	- ``rx_packets``: number of RX packets
+	- ``rx_unicast_packets``: number of RX unicast packets
+	- ``rx_errors``: number of RX MAC errors
+	- ``rx_fullq_dropped``: number of RX packets dropped due to full RX queue
+	- ``rx_bm_dropped``: number of RX packets dropped due to no available buffers in the HW pool
+	- ``rx_early_dropped``: number of RX packets that were early dropped
+	- ``rx_fifo_dropped``: number of RX packets dropped due to RX fifo overrun
+	- ``rx_cls_dropped``: number of RX packets dropped by classifier
+	- ``tx_bytes``: number of TX bytes
+	- ``tx_packets``: number of TX packets
+	- ``tx_unicast_packets``: number of TX unicast packets
+	- ``tx_errors``: number of TX MAC errors
+
+
+.. _qossupport:
+
 QoS Configuration
 -----------------
 
@@ -302,39 +403,14 @@  Usage example
    ./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2,cfg=/home/user/mrvl.conf \
      -c 7 -- -i -a --disable-hw-vlan-strip --rxq=3 --txq=3
 
-
-Building DPDK
--------------
-
-Driver needs precompiled MUSDK library during compilation.
-
-.. code-block:: console
-
-   export CROSS_COMPILE=<toolchain>/bin/aarch64-linux-gnu-
-   ./bootstrap
-   ./configure --host=aarch64-linux-gnu
-   make install
-
-MUSDK will be installed to `usr/local` under current directory.
-For the detailed build instructions please consult ``doc/musdk_get_started.txt``.
-
-Before the DPDK build process the environmental variable ``LIBMUSDK_PATH`` with
-the path to the MUSDK installation directory needs to be exported.
-
-.. code-block:: console
-
-   export LIBMUSDK_PATH=<musdk>/usr/local
-   export CROSS=aarch64-linux-gnu-
-   make config T=arm64-armv8a-linuxapp-gcc
-   sed -i "s/MVNETA_PMD=y/MVNETA_PMD=n/" build/.config
-   sed -i "s/MVPP2_PMD=n/MVPP2_PMD=y/" build/.config
-   make
+.. _flowapi:
 
 Flow API
 --------
 
 PPv2 offers packet classification capabilities via classifier engine which
 can be configured via generic flow API offered by DPDK.
+For an additional description please refer to DPDK :ref:`Generic flow API <Generic_flow_API>`.
 
 Supported flow actions
 ~~~~~~~~~~~~~~~~~~~~~~
@@ -495,31 +571,239 @@  Following limitations need to be taken into account while creating flow rules:
 For additional information about classifier please consult
 ``doc/musdk_cls_user_guide.txt``.
 
-Usage Example
--------------
+.. _mtrapi:
 
-MVPP2 PMD requires extra out of tree kernel modules to function properly.
-`musdk_cma` sources are part of the MUSDK. Please consult
-``doc/musdk_get_started.txt`` for the detailed build instructions.
-For `mvpp2x_sysfs` please consult ``Documentation/pp22_sysfs.txt`` for the
-detailed build instructions.
+Traffic metering and policing
+-----------------------------
 
-.. code-block:: console
+MVPP2 PMD supports DPDK traffic metering and policing that allows the following:
 
-   insmod musdk_cma.ko
-   insmod mvpp2x_sysfs.ko
+1. Meter ingress traffic.
+2. Do policing.
+3. Gather statistics.
 
-Additionally interfaces used by DPDK application need to be put up:
+For an additional description please refer to DPDK :doc:`Traffic Metering and Policing API <../prog_guide/traffic_metering_and_policing>`.
 
-.. code-block:: console
+The policer objects defined by this feature can work with the default policer defined via config file as discribed in :ref:`QoS Support <qossupport>`.
 
-   ip link set eth0 up
-   ip link set eth2 up
+Limitations
+~~~~~~~~~~~
 
-In order to run testpmd example application following command can be used:
+The following capabilities are not supported:
 
-.. code-block:: console
+- MTR object meter DSCP table update
+- MTR object policer action update
+- MTR object enabled statistics
 
-   ./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 7 -- \
-     --burst=128 --txd=2048 --rxd=1024 --rxq=2 --txq=2  --nb-cores=2 \
-     -i -a --rss-udp
+Usage example
+~~~~~~~~~~~~~
+
+1. Run testpmd user app:
+
+   .. code-block:: console
+
+		./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 6 -- -i -p 3 -a --txd 1024 --rxd 1024
+
+2. Create meter profile:
+
+   .. code-block:: console
+
+		testpmd> add port meter profile 0 0 srtcm_rfc2697 2000 256 256
+
+3. Create meter:
+
+   .. code-block:: console
+
+		testpmd> create port meter 0 0 0 yes d d d 0 1 0
+
+4. Create flow rule witch meter attached:
+
+   .. code-block:: console
+
+		testpmd> flow create 0 ingress pattern ipv4 src is 10.10.10.1 / end actions meter mtr_id 0 / end
+
+For a detailed usage description please refer to "Traffic Metering and Policing" section in DPDK :doc:`Testpmd Runtime Functions <../testpmd_app_ug/testpmd_funcs>`.
+
+
+
+.. _tmapi:
+
+Traffic Management API
+----------------------
+
+MVPP2 PMD supports generic DPDK Traffic Management API which allows to
+configure the following features:
+
+1. Hierarchical scheduling
+2. Traffic shaping
+3. Congestion management
+4. Packet marking
+
+Internally TM is represented by a hierarchy (tree) of nodes.
+Node which has a parent is called a leaf whereas node without
+parent is called a non-leaf (root).
+MVPP2 PMD supports two level hierarchy where level 0 represents ports and level 1 represents tx queues of a given port.
+
+.. figure:: img/mvpp2_tm.png
+
+Nodes hold following types of settings:
+
+- for egress scheduler configuration: weight
+- for egress rate limiter: private shaper
+- bitmask indicating which statistics counters will be read
+
+Hierarchy is always constructed from the top, i.e first a root node is added
+then some number of leaf nodes. Number of leaf nodes cannot exceed number
+of configured tx queues.
+
+After hierarchy is complete it can be committed.
+
+
+For an additional description please refer to DPDK :doc:`Traffic Management API <../prog_guide/traffic_management>`.
+
+Limitations
+~~~~~~~~~~~
+
+The following capabilities are not supported:
+
+- Traffic manager WRED profile and WRED context
+- Traffic manager shared shaper update
+- Traffic manager packet marking
+- Maximum number of levels in hierarchy is 2
+- Currently dynamic change of a hierarchy is not supported
+
+Usage example
+~~~~~~~~~~~~~
+
+For a detailed usage description please refer to "Traffic Management" section in DPDK :doc:`Testpmd Runtime Functions <../testpmd_app_ug/testpmd_funcs>`.
+
+1. Run testpmd as follows:
+
+   .. code-block:: console
+
+		./testpmd --vdev=net_mrvl,iface=eth0,iface=eth2,cfg=./qos_config -c 7 -- \
+		-i -p 3 --disable-hw-vlan-strip --rxq 3 --txq 3 --txd 1024 --rxd 1024
+
+2. Stop all ports:
+
+   .. code-block:: console
+
+		testpmd> port stop all
+
+3. Add shaper profile:
+
+   .. code-block:: console
+
+		testpmd> add port tm node shaper profile 0 0 900000 70000 0
+
+   Parameters have following meaning::
+
+		0       - Id of a port.
+		0       - Id of a new shaper profile.
+		900000  - Shaper rate in bytes/s.
+		70000   - Bucket size in bytes.
+		0       - Packet length adjustment - ignored.
+
+4. Add non-leaf node for port 0:
+
+   .. code-block:: console
+
+		testpmd> add port tm nonleaf node 0 3 -1 0 0 0 0 0 1 3 0
+
+   Parameters have following meaning::
+
+		 0  - Id of a port
+		 3  - Id of a new node.
+		-1  - Indicate that root does not have a parent.
+		 0  - Priority of the node.
+		 0  - Weight of the node.
+		 0  - Id of a level. Since this is a root 0 is passed.
+		 0  - Id of the shaper profile.
+		 0  - Number of SP priorities.
+		 3  - Enable statistics for both number of transmitted packets and bytes.
+		 0  - Number of shared shapers.
+
+5. Add leaf node for tx queue 0:
+
+   .. code-block:: console
+
+		testpmd> add port tm leaf node 0 0 3 0 30 1 -1 0 0 1 0
+
+   Parameters have following meaning::
+
+		 0  - Id of a port.
+		 0  - Id of a new node.
+		 3  - Id of the parent node.
+		 0  - Priority of a node.
+		 30 - WRR weight.
+		 1  - Id of a level. Since this is a leaf node 1 is passed.
+		-1  - Id of a shaper. -1 indicates that shaper is not attached.
+		 0  - Congestion management is not supported.
+		 0  - Congestion management is not supported.
+		 1  - Enable statistics counter for number of transmitted packets.
+		 0  - Number of shared shapers.
+
+6. Add leaf node for tx queue 1:
+
+   .. code-block:: console
+
+	testpmd> add port tm leaf node 0 1 3 0 60 1 -1 0 0 1 0
+
+   Parameters have following meaning::
+
+		 0  - Id of a port.
+		 1  - Id of a new node.
+		 3  - Id of the parent node.
+		 0  - Priority of a node.
+		 60 - WRR weight.
+		 1  - Id of a level. Since this is a leaf node 1 is passed.
+		-1  - Id of a shaper. -1 indicates that shaper is not attached.
+		 0  - Congestion management is not supported.
+		 0  - Congestion management is not supported.
+		 1  - Enable statistics counter for number of transmitted packets.
+		 0  - Number of shared shapers.
+
+7. Add leaf node for tx queue 2:
+
+   .. code-block:: console
+
+		testpmd> add port tm leaf node 0 2 3 0 99 1 -1 0 0 1 0
+
+   Parameters have following meaning::
+
+		 0  - Id of a port.
+		 2  - Id of a new node.
+		 3  - Id of the parent node.
+		 0  - Priority of a node.
+		 99 - WRR weight.
+		 1  - Id of a level. Since this is a leaf node 1 is passed.
+		-1  - Id of a shaper. -1 indicates that shaper is not attached.
+		 0  - Congestion management is not supported.
+		 0  - Congestion management is not supported.
+		 1  - Enable statistics counter for number of transmitted packets.
+		 0  - Number of shared shapers.
+
+8. Commit hierarchy:
+
+   .. code-block:: console
+
+		testpmd> port tm hierarchy commit 0 no
+
+  Parameters have following meaning::
+
+		0  - Id of a port.
+		no - Do not flush TM hierarchy if commit fails.
+
+9. Start all ports
+
+   .. code-block:: console
+
+		testpmd> port start all
+
+
+
+10. Enable forwarding
+
+   .. code-block:: console
+
+		testpmd> start