[v6,00/11] sched: feature enhancements

  This patchset refactors the dpdk qos sched library to allow flexibile
configuration of the pipe traffic classes and queue sizes.

Currently, each pipe has 16 queues hardwired into 4 TCs scheduled with
strict priority, and each TC has exactly with 4 queues that are
scheduled with Weighted Fair Queuing (WFQ).

Instead of hardwiring queues to traffic class within the specific pipe,
the new implementation allows more flexible/configurable split of pipe
queues between strict priority (SP) and best-effort (BE) traffic classes
along with the support of more number of traffic classes i.e. max 16.
   
All the high priority TCs (TC1, TC2, ...) have exactly 1 queue, while
the lowest priority best-effort traffic class can have 1, 4 or 8 queues.
This is justified by the fact that all the high priority TCs are fully
provisioned (small to medium traffic rates), while most of the traffic
fits into the BE class, which is typically oversubscribed.

Furthermore, this change allows to use less than 16 queues per pipe when
not all the 16 queues are needed. Therefore, no memory will be allocated
to the queues that are not needed.

v6:
- add functions to access port internal struct fields (e.g. pipe queues and tc)
- Move definition of RTE_SCHED_TRAFFIC_CLASS_BE to rte_sched.h
- fix doxygen comments

v5:
- fix traffic class and queue mapping in api function
- remove n_be_queues parameter from internal pipe profile and pipe struct
- replace int multiplication in grinder_schedule func with bitwise & operation
- remove TC_OV logic flag from all the configuration/initialization code
- fix traffic qsize per traffic class instead of individual queue of the pipe

v4:
- fix build errors
- fix checkpatch errors

v3:
- remove code related to subport level configuration of the pipe 
- remove tc oversubscription flag from struct rte_sched_pipe_params
- replace RTE_SCHED_PIPE_PROFILES_PER_PORT with port param field

v2:
- fix bug in subport parameters check
- remove redundant RTE_SCHED_SUBPORT_PER_PORT macro
- fix bug in grinder_scheduler function
- improve doxygen comments 
- add error log information

Jasvinder Singh (11):
  sched: remove wrr from strict priority tc queues
  sched: add config flexibility to tc queue sizes
  sched: add max pipe profiles config in run time
  sched: rename tc3 params to best-effort tc
  sched: improve error log messages
  sched: improve doxygen comments
  net/softnic: add config flexibility to softnic tm
  test_sched: modify tests for config flexibility
  examples/ip_pipeline: add config flexibility to tm function
  examples/qos_sched: add tc and queue config flexibility
  sched: remove redundant macros

 app/test/test_sched.c                         |  15 +-
 doc/guides/rel_notes/release_19_08.rst        |  10 +-
 drivers/net/softnic/rte_eth_softnic.c         |  98 ++
 drivers/net/softnic/rte_eth_softnic_cli.c     | 448 ++++++++-
 .../net/softnic/rte_eth_softnic_internals.h   |   6 +-
 drivers/net/softnic/rte_eth_softnic_tm.c      | 121 ++-
 examples/ip_pipeline/cli.c                    |  43 +-
 examples/ip_pipeline/tmgr.h                   |   4 +-
 examples/qos_sched/app_thread.c               |  11 +-
 examples/qos_sched/cfg_file.c                 | 130 ++-
 examples/qos_sched/init.c                     |  65 +-
 examples/qos_sched/main.h                     |   4 +
 examples/qos_sched/profile.cfg                |  67 +-
 examples/qos_sched/profile_ov.cfg             |  54 +-
 examples/qos_sched/stats.c                    | 517 ++++++-----
 lib/librte_pipeline/rte_table_action.c        |   1 -
 lib/librte_pipeline/rte_table_action.h        |   4 +-
 lib/librte_sched/Makefile                     |   2 +-
 lib/librte_sched/meson.build                  |   2 +-
 lib/librte_sched/rte_sched.c                  | 857 ++++++++++++------
 lib/librte_sched/rte_sched.h                  | 187 ++--
 21 files changed, 1874 insertions(+), 772 deletions(-)
  

19/07/2019 16:18, Jasvinder Singh:
> v6:
> - add functions to access port internal struct fields (e.g. pipe queues and tc)
> - Move definition of RTE_SCHED_TRAFFIC_CLASS_BE to rte_sched.h
> - fix doxygen comments

Thank you

It would be perfect if checkpatches issues were resolved:

### [dpdk-dev] [PATCH v6 02/11] sched: add config flexibility to tc queue sizes

WARNING:LONG_LINE: line over 100 characters
#579: FILE: lib/librte_sched/rte_sched.c:961:
+               double subport_tc3_rate = (double) s->tc_credits_per_period[RTE_SCHED_TRAFFIC_CLASS_BE]

WARNING:LONG_LINE: line over 100 characters
#582: FILE: lib/librte_sched/rte_sched.c:963:
+               double pipe_tc3_rate = (double) params->tc_credits_per_period[RTE_SCHED_TRAFFIC_CLASS_BE]

total: 0 errors, 2 warnings, 728 lines checked

### [dpdk-dev] [PATCH v6 04/11] sched: rename tc3 params to best-effort tc

WARNING:LONG_LINE: line over 100 characters
#235: FILE: lib/librte_sched/rte_sched.c:963:
+               double subport_tc_be_rate = (double) s->tc_credits_per_period[RTE_SCHED_TRAFFIC_CLASS_BE]

WARNING:LONG_LINE: line over 100 characters
#238: FILE: lib/librte_sched/rte_sched.c:965:
+               double pipe_tc_be_rate = (double) params->tc_credits_per_period[RTE_SCHED_TRAFFIC_CLASS_BE]

total: 0 errors, 2 warnings, 128 lines checked

### [dpdk-dev] [PATCH v6 07/11] net/softnic: add config flexibility to softnic tm

WARNING:DEEP_INDENTATION: Too many leading tabs - consider code refactoring
#479: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:843:
+                                               if (status)

CHECK:BRACES: braces {} should be used on all arms of this statement
#737: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1205:
+       if (strcmp(tokens[51], "none") == 0)
[...]
+       else {
[...]

CHECK:BRACES: Unbalanced braces around else statement
#739: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1207:
+       else {

CHECK:BRACES: braces {} should be used on all arms of this statement
#754: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1222:
+       if (strcmp(tokens[53], "none") == 0)
[...]
+       else {
[...]

CHECK:BRACES: Unbalanced braces around else statement
#756: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1224:
+       else {

CHECK:BRACES: braces {} should be used on all arms of this statement
#771: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1239:
+       if (strcmp(tokens[55], "none") == 0)
[...]
+       else {
[...]

CHECK:BRACES: Unbalanced braces around else statement
#773: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1241:
+       else {

CHECK:BRACES: braces {} should be used on all arms of this statement
#788: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1256:
+       if (strcmp(tokens[57], "none") == 0)
[...]
+       else {
[...]

CHECK:BRACES: Unbalanced braces around else statement
#790: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1258:
+       else {

CHECK:BRACES: braces {} should be used on all arms of this statement
#805: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1273:
+       if (strcmp(tokens[59], "none") == 0)
[...]
+       else {
[...]

CHECK:BRACES: Unbalanced braces around else statement
#807: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1275:
+       else {

CHECK:BRACES: braces {} should be used on all arms of this statement
#822: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1290:
+       if (strcmp(tokens[61], "none") == 0)
[...]
+       else {
[...]

CHECK:BRACES: Unbalanced braces around else statement
#824: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1292:
+       else {

CHECK:BRACES: braces {} should be used on all arms of this statement
#839: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1307:
+       if (strcmp(tokens[63], "none") == 0)
[...]
+       else {
[...]

CHECK:BRACES: Unbalanced braces around else statement
#841: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1309:
+       else {

CHECK:BRACES: braces {} should be used on all arms of this statement
#856: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1324:
+       if (strcmp(tokens[65], "none") == 0)
[...]
+       else {
[...]

CHECK:BRACES: Unbalanced braces around else statement
#858: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1326:
+       else {

CHECK:BRACES: braces {} should be used on all arms of this statement
#873: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1341:
+       if (strcmp(tokens[67], "none") == 0)
[...]
+       else {
[...]

CHECK:BRACES: Unbalanced braces around else statement
#875: FILE: drivers/net/softnic/rte_eth_softnic_cli.c:1343:
+       else {

WARNING:SPACE_BEFORE_TAB: please, no space before tabs
#1129: FILE: drivers/net/softnic/rte_eth_softnic_tm.c:3045:
+^Ireturn ^Iport_queue_id;$

total: 0 errors, 2 warnings, 18 checks, 1002 lines checked

### [dpdk-dev] [PATCH v6 10/11] examples/qos_sched: add tc and queue config flexibility

WARNING:LONG_LINE_COMMENT: line over 100 characters
#201: FILE: examples/qos_sched/app_thread.c:42:
+       pipe_queue = active_queues[(pdata[QUEUE_OFFSET] >> 8) % n_active_queues]; /* Destination IP */

WARNING:LONG_LINE: line over 100 characters
#1042: FILE: examples/qos_sched/stats.c:206:
+                               part_average / (port_params.n_pipes_per_subport) * RTE_SCHED_BE_QUEUES_PER_PIPE;

WARNING:LONG_LINE: line over 100 characters
#1116: FILE: examples/qos_sched/stats.c:255:
+               average += part_average / (port_params.n_pipes_per_subport * RTE_SCHED_QUEUES_PER_PIPE);

WARNING:LONG_LINE: line over 100 characters
#1188: FILE: examples/qos_sched/stats.c:293:
+               printf("|  %d | %11" PRIu32 " | %11" PRIu32 " | %11" PRIu32 " | %11" PRIu32 " | %11" PRIu32 " |\n", i,

WARNING:LONG_LINE: line over 100 characters
#1268: FILE: examples/qos_sched/stats.c:336:
+                       printf("|  %d |   %d   | %11" PRIu32 " | %11" PRIu32 " | %11" PRIu32 " | %11" PRIu32 " | %11i |\n", i, 0,

WARNING:LONG_LINE: line over 100 characters
#1269: FILE: examples/qos_sched/stats.c:337:
+                               stats.n_pkts, stats.n_pkts_dropped, stats.n_bytes, stats.n_bytes_dropped, qlen);

WARNING:LONG_LINE: line over 100 characters
#1274: FILE: examples/qos_sched/stats.c:342:
+                               printf("|  %d |   %d   | %11" PRIu32 " | %11" PRIu32 " | %11" PRIu32 " | %11" PRIu32 " | %11i |\n", i, j,

WARNING:LONG_LINE: line over 100 characters
#1275: FILE: examples/qos_sched/stats.c:343:
+                                       stats.n_pkts, stats.n_pkts_dropped, stats.n_bytes, stats.n_bytes_dropped, qlen);

total: 0 errors, 8 warnings, 1063 lines checked
  

> -----Original Message-----
> From: Singh, Jasvinder
> Sent: Friday, July 19, 2019 3:18 PM
> To: dev@dpdk.org
> Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> Subject: [PATCH v6 00/11] sched: feature enhancements
> 
> This patchset refactors the dpdk qos sched library to allow flexibile
> configuration of the pipe traffic classes and queue sizes.
> 
> Currently, each pipe has 16 queues hardwired into 4 TCs scheduled with
> strict priority, and each TC has exactly with 4 queues that are
> scheduled with Weighted Fair Queuing (WFQ).
> 
> Instead of hardwiring queues to traffic class within the specific pipe,
> the new implementation allows more flexible/configurable split of pipe
> queues between strict priority (SP) and best-effort (BE) traffic classes
> along with the support of more number of traffic classes i.e. max 16.
> 
> All the high priority TCs (TC1, TC2, ...) have exactly 1 queue, while
> the lowest priority best-effort traffic class can have 1, 4 or 8 queues.
> This is justified by the fact that all the high priority TCs are fully
> provisioned (small to medium traffic rates), while most of the traffic
> fits into the BE class, which is typically oversubscribed.
> 
> Furthermore, this change allows to use less than 16 queues per pipe when
> not all the 16 queues are needed. Therefore, no memory will be allocated
> to the queues that are not needed.
> 
> v6:
> - add functions to access port internal struct fields (e.g. pipe queues and tc)
> - Move definition of RTE_SCHED_TRAFFIC_CLASS_BE to rte_sched.h
> - fix doxygen comments
> 
> v5:
> - fix traffic class and queue mapping in api function
> - remove n_be_queues parameter from internal pipe profile and pipe struct
> - replace int multiplication in grinder_schedule func with bitwise & operation
> - remove TC_OV logic flag from all the configuration/initialization code
> - fix traffic qsize per traffic class instead of individual queue of the pipe
> 
> v4:
> - fix build errors
> - fix checkpatch errors
> 
> v3:
> - remove code related to subport level configuration of the pipe
> - remove tc oversubscription flag from struct rte_sched_pipe_params
> - replace RTE_SCHED_PIPE_PROFILES_PER_PORT with port param field
> 
> v2:
> - fix bug in subport parameters check
> - remove redundant RTE_SCHED_SUBPORT_PER_PORT macro
> - fix bug in grinder_scheduler function
> - improve doxygen comments
> - add error log information
> 
> Jasvinder Singh (11):
>   sched: remove wrr from strict priority tc queues
>   sched: add config flexibility to tc queue sizes
>   sched: add max pipe profiles config in run time
>   sched: rename tc3 params to best-effort tc
>   sched: improve error log messages
>   sched: improve doxygen comments
>   net/softnic: add config flexibility to softnic tm
>   test_sched: modify tests for config flexibility
>   examples/ip_pipeline: add config flexibility to tm function
>   examples/qos_sched: add tc and queue config flexibility
>   sched: remove redundant macros
> 
>  app/test/test_sched.c                         |  15 +-
>  doc/guides/rel_notes/release_19_08.rst        |  10 +-
>  drivers/net/softnic/rte_eth_softnic.c         |  98 ++
>  drivers/net/softnic/rte_eth_softnic_cli.c     | 448 ++++++++-
>  .../net/softnic/rte_eth_softnic_internals.h   |   6 +-
>  drivers/net/softnic/rte_eth_softnic_tm.c      | 121 ++-
>  examples/ip_pipeline/cli.c                    |  43 +-
>  examples/ip_pipeline/tmgr.h                   |   4 +-
>  examples/qos_sched/app_thread.c               |  11 +-
>  examples/qos_sched/cfg_file.c                 | 130 ++-
>  examples/qos_sched/init.c                     |  65 +-
>  examples/qos_sched/main.h                     |   4 +
>  examples/qos_sched/profile.cfg                |  67 +-
>  examples/qos_sched/profile_ov.cfg             |  54 +-
>  examples/qos_sched/stats.c                    | 517 ++++++-----
>  lib/librte_pipeline/rte_table_action.c        |   1 -
>  lib/librte_pipeline/rte_table_action.h        |   4 +-
>  lib/librte_sched/Makefile                     |   2 +-
>  lib/librte_sched/meson.build                  |   2 +-
>  lib/librte_sched/rte_sched.c                  | 857 ++++++++++++------
>  lib/librte_sched/rte_sched.h                  | 187 ++--
>  21 files changed, 1874 insertions(+), 772 deletions(-)
> 
> --
> 2.21.0


This patchset already has my ack starting from V5.

Series-acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
  

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Monday, July 22, 2019 9:20 AM
> To: Singh, Jasvinder <jasvinder.singh@intel.com>
> Cc: dev@dpdk.org; Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v6 00/11] sched: feature enhancements
> 
> 19/07/2019 16:18, Jasvinder Singh:
> > v6:
> > - add functions to access port internal struct fields (e.g. pipe queues and tc)
> > - Move definition of RTE_SCHED_TRAFFIC_CLASS_BE to rte_sched.h
> > - fix doxygen comments
> 
> Thank you
> 
> It would be perfect if checkpatches issues were resolved:
> 

Sent version 7 with most of the checkpatch warnings fixed. Thanks.