[v7,00/11] sched: feature enhancements

  This patchset refactors the dpdk qos sched library to allow flexibile
configuration of the pipe traffic classes and queue sizes.

Currently, each pipe has 16 queues hardwired into 4 TCs scheduled with
strict priority, and each TC has exactly with 4 queues that are
scheduled with Weighted Fair Queuing (WFQ).

Instead of hardwiring queues to traffic class within the specific pipe,
the new implementation allows more flexible/configurable split of pipe
queues between strict priority (SP) and best-effort (BE) traffic classes
along with the support of more number of traffic classes i.e. max 16.
   
All the high priority TCs (TC1, TC2, ...) have exactly 1 queue, while
the lowest priority best-effort traffic class can have 1, 4 or 8 queues.
This is justified by the fact that all the high priority TCs are fully
provisioned (small to medium traffic rates), while most of the traffic
fits into the BE class, which is typically oversubscribed.

Furthermore, this change allows to use less than 16 queues per pipe when
not all the 16 queues are needed. Therefore, no memory will be allocated
to the queues that are not needed.

v7:
- fix checkpatch warinings

v6:
- add functions to access port internal struct fields (e.g. pipe queues and tc)
- Move definition of RTE_SCHED_TRAFFIC_CLASS_BE to rte_sched.h
- fix doxygen comments

v5:
- fix traffic class and queue mapping in api function
- remove n_be_queues parameter from internal pipe profile and pipe struct
- replace int multiplication in grinder_schedule func with bitwise & operation
- remove TC_OV logic flag from all the configuration/initialization code
- fix traffic qsize per traffic class instead of individual queue of the pipe

v4:
- fix build errors
- fix checkpatch errors

v3:
- remove code related to subport level configuration of the pipe 
- remove tc oversubscription flag from struct rte_sched_pipe_params
- replace RTE_SCHED_PIPE_PROFILES_PER_PORT with port param field

v2:
- fix bug in subport parameters check
- remove redundant RTE_SCHED_SUBPORT_PER_PORT macro
- fix bug in grinder_scheduler function
- improve doxygen comments 
- add error log information

Jasvinder Singh (11):
  sched: remove wrr from strict priority tc queues
  sched: add config flexibility to tc queue sizes
  sched: add max pipe profiles config in run time
  sched: rename tc3 params to best-effort tc
  sched: improve error log messages
  sched: improve doxygen comments
  net/softnic: add config flexibility to softnic tm
  test_sched: modify tests for config flexibility
  examples/ip_pipeline: add config flexibility to tm function
  examples/qos_sched: add tc and queue config flexibility
  sched: remove redundant macros

 app/test/test_sched.c                         |  15 +-
 doc/guides/rel_notes/release_19_08.rst        |  10 +-
 drivers/net/softnic/rte_eth_softnic.c         |  98 ++
 drivers/net/softnic/rte_eth_softnic_cli.c     | 449 ++++++++-
 .../net/softnic/rte_eth_softnic_internals.h   |   6 +-
 drivers/net/softnic/rte_eth_softnic_tm.c      | 121 ++-
 examples/ip_pipeline/cli.c                    |  43 +-
 examples/ip_pipeline/tmgr.h                   |   4 +-
 examples/qos_sched/app_thread.c               |  13 +-
 examples/qos_sched/cfg_file.c                 | 130 ++-
 examples/qos_sched/init.c                     |  65 +-
 examples/qos_sched/main.h                     |   4 +
 examples/qos_sched/profile.cfg                |  67 +-
 examples/qos_sched/profile_ov.cfg             |  54 +-
 examples/qos_sched/stats.c                    | 526 ++++++-----
 lib/librte_pipeline/rte_table_action.c        |   1 -
 lib/librte_pipeline/rte_table_action.h        |   4 +-
 lib/librte_sched/Makefile                     |   2 +-
 lib/librte_sched/meson.build                  |   2 +-
 lib/librte_sched/rte_sched.c                  | 859 ++++++++++++------
 lib/librte_sched/rte_sched.h                  | 187 ++--
 21 files changed, 1888 insertions(+), 772 deletions(-)
  

> Jasvinder Singh (11):
>   sched: remove wrr from strict priority tc queues
>   sched: add config flexibility to tc queue sizes
>   sched: add max pipe profiles config in run time
>   sched: rename tc3 params to best-effort tc
>   sched: improve error log messages
>   sched: improve doxygen comments
>   net/softnic: add config flexibility to softnic tm
>   test_sched: modify tests for config flexibility
>   examples/ip_pipeline: add config flexibility to tm function
>   examples/qos_sched: add tc and queue config flexibility
>   sched: remove redundant macros

Applied, thanks

PS: please make all versions replying to the cover letter of v1.
  

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Monday, July 22, 2019 2:16 PM
> To: Singh, Jasvinder <jasvinder.singh@intel.com>
> Cc: dev@dpdk.org; Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v7 00/11] sched: feature enhancements
> 
> > Jasvinder Singh (11):
> >   sched: remove wrr from strict priority tc queues
> >   sched: add config flexibility to tc queue sizes
> >   sched: add max pipe profiles config in run time
> >   sched: rename tc3 params to best-effort tc
> >   sched: improve error log messages
> >   sched: improve doxygen comments
> >   net/softnic: add config flexibility to softnic tm
> >   test_sched: modify tests for config flexibility
> >   examples/ip_pipeline: add config flexibility to tm function
> >   examples/qos_sched: add tc and queue config flexibility
> >   sched: remove redundant macros
> 
> Applied, thanks
> 
> PS: please make all versions replying to the cover letter of v1.
> 
Thank you, Thomas. 

For sending new version, followed suggestion from Ferruh by replying to previous version, not the first version.
  

22/07/2019 15:22, Singh, Jasvinder:
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > PS: please make all versions replying to the cover letter of v1.
> 
> For sending new version, followed suggestion from Ferruh by replying to previous version, not the first version.

Replying to previous version creates an unneeded indentation.
And, as you are replying to first patch (not cover letter),
it makes new version in the middle of the previous one.
  

On 7/22/2019 2:33 PM, Thomas Monjalon wrote:
> 22/07/2019 15:22, Singh, Jasvinder:
>> From: Thomas Monjalon [mailto:thomas@monjalon.net]
>>> PS: please make all versions replying to the cover letter of v1.
>>
>> For sending new version, followed suggestion from Ferruh by replying to previous version, not the first version.
> 
> Replying to previous version creates an unneeded indentation.
> And, as you are replying to first patch (not cover letter),
> it makes new version in the middle of the previous one.
> 

Ahh, as Jasvinder said, I was always suggesting replying to first mail of the
previous version. As long as agreed on one, I don't really mind one against other.

What is the rule now, a new version replied to first mail of *first* version?
  

On Mon, Jul 22, 2019 at 02:53:04PM +0100, Ferruh Yigit wrote:
> On 7/22/2019 2:33 PM, Thomas Monjalon wrote:
> > 22/07/2019 15:22, Singh, Jasvinder:
> >> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> >>> PS: please make all versions replying to the cover letter of v1.
> >>
> >> For sending new version, followed suggestion from Ferruh by replying to previous version, not the first version.
> > 
> > Replying to previous version creates an unneeded indentation.
> > And, as you are replying to first patch (not cover letter),
> > it makes new version in the middle of the previous one.
> > 
> 
> Ahh, as Jasvinder said, I was always suggesting replying to first mail of the
> previous version. As long as agreed on one, I don't really mind one against other.
> 
> What is the rule now, a new version replied to first mail of *first* version?

I would not use the term "first mail" as that is ambiguous and could imply
the first patch. I always try and reply to the cover letter of the v1. That
keeps things in thread without a new level of indentation each subsequent
version.
  

On 7/22/2019 2:56 PM, Bruce Richardson wrote:
> On Mon, Jul 22, 2019 at 02:53:04PM +0100, Ferruh Yigit wrote:
>> On 7/22/2019 2:33 PM, Thomas Monjalon wrote:
>>> 22/07/2019 15:22, Singh, Jasvinder:
>>>> From: Thomas Monjalon [mailto:thomas@monjalon.net]
>>>>> PS: please make all versions replying to the cover letter of v1.
>>>>
>>>> For sending new version, followed suggestion from Ferruh by replying to previous version, not the first version.
>>>
>>> Replying to previous version creates an unneeded indentation.
>>> And, as you are replying to first patch (not cover letter),
>>> it makes new version in the middle of the previous one.
>>>
>>
>> Ahh, as Jasvinder said, I was always suggesting replying to first mail of the
>> previous version. As long as agreed on one, I don't really mind one against other.
>>
>> What is the rule now, a new version replied to first mail of *first* version?
> 
> I would not use the term "first mail" as that is ambiguous and could imply
> the first patch. I always try and reply to the cover letter of the v1. That
> keeps things in thread without a new level of indentation each subsequent
> version.
> 

I said "first mail" to escape from mentioned ambiguity :)

If first mail is "cover letter" reply to that, if there is no cover letter,
first mail is the 1/x patch, in this case reply to that patch.
I think we are saying same thing at the end of the day.
  

22/07/2019 15:56, Bruce Richardson:
> On Mon, Jul 22, 2019 at 02:53:04PM +0100, Ferruh Yigit wrote:
> > On 7/22/2019 2:33 PM, Thomas Monjalon wrote:
> > > 22/07/2019 15:22, Singh, Jasvinder:
> > >> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > >>> PS: please make all versions replying to the cover letter of v1.
> > >>
> > >> For sending new version, followed suggestion from Ferruh by replying to previous version, not the first version.
> > > 
> > > Replying to previous version creates an unneeded indentation.
> > > And, as you are replying to first patch (not cover letter),
> > > it makes new version in the middle of the previous one.
> > > 
> > 
> > Ahh, as Jasvinder said, I was always suggesting replying to first mail of the
> > previous version. As long as agreed on one, I don't really mind one against other.
> > 
> > What is the rule now, a new version replied to first mail of *first* version?
> 
> I would not use the term "first mail" as that is ambiguous and could imply
> the first patch. I always try and reply to the cover letter of the v1. That
> keeps things in thread without a new level of indentation each subsequent
> version.

I agree with Bruce