[0/8] DSW performance and statistics improvements

  Performance and statistics improvements for the distributed software
(DSW) event device.

Mattias Rönnblom (8):
  event/dsw: reduce latency in low-load situations
  event/dsw: reduce max flows to speed up load balancing
  event/dsw: extend statistics
  event/dsw: improve migration mechanism
  event/dsw: avoid migration waves in large systems
  event/dsw: remove redundant control ring poll
  event/dsw: remove unnecessary read barrier
  event/dsw: add port busy cycles xstats

 drivers/event/dsw/dsw_evdev.c  |   1 +
 drivers/event/dsw/dsw_evdev.h  |  45 ++-
 drivers/event/dsw/dsw_event.c  | 602 ++++++++++++++++++++-------------
 drivers/event/dsw/dsw_xstats.c |  26 +-
 4 files changed, 425 insertions(+), 249 deletions(-)
  

> -----Original Message-----
> From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> Sent: Monday, March 9, 2020 12:21 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> Cc: dev@dpdk.org; stefan.sundkvist@ericsson.com; Ola.Liljedahl@arm.com;
> Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> Subject: [EXT] [PATCH 0/8] DSW performance and statistics improvements
> 
> External Email
> 
> ----------------------------------------------------------------------
> Performance and statistics improvements for the distributed software
> (DSW) event device.
> 
> Mattias Rönnblom (8):
>   event/dsw: reduce latency in low-load situations
>   event/dsw: reduce max flows to speed up load balancing
>   event/dsw: extend statistics
>   event/dsw: improve migration mechanism
>   event/dsw: avoid migration waves in large systems
>   event/dsw: remove redundant control ring poll
>   event/dsw: remove unnecessary read barrier
>   event/dsw: add port busy cycles xstats

Series applied to dpdk-next-eventdev/master. Thanks.



> 
>  drivers/event/dsw/dsw_evdev.c  |   1 +
>  drivers/event/dsw/dsw_evdev.h  |  45 ++-  drivers/event/dsw/dsw_event.c  |
> 602 ++++++++++++++++++++-------------  drivers/event/dsw/dsw_xstats.c |  26 +-
>  4 files changed, 425 insertions(+), 249 deletions(-)
> 
> --
> 2.17.1
  

On Sat, Apr 4, 2020 at 4:35 PM Jerin Jacob Kollanukkaran
<jerinj@marvell.com> wrote:
>
> > -----Original Message-----
> > From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> > Sent: Monday, March 9, 2020 12:21 PM
> > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> > Cc: dev@dpdk.org; stefan.sundkvist@ericsson.com; Ola.Liljedahl@arm.com;
> > Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> > Subject: [EXT] [PATCH 0/8] DSW performance and statistics improvements
> >
> > External Email
> >
> > ----------------------------------------------------------------------
> > Performance and statistics improvements for the distributed software
> > (DSW) event device.
> >
> > Mattias Rönnblom (8):
> >   event/dsw: reduce latency in low-load situations
> >   event/dsw: reduce max flows to speed up load balancing
> >   event/dsw: extend statistics
> >   event/dsw: improve migration mechanism
> >   event/dsw: avoid migration waves in large systems
> >   event/dsw: remove redundant control ring poll
> >   event/dsw: remove unnecessary read barrier
> >   event/dsw: add port busy cycles xstats
>
> Series applied to dpdk-next-eventdev/master. Thanks.

I get a compilation issue on rhel7.
Too bad the CI did not help.
http://patchwork.dpdk.org/project/dpdk/list/?series=8828&state=*


[1583/1808] Compiling C object
'drivers/drivers@@tmp_rte_pmd_dsw_event@sta/event_dsw_dsw_event.c.o'.
../drivers/event/dsw/dsw_event.c: In function ‘dsw_port_consider_emigration’:
../drivers/event/dsw/dsw_event.c:502:27: warning: ‘candidate_qf’ may
be used uninitialized in this function [-Wmaybe-uninitialized]
  target_qfs[*targets_len] = *candidate_qf;
                           ^
../drivers/event/dsw/dsw_event.c:445:25: note: ‘candidate_qf’ was declared here
  struct dsw_queue_flow *candidate_qf;
                         ^
In file included from ../lib/librte_eal/x86/include/rte_atomic.h:16:0,
                 from ../lib/librte_eal/include/generic/rte_rwlock.h:25,
                 from ../lib/librte_eal/x86/include/rte_rwlock.h:12,
                 from ../lib/librte_eal/include/rte_fbarray.h:40,
                 from ../lib/librte_eal/include/rte_memory.h:25,
                 from ../lib/librte_eventdev/rte_event_ring.h:20,
                 from ../drivers/event/dsw/dsw_evdev.h:8,
                 from ../drivers/event/dsw/dsw_event.c:5:
../lib/librte_eal/include/generic/rte_atomic.h:566:22: warning:
‘candidate_flow_load’ may be used uninitialized in this function
[-Wmaybe-uninitialized]
  __sync_fetch_and_add(&v->cnt, inc);
                      ^
../drivers/event/dsw/dsw_event.c:448:10: note: ‘candidate_flow_load’
was declared here
  int16_t candidate_flow_load;
          ^
../drivers/event/dsw/dsw_event.c:505:49: warning: ‘candidate_port_id’
may be used uninitialized in this function [-Wmaybe-uninitialized]
  rte_atomic32_add(&dsw->ports[candidate_port_id].immigration_load,
                                                 ^
../drivers/event/dsw/dsw_event.c:446:10: note: ‘candidate_port_id’ was
declared here
  uint8_t candidate_port_id;
          ^
  

On 2020-04-15 18:37, David Marchand wrote:
> On Sat, Apr 4, 2020 at 4:35 PM Jerin Jacob Kollanukkaran
> <jerinj@marvell.com> wrote:
>>> -----Original Message-----
>>> From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>>> Sent: Monday, March 9, 2020 12:21 PM
>>> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
>>> Cc: dev@dpdk.org; stefan.sundkvist@ericsson.com; Ola.Liljedahl@arm.com;
>>> Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>>> Subject: [EXT] [PATCH 0/8] DSW performance and statistics improvements
>>>
>>> External Email
>>>
>>> ----------------------------------------------------------------------
>>> Performance and statistics improvements for the distributed software
>>> (DSW) event device.
>>>
>>> Mattias Rönnblom (8):
>>>    event/dsw: reduce latency in low-load situations
>>>    event/dsw: reduce max flows to speed up load balancing
>>>    event/dsw: extend statistics
>>>    event/dsw: improve migration mechanism
>>>    event/dsw: avoid migration waves in large systems
>>>    event/dsw: remove redundant control ring poll
>>>    event/dsw: remove unnecessary read barrier
>>>    event/dsw: add port busy cycles xstats
>> Series applied to dpdk-next-eventdev/master. Thanks.
> I get a compilation issue on rhel7.
> Too bad the CI did not help.
> https://protect2.fireeye.com/v1/url?k=6777ccba-3ba3c0e4-67778c21-8691959ed9b7-56149d0a5dee8fea&q=1&e=d034618d-5861-42ba-adf3-fc0aafd4892a&u=http%3A%2F%2Fpatchwork.dpdk.org%2Fproject%2Fdpdk%2Flist%2F%3Fseries%3D8828%26state%3D%2A
>
>
> [1583/1808] Compiling C object
> 'drivers/drivers@@tmp_rte_pmd_dsw_event@sta/event_dsw_dsw_event.c.o'.
> ../drivers/event/dsw/dsw_event.c: In function ‘dsw_port_consider_emigration’:
> ../drivers/event/dsw/dsw_event.c:502:27: warning: ‘candidate_qf’ may
> be used uninitialized in this function [-Wmaybe-uninitialized]
>    target_qfs[*targets_len] = *candidate_qf;
>                             ^
> ../drivers/event/dsw/dsw_event.c:445:25: note: ‘candidate_qf’ was declared here
>    struct dsw_queue_flow *candidate_qf;
>                           ^
> In file included from ../lib/librte_eal/x86/include/rte_atomic.h:16:0,
>                   from ../lib/librte_eal/include/generic/rte_rwlock.h:25,
>                   from ../lib/librte_eal/x86/include/rte_rwlock.h:12,
>                   from ../lib/librte_eal/include/rte_fbarray.h:40,
>                   from ../lib/librte_eal/include/rte_memory.h:25,
>                   from ../lib/librte_eventdev/rte_event_ring.h:20,
>                   from ../drivers/event/dsw/dsw_evdev.h:8,
>                   from ../drivers/event/dsw/dsw_event.c:5:
> ../lib/librte_eal/include/generic/rte_atomic.h:566:22: warning:
> ‘candidate_flow_load’ may be used uninitialized in this function
> [-Wmaybe-uninitialized]
>    __sync_fetch_and_add(&v->cnt, inc);
>                        ^
> ../drivers/event/dsw/dsw_event.c:448:10: note: ‘candidate_flow_load’
> was declared here
>    int16_t candidate_flow_load;
>            ^
> ../drivers/event/dsw/dsw_event.c:505:49: warning: ‘candidate_port_id’
> may be used uninitialized in this function [-Wmaybe-uninitialized]
>    rte_atomic32_add(&dsw->ports[candidate_port_id].immigration_load,
>                                                   ^
> ../drivers/event/dsw/dsw_event.c:446:10: note: ‘candidate_port_id’ was
> declared here
>    uint8_t candidate_port_id;
>            ^
>

Looks like a false positive. What GCC version is this?
  

15/04/2020 19:39, Mattias Rönnblom:
> On 2020-04-15 18:37, David Marchand wrote:
> > On Sat, Apr 4, 2020 at 4:35 PM Jerin Jacob Kollanukkaran
> > <jerinj@marvell.com> wrote:
> >> From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> >>> Performance and statistics improvements for the distributed software
> >>> (DSW) event device.
> >>>
> >>> Mattias Rönnblom (8):
> >>>    event/dsw: reduce latency in low-load situations
> >>>    event/dsw: reduce max flows to speed up load balancing
> >>>    event/dsw: extend statistics
> >>>    event/dsw: improve migration mechanism
> >>>    event/dsw: avoid migration waves in large systems
> >>>    event/dsw: remove redundant control ring poll
> >>>    event/dsw: remove unnecessary read barrier
> >>>    event/dsw: add port busy cycles xstats
> >> Series applied to dpdk-next-eventdev/master. Thanks.
> > 
> > I get a compilation issue on rhel7.
> > Too bad the CI did not help.
> > https://protect2.fireeye.com/v1/url?k=6777ccba-3ba3c0e4-67778c21-8691959ed9b7-56149d0a5dee8fea&q=1&e=d034618d-5861-42ba-adf3-fc0aafd4892a&u=http%3A%2F%2Fpatchwork.dpdk.org%2Fproject%2Fdpdk%2Flist%2F%3Fseries%3D8828%26state%3D%2A
> >
> >
> > [1583/1808] Compiling C object
> > 'drivers/drivers@@tmp_rte_pmd_dsw_event@sta/event_dsw_dsw_event.c.o'.
> > ../drivers/event/dsw/dsw_event.c: In function ‘dsw_port_consider_emigration’:
> > ../drivers/event/dsw/dsw_event.c:502:27: warning: ‘candidate_qf’ may
> > be used uninitialized in this function [-Wmaybe-uninitialized]
> >    target_qfs[*targets_len] = *candidate_qf;
> >                             ^
> > ../drivers/event/dsw/dsw_event.c:445:25: note: ‘candidate_qf’ was declared here
> >    struct dsw_queue_flow *candidate_qf;
> >                           ^
> > In file included from ../lib/librte_eal/x86/include/rte_atomic.h:16:0,
> >                   from ../lib/librte_eal/include/generic/rte_rwlock.h:25,
> >                   from ../lib/librte_eal/x86/include/rte_rwlock.h:12,
> >                   from ../lib/librte_eal/include/rte_fbarray.h:40,
> >                   from ../lib/librte_eal/include/rte_memory.h:25,
> >                   from ../lib/librte_eventdev/rte_event_ring.h:20,
> >                   from ../drivers/event/dsw/dsw_evdev.h:8,
> >                   from ../drivers/event/dsw/dsw_event.c:5:
> > ../lib/librte_eal/include/generic/rte_atomic.h:566:22: warning:
> > ‘candidate_flow_load’ may be used uninitialized in this function
> > [-Wmaybe-uninitialized]
> >    __sync_fetch_and_add(&v->cnt, inc);
> >                        ^
> > ../drivers/event/dsw/dsw_event.c:448:10: note: ‘candidate_flow_load’
> > was declared here
> >    int16_t candidate_flow_load;
> >            ^
> > ../drivers/event/dsw/dsw_event.c:505:49: warning: ‘candidate_port_id’
> > may be used uninitialized in this function [-Wmaybe-uninitialized]
> >    rte_atomic32_add(&dsw->ports[candidate_port_id].immigration_load,
> >                                                   ^
> > ../drivers/event/dsw/dsw_event.c:446:10: note: ‘candidate_port_id’ was
> > declared here
> >    uint8_t candidate_port_id;
> >            ^
> >
> 
> Looks like a false positive. What GCC version is this?

This is with RHEL 7.
Please do you have such distro to test and fix the false positive?
A quick fix would be very welcome.
  

On 2020-04-15 19:45, Thomas Monjalon wrote:
> 15/04/2020 19:39, Mattias Rönnblom:
>> On 2020-04-15 18:37, David Marchand wrote:
>>> On Sat, Apr 4, 2020 at 4:35 PM Jerin Jacob Kollanukkaran
>>> <jerinj@marvell.com> wrote:
>>>> From: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>>>>> Performance and statistics improvements for the distributed software
>>>>> (DSW) event device.
>>>>>
>>>>> Mattias Rönnblom (8):
>>>>>     event/dsw: reduce latency in low-load situations
>>>>>     event/dsw: reduce max flows to speed up load balancing
>>>>>     event/dsw: extend statistics
>>>>>     event/dsw: improve migration mechanism
>>>>>     event/dsw: avoid migration waves in large systems
>>>>>     event/dsw: remove redundant control ring poll
>>>>>     event/dsw: remove unnecessary read barrier
>>>>>     event/dsw: add port busy cycles xstats
>>>> Series applied to dpdk-next-eventdev/master. Thanks.
>>> I get a compilation issue on rhel7.
>>> Too bad the CI did not help.
>>> https://protect2.fireeye.com/v1/url?k=6777ccba-3ba3c0e4-67778c21-8691959ed9b7-56149d0a5dee8fea&q=1&e=d034618d-5861-42ba-adf3-fc0aafd4892a&u=http%3A%2F%2Fpatchwork.dpdk.org%2Fproject%2Fdpdk%2Flist%2F%3Fseries%3D8828%26state%3D%2A
>>>
>>>
>>> [1583/1808] Compiling C object
>>> 'drivers/drivers@@tmp_rte_pmd_dsw_event@sta/event_dsw_dsw_event.c.o'.
>>> ../drivers/event/dsw/dsw_event.c: In function ‘dsw_port_consider_emigration’:
>>> ../drivers/event/dsw/dsw_event.c:502:27: warning: ‘candidate_qf’ may
>>> be used uninitialized in this function [-Wmaybe-uninitialized]
>>>     target_qfs[*targets_len] = *candidate_qf;
>>>                              ^
>>> ../drivers/event/dsw/dsw_event.c:445:25: note: ‘candidate_qf’ was declared here
>>>     struct dsw_queue_flow *candidate_qf;
>>>                            ^
>>> In file included from ../lib/librte_eal/x86/include/rte_atomic.h:16:0,
>>>                    from ../lib/librte_eal/include/generic/rte_rwlock.h:25,
>>>                    from ../lib/librte_eal/x86/include/rte_rwlock.h:12,
>>>                    from ../lib/librte_eal/include/rte_fbarray.h:40,
>>>                    from ../lib/librte_eal/include/rte_memory.h:25,
>>>                    from ../lib/librte_eventdev/rte_event_ring.h:20,
>>>                    from ../drivers/event/dsw/dsw_evdev.h:8,
>>>                    from ../drivers/event/dsw/dsw_event.c:5:
>>> ../lib/librte_eal/include/generic/rte_atomic.h:566:22: warning:
>>> ‘candidate_flow_load’ may be used uninitialized in this function
>>> [-Wmaybe-uninitialized]
>>>     __sync_fetch_and_add(&v->cnt, inc);
>>>                         ^
>>> ../drivers/event/dsw/dsw_event.c:448:10: note: ‘candidate_flow_load’
>>> was declared here
>>>     int16_t candidate_flow_load;
>>>             ^
>>> ../drivers/event/dsw/dsw_event.c:505:49: warning: ‘candidate_port_id’
>>> may be used uninitialized in this function [-Wmaybe-uninitialized]
>>>     rte_atomic32_add(&dsw->ports[candidate_port_id].immigration_load,
>>>                                                    ^
>>> ../drivers/event/dsw/dsw_event.c:446:10: note: ‘candidate_port_id’ was
>>> declared here
>>>     uint8_t candidate_port_id;
>>>             ^
>>>
>> Looks like a false positive. What GCC version is this?
> This is with RHEL 7.
> Please do you have such distro to test and fix the false positive?
> A quick fix would be very welcome.
>
>

Most distributions support several compilers. I'm assuming it's the 
default 4.8 compiler, and I unfortunately don't have a system with that 
compiler.