[RFC,0/5] Use correct memory ordering in eal functions

rte_eal_remote_launch and rte_eal_wait_lcore need to provide
correct memory ordering to address the data communication from
main core to worker core and vice versa.

There are 2 use cases:
1) All the store operations (meant for worker) by main core
should be visible to worker core before the worker core runs the
assigned function

2) All the store operations by the worker core should be visible
to the main core after rte_eal_wait_lcore returns.

For the data that needs to be communicated to the worker after
the rte_eal_remote_launch returns, load-acquire and store-release
semantics should be used (just like any other writer-reader use case).

For the main to worker communication, the pointer to function
to execute is used as the guard variable. Hence, resetting of
the function pointer is important.

For the worker to main communication, the existing code uses the
lcore state as the guard variable. However, it looks like
the FINISHED state is not really required. Hence the FINISHED state
is removed before using the state as the guard variable. I would like
some feedback on why the FINISHED state is required. I have not
paid attention to what it means for backward compatibility.
If it is decided to remove this state, documentation changes are
required.


Honnappa Nagarahalli (5):
  eal: reset lcore function pointer and argument
  eal: ensure memory operations are visible to worker
  eal: lcore state FINISHED is not required
  eal: ensure memory operations are visible to main
  test/ring: use relaxed barriers for ring stress test

 app/test/test_ring_stress_impl.h              | 18 ++++-----
 drivers/event/dpaa2/dpaa2_eventdev_selftest.c |  2 +-
 drivers/event/octeontx/ssovf_evdev_selftest.c |  2 +-
 drivers/event/sw/sw_evdev_selftest.c          |  4 +-
 examples/l2fwd-keepalive/main.c               |  2 +-
 lib/librte_eal/common/eal_common_launch.c     | 13 +++---
 lib/librte_eal/freebsd/eal_thread.c           | 31 +++++++++++---
 lib/librte_eal/linux/eal_thread.c             | 40 +++++++++++++------
 lib/librte_eal/windows/eal_thread.c           | 34 +++++++++++-----
 9 files changed, 94 insertions(+), 52 deletions(-)

On Wed, 24 Feb 2021 15:20:13 -0600
Honnappa Nagarahalli <honnappa.nagarahalli@arm.com> wrote:

> rte_eal_remote_launch and rte_eal_wait_lcore need to provide
> correct memory ordering to address the data communication from
> main core to worker core and vice versa.
> 
> There are 2 use cases:
> 1) All the store operations (meant for worker) by main core
> should be visible to worker core before the worker core runs the
> assigned function
> 
> 2) All the store operations by the worker core should be visible
> to the main core after rte_eal_wait_lcore returns.
> 
> For the data that needs to be communicated to the worker after
> the rte_eal_remote_launch returns, load-acquire and store-release
> semantics should be used (just like any other writer-reader use case).
> 
> For the main to worker communication, the pointer to function
> to execute is used as the guard variable. Hence, resetting of
> the function pointer is important.
> 
> For the worker to main communication, the existing code uses the
> lcore state as the guard variable. However, it looks like
> the FINISHED state is not really required. Hence the FINISHED state
> is removed before using the state as the guard variable. I would like
> some feedback on why the FINISHED state is required. I have not
> paid attention to what it means for backward compatibility.
> If it is decided to remove this state, documentation changes are
> required.
> 


Most likely all use of volatile in DPDK is suspect.
Perhaps we should re-enable the "volatile considered harmful" warning
in checkpatch

<snip>

> 
> > rte_eal_remote_launch and rte_eal_wait_lcore need to provide correct
> > memory ordering to address the data communication from main core to
> > worker core and vice versa.
> >
> > There are 2 use cases:
> > 1) All the store operations (meant for worker) by main core should be
> > visible to worker core before the worker core runs the assigned
> > function
> >
> > 2) All the store operations by the worker core should be visible to
> > the main core after rte_eal_wait_lcore returns.
> >
> > For the data that needs to be communicated to the worker after the
> > rte_eal_remote_launch returns, load-acquire and store-release
> > semantics should be used (just like any other writer-reader use case).
> >
> > For the main to worker communication, the pointer to function to
> > execute is used as the guard variable. Hence, resetting of the
> > function pointer is important.
> >
> > For the worker to main communication, the existing code uses the lcore
> > state as the guard variable. However, it looks like the FINISHED state
> > is not really required. Hence the FINISHED state is removed before
> > using the state as the guard variable. I would like some feedback on
> > why the FINISHED state is required. I have not paid attention to what
> > it means for backward compatibility.
> > If it is decided to remove this state, documentation changes are
> > required.
> >
> 
> 
> Most likely all use of volatile in DPDK is suspect.
> Perhaps we should re-enable the "volatile considered harmful" warning in
> checkpatch
Agree.