[v3,00/14] Make shared memory config non-public
mbox series

Message ID cover.1561635481.git.anatoly.burakov@intel.com
Headers show
Series
  • Make shared memory config non-public
Related show

Message

Burakov, Anatoly June 27, 2019, 11:38 a.m. UTC
This patchset removes the shared memory config from public
API, and replaces all usages of said config with new API
calls.

A lot of the patchset is a search-and-replace job and should
be pretty easy to review. The rest are pretty trivial EAL
changes.

This patchset depends on FreeBSD fixes patchset:

http://patches.dpdk.org/project/dpdk/list/?series=5196

v3:
- Rebase on top of latest master

v2:
- Collapsed all changes into fewer patches
- Addressed review comments
- Created a new file to store the code
- Changed namespace to "rte_mcfg_"
- Added some unification around config init
- Removed "packed" attribute from mem config
- Removed unnecessary inlining
- Added a check to explicitly forbid running multiprocess
  applications that differ in their DPDK versions

Anatoly Burakov (14):
  eal: add API to lock/unlock memory hotplug
  drivers: use new memory locking API
  lib: use new memory locking API
  eal: add EAL tailq list lock/unlock API
  lib: use new tailq locking API
  eal: add new API to lock/unlock mempool list
  mempool: use new mempool list locking API
  eal: remove unused macros
  eal: hide shared memory config
  eal: remove packed attribute from mcfg structure
  eal: uninline wait for mcfg complete function
  eal: unify and move mcfg complete function
  eal: unify internal config initialization
  eal: prevent different primary/secondary process versions

 app/test/test_memzone.c                       |   1 +
 app/test/test_tailq.c                         |   1 +
 doc/guides/rel_notes/deprecation.rst          |   3 -
 doc/guides/rel_notes/release_19_08.rst        |   8 +-
 drivers/bus/fslmc/fslmc_vfio.c                |   8 +-
 drivers/bus/pci/linux/pci_vfio.c              |   1 +
 drivers/net/mlx4/mlx4_mr.c                    |  11 +-
 drivers/net/mlx5/mlx5_mr.c                    |  11 +-
 .../net/virtio/virtio_user/virtio_user_dev.c  |   7 +-
 lib/librte_acl/rte_acl.c                      |  20 +--
 lib/librte_distributor/rte_distributor.c      |   5 +-
 lib/librte_distributor/rte_distributor_v20.c  |   5 +-
 lib/librte_eal/common/eal_common_mcfg.c       | 149 ++++++++++++++++++
 lib/librte_eal/common/eal_common_memory.c     |  44 +++---
 lib/librte_eal/common/eal_common_memzone.c    |   1 +
 lib/librte_eal/common/eal_common_tailqs.c     |   5 +-
 lib/librte_eal/common/eal_memcfg.h            |  93 +++++++++++
 lib/librte_eal/common/include/rte_eal.h       |  10 --
 .../common/include/rte_eal_memconfig.h        | 143 +++++++++--------
 lib/librte_eal/common/malloc_heap.c           |  16 +-
 lib/librte_eal/common/malloc_mp.c             |   1 +
 lib/librte_eal/common/meson.build             |   1 +
 lib/librte_eal/common/rte_malloc.c            |  33 ++--
 lib/librte_eal/freebsd/eal/Makefile           |   3 +-
 lib/librte_eal/freebsd/eal/eal.c              |  22 ++-
 lib/librte_eal/freebsd/eal/eal_memory.c       |   1 +
 lib/librte_eal/linux/eal/Makefile             |   3 +-
 lib/librte_eal/linux/eal/eal.c                |  42 ++---
 lib/librte_eal/linux/eal/eal_memalloc.c       |   1 +
 lib/librte_eal/linux/eal/eal_memory.c         |   1 +
 lib/librte_eal/linux/eal/eal_vfio.c           |  17 +-
 lib/librte_eal/meson.build                    |   2 +-
 lib/librte_eal/rte_eal_version.map            |  12 ++
 lib/librte_efd/rte_efd.c                      |  15 +-
 lib/librte_eventdev/rte_event_ring.c          |  16 +-
 lib/librte_hash/rte_cuckoo_hash.c             |  17 +-
 lib/librte_hash/rte_fbk_hash.c                |  15 +-
 lib/librte_kni/rte_kni.c                      |  16 +-
 lib/librte_lpm/rte_lpm.c                      |  25 +--
 lib/librte_lpm/rte_lpm6.c                     |  15 +-
 lib/librte_member/rte_member.c                |  17 +-
 lib/librte_mempool/rte_mempool.c              |  27 ++--
 lib/librte_reorder/rte_reorder.c              |  15 +-
 lib/librte_ring/rte_ring.c                    |  19 +--
 lib/librte_stack/rte_stack.c                  |  18 +--
 45 files changed, 566 insertions(+), 330 deletions(-)
 create mode 100644 lib/librte_eal/common/eal_common_mcfg.c
 create mode 100644 lib/librte_eal/common/eal_memcfg.h

Comments

Stephen Hemminger June 27, 2019, 3:36 p.m. UTC | #1
On Thu, 27 Jun 2019 12:38:55 +0100
Anatoly Burakov <anatoly.burakov@intel.com> wrote:

> This patchset removes the shared memory config from public
> API, and replaces all usages of said config with new API
> calls.
> 
> A lot of the patchset is a search-and-replace job and should
> be pretty easy to review. The rest are pretty trivial EAL
> changes.
> 
> This patchset depends on FreeBSD fixes patchset:
> 
> http://patches.dpdk.org/project/dpdk/list/?series=5196
> 
> v3:
> - Rebase on top of latest master
> 
> v2:
> - Collapsed all changes into fewer patches
> - Addressed review comments
> - Created a new file to store the code
> - Changed namespace to "rte_mcfg_"
> - Added some unification around config init
> - Removed "packed" attribute from mem config
> - Removed unnecessary inlining
> - Added a check to explicitly forbid running multiprocess
>   applications that differ in their DPDK versions
> 
> Anatoly Burakov (14):
>   eal: add API to lock/unlock memory hotplug
>   drivers: use new memory locking API
>   lib: use new memory locking API
>   eal: add EAL tailq list lock/unlock API
>   lib: use new tailq locking API
>   eal: add new API to lock/unlock mempool list
>   mempool: use new mempool list locking API
>   eal: remove unused macros
>   eal: hide shared memory config
>   eal: remove packed attribute from mcfg structure
>   eal: uninline wait for mcfg complete function
>   eal: unify and move mcfg complete function
>   eal: unify internal config initialization
>   eal: prevent different primary/secondary process versions
> 
>  app/test/test_memzone.c                       |   1 +
>  app/test/test_tailq.c                         |   1 +
>  doc/guides/rel_notes/deprecation.rst          |   3 -
>  doc/guides/rel_notes/release_19_08.rst        |   8 +-
>  drivers/bus/fslmc/fslmc_vfio.c                |   8 +-
>  drivers/bus/pci/linux/pci_vfio.c              |   1 +
>  drivers/net/mlx4/mlx4_mr.c                    |  11 +-
>  drivers/net/mlx5/mlx5_mr.c                    |  11 +-
>  .../net/virtio/virtio_user/virtio_user_dev.c  |   7 +-
>  lib/librte_acl/rte_acl.c                      |  20 +--
>  lib/librte_distributor/rte_distributor.c      |   5 +-
>  lib/librte_distributor/rte_distributor_v20.c  |   5 +-
>  lib/librte_eal/common/eal_common_mcfg.c       | 149 ++++++++++++++++++
>  lib/librte_eal/common/eal_common_memory.c     |  44 +++---
>  lib/librte_eal/common/eal_common_memzone.c    |   1 +
>  lib/librte_eal/common/eal_common_tailqs.c     |   5 +-
>  lib/librte_eal/common/eal_memcfg.h            |  93 +++++++++++
>  lib/librte_eal/common/include/rte_eal.h       |  10 --
>  .../common/include/rte_eal_memconfig.h        | 143 +++++++++--------
>  lib/librte_eal/common/malloc_heap.c           |  16 +-
>  lib/librte_eal/common/malloc_mp.c             |   1 +
>  lib/librte_eal/common/meson.build             |   1 +
>  lib/librte_eal/common/rte_malloc.c            |  33 ++--
>  lib/librte_eal/freebsd/eal/Makefile           |   3 +-
>  lib/librte_eal/freebsd/eal/eal.c              |  22 ++-
>  lib/librte_eal/freebsd/eal/eal_memory.c       |   1 +
>  lib/librte_eal/linux/eal/Makefile             |   3 +-
>  lib/librte_eal/linux/eal/eal.c                |  42 ++---
>  lib/librte_eal/linux/eal/eal_memalloc.c       |   1 +
>  lib/librte_eal/linux/eal/eal_memory.c         |   1 +
>  lib/librte_eal/linux/eal/eal_vfio.c           |  17 +-
>  lib/librte_eal/meson.build                    |   2 +-
>  lib/librte_eal/rte_eal_version.map            |  12 ++
>  lib/librte_efd/rte_efd.c                      |  15 +-
>  lib/librte_eventdev/rte_event_ring.c          |  16 +-
>  lib/librte_hash/rte_cuckoo_hash.c             |  17 +-
>  lib/librte_hash/rte_fbk_hash.c                |  15 +-
>  lib/librte_kni/rte_kni.c                      |  16 +-
>  lib/librte_lpm/rte_lpm.c                      |  25 +--
>  lib/librte_lpm/rte_lpm6.c                     |  15 +-
>  lib/librte_member/rte_member.c                |  17 +-
>  lib/librte_mempool/rte_mempool.c              |  27 ++--
>  lib/librte_reorder/rte_reorder.c              |  15 +-
>  lib/librte_ring/rte_ring.c                    |  19 +--
>  lib/librte_stack/rte_stack.c                  |  18 +--
>  45 files changed, 566 insertions(+), 330 deletions(-)
>  create mode 100644 lib/librte_eal/common/eal_common_mcfg.c
>  create mode 100644 lib/librte_eal/common/eal_memcfg.h
> 

This looks good thanks.

Acked-by: Stephen Hemminger <stephen@networkplumber.org>
David Marchand July 3, 2019, 9:38 a.m. UTC | #2
Hello Anatoly,

On Thu, Jun 27, 2019 at 1:39 PM Anatoly Burakov <anatoly.burakov@intel.com>
wrote:

> This patchset removes the shared memory config from public
> API, and replaces all usages of said config with new API
> calls.
>
> A lot of the patchset is a search-and-replace job and should
> be pretty easy to review. The rest are pretty trivial EAL
> changes.
>
> This patchset depends on FreeBSD fixes patchset:
>
> http://patches.dpdk.org/project/dpdk/list/?series=5196
>
> v3:
> - Rebase on top of latest master
>
> v2:
> - Collapsed all changes into fewer patches
> - Addressed review comments
> - Created a new file to store the code
> - Changed namespace to "rte_mcfg_"
> - Added some unification around config init
> - Removed "packed" attribute from mem config
> - Removed unnecessary inlining
> - Added a check to explicitly forbid running multiprocess
>   applications that differ in their DPDK versions
>


For the parts I already had a look at, I still think the changes are in too
many patches.
A lot of this is just search/replace we can have it with the patch that
introduces it.
- patch 1, 2 and 3 could be squashed as a single one (plus removing the
unused macro from patch 8)
- idem with patch 4 and 5
- idem with patch 6 and 7 (plus removing the unused macro from patch 8)


I will look at the rest tomorrow (hopefully).
Burakov, Anatoly July 3, 2019, 10:47 a.m. UTC | #3
On 03-Jul-19 10:38 AM, David Marchand wrote:
> Hello Anatoly,
> 
> On Thu, Jun 27, 2019 at 1:39 PM Anatoly Burakov 
> <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
> 
>     This patchset removes the shared memory config from public
>     API, and replaces all usages of said config with new API
>     calls.
> 
>     A lot of the patchset is a search-and-replace job and should
>     be pretty easy to review. The rest are pretty trivial EAL
>     changes.
> 
>     This patchset depends on FreeBSD fixes patchset:
> 
>     http://patches.dpdk.org/project/dpdk/list/?series=5196
> 
>     v3:
>     - Rebase on top of latest master
> 
>     v2:
>     - Collapsed all changes into fewer patches
>     - Addressed review comments
>     - Created a new file to store the code
>     - Changed namespace to "rte_mcfg_"
>     - Added some unification around config init
>     - Removed "packed" attribute from mem config
>     - Removed unnecessary inlining
>     - Added a check to explicitly forbid running multiprocess
>        applications that differ in their DPDK versions
> 
> 
> 
> For the parts I already had a look at, I still think the changes are in 
> too many patches.
> A lot of this is just search/replace we can have it with the patch that 
> introduces it.
> - patch 1, 2 and 3 could be squashed as a single one (plus removing the 
> unused macro from patch 8)
> - idem with patch 4 and 5
> - idem with patch 6 and 7 (plus removing the unused macro from patch 8)
> 
> 
> I will look at the rest tomorrow (hopefully).
> 
> -- 
> David Marchand

Sure, i can squash it down into single patches for search/replace if 
that's the preference.
David Marchand July 4, 2019, 8:09 a.m. UTC | #4
On Wed, Jul 3, 2019 at 11:38 AM David Marchand <david.marchand@redhat.com>
wrote:

> Hello Anatoly,
>
> On Thu, Jun 27, 2019 at 1:39 PM Anatoly Burakov <anatoly.burakov@intel.com>
> wrote:
>
>> This patchset removes the shared memory config from public
>> API, and replaces all usages of said config with new API
>> calls.
>>
>> A lot of the patchset is a search-and-replace job and should
>> be pretty easy to review. The rest are pretty trivial EAL
>> changes.
>>
>> This patchset depends on FreeBSD fixes patchset:
>>
>> http://patches.dpdk.org/project/dpdk/list/?series=5196
>>
>> v3:
>> - Rebase on top of latest master
>>
>> v2:
>> - Collapsed all changes into fewer patches
>> - Addressed review comments
>> - Created a new file to store the code
>> - Changed namespace to "rte_mcfg_"
>> - Added some unification around config init
>> - Removed "packed" attribute from mem config
>> - Removed unnecessary inlining
>> - Added a check to explicitly forbid running multiprocess
>>   applications that differ in their DPDK versions
>>
>
>
> For the parts I already had a look at, I still think the changes are in
> too many patches.
> A lot of this is just search/replace we can have it with the patch that
> introduces it.
> - patch 1, 2 and 3 could be squashed as a single one (plus removing the
> unused macro from patch 8)
> - idem with patch 4 and 5
> - idem with patch 6 and 7 (plus removing the unused macro from patch 8)
>
>
Overall, I am ok with the changes, once the patch 13 is fixed.
You can add my ack on the n+1 patchset.


I just want to state two approaches to merge these changes:
- the first one would be to split this series in two
  - take the first part of this series now, but mark the new API
"experimental"
  - postpone the merge to 19.11 of the second part, which starts at the
hiding rte_mem_config patch
- the second one is taking these changes in one go

The second one is the quicker and the more straightforward but it leaves
the risk of having missed something and we must break the ABI again in
19.11.
The risk is quite low, given the changes.


Thomas, comments?
Thomas Monjalon July 4, 2019, 7:52 p.m. UTC | #5
04/07/2019 10:09, David Marchand:
> I just want to state two approaches to merge these changes:
> - the first one would be to split this series in two
>   - take the first part of this series now, but mark the new API
> "experimental"
>   - postpone the merge to 19.11 of the second part, which starts at the
> hiding rte_mem_config patch
> - the second one is taking these changes in one go
> 
> The second one is the quicker and the more straightforward but it leaves
> the risk of having missed something and we must break the ABI again in
> 19.11.
> The risk is quite low, given the changes.
> 
> 
> Thomas, comments?

OK to merge it in one go.
Please, can we merge it tomorrow?