mbox series

[v2,00/10] net/mlx5: optimize flow resource allocation

Message ID 1587004928-328077-1-git-send-email-suanmingm@mellanox.com (mailing list archive)
Headers
Series net/mlx5: optimize flow resource allocation |

Message

Suanming Mou April 16, 2020, 2:41 a.m. UTC 
Currently, flow may allocate several action resources during creation.
Both allocate the individual resources and save the object handles have
big memory overhead for the flow.

Indexed memory pool is introduced to optimize the memory overhead. The
resources can be allocated from the resource's own individual memory pool
and only the 32-bits index is saved to the flow.

In this case, it saves MALLOC_ELEM_OVERHEAD and 4 bytes for every flow
resources.

This patch series depends on the path which has been acked as below:
https://patches.dpdk.org/patch/68143/

v2: 
- rebase to the lastet version.
- add the missing 10th patch.

Suanming Mou (10):
  net/mlx5: add indexed memory pool
  net/mlx5: add trunk dynamic grow for indexed pool
  net/mlx5: add trunk release for indexed pool
  net/mlx5: convert encap/decap resource to indexed
  net/mlx5: convert push VLAN resource to indexed
  net/mlx5: convert tag resource to indexed
  net/mlx5: convert port id action to indexed
  net/mlx5: convert jump resource to indexed
  net/mlx5: convert hrxq to indexed
  net/mlx5: convert flow dev handle to indexed

 drivers/net/mlx5/mlx5.c            | 148 ++++++++++++++-
 drivers/net/mlx5/mlx5.h            |  26 ++-
 drivers/net/mlx5/mlx5_flow.c       |  23 ++-
 drivers/net/mlx5/mlx5_flow.h       |  47 +++--
 drivers/net/mlx5/mlx5_flow_dv.c    | 271 +++++++++++++++++++---------
 drivers/net/mlx5/mlx5_flow_verbs.c |  53 ++++--
 drivers/net/mlx5/mlx5_rxq.c        |  47 +++--
 drivers/net/mlx5/mlx5_rxtx.h       |  22 +--
 drivers/net/mlx5/mlx5_utils.c      | 361 +++++++++++++++++++++++++++++++++++++
 drivers/net/mlx5/mlx5_utils.h      | 247 +++++++++++++++++++++++++
 10 files changed, 1071 insertions(+), 174 deletions(-)

Comments

Raslan Darawsheh April 16, 2020, 3:08 p.m. UTC | #1 
Hi,

> -----Original Message-----
> From: Suanming Mou <suanmingm@mellanox.com>
> Sent: Thursday, April 16, 2020 5:42 AM
> To: Slava Ovsiienko <viacheslavo@mellanox.com>; Matan Azrad
> <matan@mellanox.com>
> Cc: Raslan Darawsheh <rasland@mellanox.com>; dev@dpdk.org
> Subject: [PATCH v2 00/10] net/mlx5: optimize flow resource allocation
> 
> Currently, flow may allocate several action resources during creation.
> Both allocate the individual resources and save the object handles have
> big memory overhead for the flow.
> 
> Indexed memory pool is introduced to optimize the memory overhead. The
> resources can be allocated from the resource's own individual memory pool
> and only the 32-bits index is saved to the flow.
> 
> In this case, it saves MALLOC_ELEM_OVERHEAD and 4 bytes for every flow
> resources.
> 
> This patch series depends on the path which has been acked as below:
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatch
> es.dpdk.org%2Fpatch%2F68143%2F&amp;data=02%7C01%7Crasland%40mell
> anox.com%7C59e0bbb209a74266645008d7e1afc7ea%7Ca652971c7d2e4d9ba6
> a4d149256f461b%7C0%7C0%7C637226017419678693&amp;sdata=HlgoYtV3%
> 2BaC2XQn8TvEA9fskBqw4j8S8aawqizbJq5E%3D&amp;reserved=0
> 
> v2:
> - rebase to the lastet version.
> - add the missing 10th patch.
> 
> Suanming Mou (10):
>   net/mlx5: add indexed memory pool
>   net/mlx5: add trunk dynamic grow for indexed pool
>   net/mlx5: add trunk release for indexed pool
>   net/mlx5: convert encap/decap resource to indexed
>   net/mlx5: convert push VLAN resource to indexed
>   net/mlx5: convert tag resource to indexed
>   net/mlx5: convert port id action to indexed
>   net/mlx5: convert jump resource to indexed
>   net/mlx5: convert hrxq to indexed
>   net/mlx5: convert flow dev handle to indexed
> 
>  drivers/net/mlx5/mlx5.c            | 148 ++++++++++++++-
>  drivers/net/mlx5/mlx5.h            |  26 ++-
>  drivers/net/mlx5/mlx5_flow.c       |  23 ++-
>  drivers/net/mlx5/mlx5_flow.h       |  47 +++--
>  drivers/net/mlx5/mlx5_flow_dv.c    | 271 +++++++++++++++++++---------
>  drivers/net/mlx5/mlx5_flow_verbs.c |  53 ++++--
>  drivers/net/mlx5/mlx5_rxq.c        |  47 +++--
>  drivers/net/mlx5/mlx5_rxtx.h       |  22 +--
>  drivers/net/mlx5/mlx5_utils.c      | 361
> +++++++++++++++++++++++++++++++++++++
>  drivers/net/mlx5/mlx5_utils.h      | 247 +++++++++++++++++++++++++
>  10 files changed, 1071 insertions(+), 174 deletions(-)
> 
> --
> 1.8.3.1


Series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh
Ferruh Yigit April 17, 2020, 2:58 p.m. UTC | #2 
On 4/16/2020 3:41 AM, Suanming Mou wrote:
> Currently, flow may allocate several action resources during creation.
> Both allocate the individual resources and save the object handles have
> big memory overhead for the flow.
> 
> Indexed memory pool is introduced to optimize the memory overhead. The
> resources can be allocated from the resource's own individual memory pool
> and only the 32-bits index is saved to the flow.
> 
> In this case, it saves MALLOC_ELEM_OVERHEAD and 4 bytes for every flow
> resources.
> 
> This patch series depends on the path which has been acked as below:
> https://patches.dpdk.org/patch/68143/
> 
> v2: 
> - rebase to the lastet version.
> - add the missing 10th patch.
> 
> Suanming Mou (10):
>   net/mlx5: add indexed memory pool
>   net/mlx5: add trunk dynamic grow for indexed pool
>   net/mlx5: add trunk release for indexed pool
>   net/mlx5: convert encap/decap resource to indexed
>   net/mlx5: convert push VLAN resource to indexed
>   net/mlx5: convert tag resource to indexed
>   net/mlx5: convert port id action to indexed
>   net/mlx5: convert jump resource to indexed
>   net/mlx5: convert hrxq to indexed
>   net/mlx5: convert flow dev handle to indexed

Hi Suanming, Raslan,

There is a 32-bit build error [1], can you please check it?

[1]
.../drivers/net/mlx5/mlx5_utils.c: In function ‘mlx5_ipool_dump’:
.../drivers/net/mlx5/mlx5_utils.c:475:26: error: format ‘%ld’ expects argument
of type ‘long int’, but argument 3 has type ‘int64_t’ {aka ‘long long int’}
[-Werror=format=]
  475 |  printf("Pool %s entry %ld, trunk alloc %ld, empty: %ld, "
      |                        ~~^
      |                          |
      |                          long int
      |                        %lld
  476 |         "available %ld free %ld\n",
  477 |         pool->cfg.type, pool->n_entry, pool->trunk_new,
      |                         ~~~~~~~~~~~~~
      |                             |
      |                             int64_t {aka long long int}
.../drivers/net/mlx5/mlx5_utils.c:475:43: error: format ‘%ld’ expects argument
of type ‘long int’, but argument 4 has type ‘int64_t’ {aka ‘long long int’}
[-Werror=format=]
  475 |  printf("Pool %s entry %ld, trunk alloc %ld, empty: %ld, "
      |                                         ~~^
      |                                           |
      |                                           long int
      |                                         %lld
  476 |         "available %ld free %ld\n",
  477 |         pool->cfg.type, pool->n_entry, pool->trunk_new,
      |                                        ~~~~~~~~~~~~~~~
      |                                            |
      |                                            int64_t {aka long long int}
.../drivers/net/mlx5/mlx5_utils.c:475:55: error: format ‘%ld’ expects argument
of type ‘long int’, but argument 5 has type ‘int64_t’ {aka ‘long long int’}
[-Werror=format=]
  475 |  printf("Pool %s entry %ld, trunk alloc %ld, empty: %ld, "
      |                                                     ~~^
      |                                                       |
      |                                                       long int
      |                                                     %lld
......
  478 |         pool->trunk_empty, pool->trunk_avail, pool->trunk_free);
      |         ~~~~~~~~~~~~~~~~~
      |             |
      |             int64_t {aka long long int}
.../drivers/net/mlx5/mlx5_utils.c:475:9: error: format ‘%ld’ expects argument of
type ‘long int’, but argument 6 has type ‘int64_t’ {aka ‘long long int’}
[-Werror=format=]
  475 |  printf("Pool %s entry %ld, trunk alloc %ld, empty: %ld, "
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
......
  478 |         pool->trunk_empty, pool->trunk_avail, pool->trunk_free);
      |                            ~~~~~~~~~~~~~~~~~
      |                                |
      |                                int64_t {aka long long int}
.../drivers/net/mlx5/mlx5_utils.c:476:22: note: format string is defined here
  476 |         "available %ld free %ld\n",
      |                    ~~^
      |                      |
      |                      long int
      |                    %lld
.../drivers/net/mlx5/mlx5_utils.c:475:9: error: format ‘%ld’ expects argument of
type ‘long int’, but argument 7 has type ‘int64_t’ {aka ‘long long int’}
[-Werror=format=]
  475 |  printf("Pool %s entry %ld, trunk alloc %ld, empty: %ld, "
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
......
  478 |         pool->trunk_empty, pool->trunk_avail, pool->trunk_free);
      |                                               ~~~~~~~~~~~~~~~~
      |                                                   |
      |                                                   int64_t {aka long long
int}
      .../drivers/net/mlx5/mlx5_utils.c:476:31: note: format string is defined here
  476 |         "available %ld free %ld\n",
      |                             ~~^
      |                               |
      |                               long int
      |                             %lld
cc1: all warnings being treated as errors
Suanming Mou April 18, 2020, 1:46 a.m. UTC | #3 
Hi Ferruh,

Sorry that we missed the 32-bit build verification.
I have created the fix path.
https://patches.dpdk.org/patch/68831/

Thanks
SuanmingMou

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Friday, April 17, 2020 10:59 PM
> To: Suanming Mou <suanmingm@mellanox.com>; Raslan Darawsheh
> <rasland@mellanox.com>
> Cc: Slava Ovsiienko <viacheslavo@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2 00/10] net/mlx5: optimize flow resource
> allocation
> 
> On 4/16/2020 3:41 AM, Suanming Mou wrote:
> > Currently, flow may allocate several action resources during creation.
> > Both allocate the individual resources and save the object handles
> > have big memory overhead for the flow.
> >
> > Indexed memory pool is introduced to optimize the memory overhead. The
> > resources can be allocated from the resource's own individual memory
> > pool and only the 32-bits index is saved to the flow.
> >
> > In this case, it saves MALLOC_ELEM_OVERHEAD and 4 bytes for every flow
> > resources.
> >
> > This patch series depends on the path which has been acked as below:
> > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatc
> >
> hes.dpdk.org%2Fpatch%2F68143%2F&amp;data=02%7C01%7Csuanmingm%40
> mellano
> >
> x.com%7Caf4087bccbb442edf12708d7e2dfd396%7Ca652971c7d2e4d9ba6a4d1
> 49256
> >
> f461b%7C0%7C0%7C637227323276924116&amp;sdata=i09jTFpq%2FUGRvhBi%
> 2BpJei
> > nBZeBm2LyE528UQ%2F7L1rWY%3D&amp;reserved=0
> >
> > v2:
> > - rebase to the lastet version.
> > - add the missing 10th patch.
> >
> > Suanming Mou (10):
> >   net/mlx5: add indexed memory pool
> >   net/mlx5: add trunk dynamic grow for indexed pool
> >   net/mlx5: add trunk release for indexed pool
> >   net/mlx5: convert encap/decap resource to indexed
> >   net/mlx5: convert push VLAN resource to indexed
> >   net/mlx5: convert tag resource to indexed
> >   net/mlx5: convert port id action to indexed
> >   net/mlx5: convert jump resource to indexed
> >   net/mlx5: convert hrxq to indexed
> >   net/mlx5: convert flow dev handle to indexed
> 
> Hi Suanming, Raslan,
> 
> There is a 32-bit build error [1], can you please check it?
> 
> [1]
> .../drivers/net/mlx5/mlx5_utils.c: In function ‘mlx5_ipool_dump’:
> .../drivers/net/mlx5/mlx5_utils.c:475:26: error: format ‘%ld’ expects argument
> of type ‘long int’, but argument 3 has type ‘int64_t’ {aka ‘long long int’} [-
> Werror=format=]
>   475 |  printf("Pool %s entry %ld, trunk alloc %ld, empty: %ld, "
>       |                        ~~^
>       |                          |
>       |                          long int
>       |                        %lld
>   476 |         "available %ld free %ld\n",
>   477 |         pool->cfg.type, pool->n_entry, pool->trunk_new,
>       |                         ~~~~~~~~~~~~~
>       |                             |
>       |                             int64_t {aka long long int}
> .../drivers/net/mlx5/mlx5_utils.c:475:43: error: format ‘%ld’ expects argument
> of type ‘long int’, but argument 4 has type ‘int64_t’ {aka ‘long long int’} [-
> Werror=format=]
>   475 |  printf("Pool %s entry %ld, trunk alloc %ld, empty: %ld, "
>       |                                         ~~^
>       |                                           |
>       |                                           long int
>       |                                         %lld
>   476 |         "available %ld free %ld\n",
>   477 |         pool->cfg.type, pool->n_entry, pool->trunk_new,
>       |                                        ~~~~~~~~~~~~~~~
>       |                                            |
>       |                                            int64_t {aka long long int}
> .../drivers/net/mlx5/mlx5_utils.c:475:55: error: format ‘%ld’ expects argument
> of type ‘long int’, but argument 5 has type ‘int64_t’ {aka ‘long long int’} [-
> Werror=format=]
>   475 |  printf("Pool %s entry %ld, trunk alloc %ld, empty: %ld, "
>       |                                                     ~~^
>       |                                                       |
>       |                                                       long int
>       |                                                     %lld
> ......
>   478 |         pool->trunk_empty, pool->trunk_avail, pool->trunk_free);
>       |         ~~~~~~~~~~~~~~~~~
>       |             |
>       |             int64_t {aka long long int}
> .../drivers/net/mlx5/mlx5_utils.c:475:9: error: format ‘%ld’ expects argument of
> type ‘long int’, but argument 6 has type ‘int64_t’ {aka ‘long long int’} [-
> Werror=format=]
>   475 |  printf("Pool %s entry %ld, trunk alloc %ld, empty: %ld, "
>       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ......
>   478 |         pool->trunk_empty, pool->trunk_avail, pool->trunk_free);
>       |                            ~~~~~~~~~~~~~~~~~
>       |                                |
>       |                                int64_t {aka long long int}
> .../drivers/net/mlx5/mlx5_utils.c:476:22: note: format string is defined here
>   476 |         "available %ld free %ld\n",
>       |                    ~~^
>       |                      |
>       |                      long int
>       |                    %lld
> .../drivers/net/mlx5/mlx5_utils.c:475:9: error: format ‘%ld’ expects argument of
> type ‘long int’, but argument 7 has type ‘int64_t’ {aka ‘long long int’} [-
> Werror=format=]
>   475 |  printf("Pool %s entry %ld, trunk alloc %ld, empty: %ld, "
>       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ......
>   478 |         pool->trunk_empty, pool->trunk_avail, pool->trunk_free);
>       |                                               ~~~~~~~~~~~~~~~~
>       |                                                   |
>       |                                                   int64_t {aka long long
> int}
>       .../drivers/net/mlx5/mlx5_utils.c:476:31: note: format string is defined here
>   476 |         "available %ld free %ld\n",
>       |                             ~~^
>       |                               |
>       |                               long int
>       |                             %lld
> cc1: all warnings being treated as errors