From patchwork Mon Jun 1 06:09:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suanming Mou X-Patchwork-Id: 70708 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6FEAAA04EF; Mon, 1 Jun 2020 08:09:57 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4F6551D521; Mon, 1 Jun 2020 08:09:51 +0200 (CEST) Received: from git-send-mailer.rdmz.labs.mlnx (unknown [37.142.13.130]) by dpdk.org (Postfix) with ESMTP id BB4991C1C4 for ; Mon, 1 Jun 2020 08:09:49 +0200 (CEST) From: Suanming Mou To: viacheslavo@mellanox.com, matan@mellanox.com Cc: rasland@mellanox.com, dev@dpdk.org Date: Mon, 1 Jun 2020 14:09:42 +0800 Message-Id: <1590991783-326932-2-git-send-email-suanmingm@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1590991783-326932-1-git-send-email-suanmingm@mellanox.com> References: <1590991783-326932-1-git-send-email-suanmingm@mellanox.com> Subject: [dpdk-dev] [PATCH 1/2] common/mlx5: add memory reclaim glue function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" While flow destroyed, rdma-core may still cache some resources for more efficiently flow recreate. In case the peak time that millions of flows created and destroyed, the cached resources will be very huge. Currently, rdma-core proivdes the new function to configure the flow resources not to be cached. Add the memory reclaim function to avoid too many resources be cached. This is the first patch for the memory reclaim. A new devarg will be added to PMD to support the reclaim can be configured. Signed-off-by: Suanming Mou Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/Makefile | 5 +++++ drivers/common/mlx5/meson.build | 2 ++ drivers/common/mlx5/mlx5_glue.c | 13 +++++++++++++ drivers/common/mlx5/mlx5_glue.h | 1 + 4 files changed, 21 insertions(+) diff --git a/drivers/common/mlx5/Makefile b/drivers/common/mlx5/Makefile index 0d8cc1b..0d761cf 100644 --- a/drivers/common/mlx5/Makefile +++ b/drivers/common/mlx5/Makefile @@ -193,6 +193,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh func mlx5dv_alloc_var \ $(AUTOCONF_OUTPUT) $Q sh -- '$<' '$@' \ + HAVE_MLX5DV_DR_MEM_RECLAIM \ + infiniband/mlx5dv.h \ + func mlx5dv_dr_domain_set_reclaim_device_memory \ + $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ HAVE_ETHTOOL_LINK_MODE_25G \ /usr/include/linux/ethtool.h \ enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT \ diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build index 5a802ba..1ef040d 100644 --- a/drivers/common/mlx5/meson.build +++ b/drivers/common/mlx5/meson.build @@ -185,6 +185,8 @@ has_sym_args = [ 'RDMA_NLDEV_ATTR_NDEV_INDEX' ], [ 'HAVE_MLX5_DR_FLOW_DUMP', 'infiniband/mlx5dv.h', 'mlx5dv_dump_dr_domain'], + [ 'HAVE_MLX5DV_DR_MEM_RECLAIM', 'infiniband/mlx5dv.h', + 'mlx5dv_dr_domain_set_reclaim_device_memory'], [ 'HAVE_DEVLINK', 'linux/devlink.h', 'DEVLINK_GENL_NAME' ], ] config = configuration_data() diff --git a/drivers/common/mlx5/mlx5_glue.c b/drivers/common/mlx5/mlx5_glue.c index f270f67..e6d7ebe 100644 --- a/drivers/common/mlx5/mlx5_glue.c +++ b/drivers/common/mlx5/mlx5_glue.c @@ -1182,6 +1182,18 @@ #endif } + +static void +mlx5_glue_dr_reclaim_domain_memory(void *domain, uint32_t enable) +{ +#ifdef HAVE_MLX5DV_DR_MEM_RECLAIM + mlx5dv_dr_domain_set_reclaim_device_memory(domain, enable); +#else + (void)(enable); + (void)(domain); +#endif +} + __rte_cache_aligned const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) { .version = MLX5_GLUE_VERSION, @@ -1281,6 +1293,7 @@ .devx_qp_query = mlx5_glue_devx_qp_query, .devx_port_query = mlx5_glue_devx_port_query, .dr_dump_domain = mlx5_glue_dr_dump_domain, + .dr_reclaim_domain_memory = mlx5_glue_dr_reclaim_domain_memory, .devx_query_eqn = mlx5_glue_devx_query_eqn, .devx_create_event_channel = mlx5_glue_devx_create_event_channel, .devx_destroy_event_channel = mlx5_glue_devx_destroy_event_channel, diff --git a/drivers/common/mlx5/mlx5_glue.h b/drivers/common/mlx5/mlx5_glue.h index 81d6a22..5d238a4 100644 --- a/drivers/common/mlx5/mlx5_glue.h +++ b/drivers/common/mlx5/mlx5_glue.h @@ -302,6 +302,7 @@ struct mlx5_glue { (struct mlx5dv_devx_event_channel *event_channel, struct mlx5dv_devx_async_event_hdr *event_data, size_t event_resp_len); + void (*dr_reclaim_domain_memory)(void *domain, uint32_t enable); }; extern const struct mlx5_glue *mlx5_glue;