From patchwork Wed Jun 30 12:46:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suanming Mou X-Patchwork-Id: 95069 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D083EA0A0F; Wed, 30 Jun 2021 14:48:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 98FC7412C1; Wed, 30 Jun 2021 14:47:07 +0200 (CEST) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam07on2077.outbound.protection.outlook.com [40.107.95.77]) by mails.dpdk.org (Postfix) with ESMTP id 23E69412B8 for ; Wed, 30 Jun 2021 14:47:03 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NbgGMRtlEGfwP0y66k7YOl+XkZfxIEnVm2Bm+p3UF+Hp6fJeNAjW/2QhpJLTWSxMK3/OSiFBfvlStj9h07OzPQh0JoUwLoBni/iYDncXLVnpWGiVgLqBwJZsoO/4NbPTK5wldMoYE+9DoyvLuQsyBqe3tdSUZesGrERzHWjDQgZLBVM93cWAD767PW2u4d0yOKC+9unRxgQ98Prqd2tZEkVnvSnVWPMkvT58B5tf76WiHCGnFRJ6olilbLN9KsctK2N9nelMtU0Stnkc8bfrSqvACRIgGW8/HZbEBcW6URWeuXqr4xJw58HLmt3UCz9Hlh3lP4qzCOawnnh5P6xMJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qzFVjkAmlyCa27Nxj0vVHsaK+lAARd7x7LftF41MS7I=; b=fXynIRyL+MJKFBBDyyoGagSXOzCeWPojwla8Qs9cLlOI43o1g2cF4IxQ95uj3NR6moSMVc9/y+bamVBApuiudPY60sRywey94EAnFtIwtgtmk1kuRTgBhUjQaDhf7One3VYKcpUOA243St+ozhenea7/tMdz/5PPNcaDfgYURqy86VaaITH8PO8IMtGnxXktoDlcPhr5cl3LnXsarqUwplJimsns8e+wV1V6mT5fR2jWM/oS75ytr3d1DIdJO0saObRSc1MOgn9O4QemGq6r1WNEFgIMvXu5j4Ps2o+YTGR50+3rvGOTc1CybzE1baKeu+ranvw4Tnvd3M1qHU632g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qzFVjkAmlyCa27Nxj0vVHsaK+lAARd7x7LftF41MS7I=; b=ac+co7hoTTdCbTXAxLA1acPdEdeT1l1dewUimpm6VKbZwckfOchaeFHmUrnt5gy14skqFpmMlCz9EK9wW1ZiwHEx9fz4uO/rqTFZc31WVApjqXaHJhsdynS6iQfQZR6OK8e8kTMePa4rF+MAjtmEtvU5ORTpS4Mco54rY0PqIrkvQUmyuY6GXnlfJ9IHfXiY7awUuVE28L9LiBIYSjCRsyB0SR/H8i1ZVzjZnkwa91skiSpB4OB/PPDm1EdYP2cceDUp0yHB0zHPUtHGWAIZifdwhpeeC6oaBXs0Kr35KXgPxjF9MCimiDIdDmmY45JTTWwQm5q+Aid49HtB6oM1vg== Received: from BN6PR1701CA0021.namprd17.prod.outlook.com (2603:10b6:405:15::31) by DM5PR1201MB2488.namprd12.prod.outlook.com (2603:10b6:3:e1::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.23; Wed, 30 Jun 2021 12:47:01 +0000 Received: from BN8NAM11FT045.eop-nam11.prod.protection.outlook.com (2603:10b6:405:15:cafe::d6) by BN6PR1701CA0021.outlook.office365.com (2603:10b6:405:15::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.21 via Frontend Transport; Wed, 30 Jun 2021 12:47:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT045.mail.protection.outlook.com (10.13.177.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Wed, 30 Jun 2021 12:47:00 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 30 Jun 2021 12:46:53 +0000 From: Suanming Mou To: , CC: , , Date: Wed, 30 Jun 2021 15:46:02 +0300 Message-ID: <20210630124609.8711-16-suanmingm@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20210630124609.8711-1-suanmingm@nvidia.com> References: <20210527093403.1153127-1-suanmingm@nvidia.com> <20210630124609.8711-1-suanmingm@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9c43cfca-fce2-4517-1449-08d93bc527b5 X-MS-TrafficTypeDiagnostic: DM5PR1201MB2488: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:71; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: I23Fm+vdkvsWjrgbKxIOUd3TrWvo2DCQaAKgXxCBN9veKb+8pNO4DEHDqZ6BZFSAfQ95tYtXXEAYAPmkRRYTY6UrXkooUTJLl/hEIbsCxdas3v3lo4XY3PDVmEklh5KnuS9Tupzy0o9hfe28a3G9ZbyMRqsjkK0DiOylLHvY0ohtWEQ6x1QQIuCMGAfYraLekp81R2bzRR0xZNrr7SdJwUpBllvf72MXk1mz2xaNEGCwgJBSO/PA9P5k7aMNBzHU1KYavC94yu5wbTg76iQE+UCzhsbA6pHWBNZP0oDw5Jtl0YWaQkAzm6grpmmSgAA6qrsXTiL5iaXMg0LsVKzjnVJ+PaNkE+3jOZ3ay0tlzzSESwXpo7/kU7m5bEfTZRBMSBc/yfY7uAuMFXLGc28pQOt+n6VBrY6DJV2s/erAXJXOhqXdU6G6zsz2yc4eIBrT+rfK9FlH15f08hrqDUs5IeM6roISqN7zJ5UE5IMiabzKz/92LFFwDu6+Y55r+Sv8MeqC8KgjrGSurO5+9GL49u/wdgKNAxhTPKBacN0Xw2BKHT88OAVG1HT6kTlFAySelrf+qT6vytu5MOTEDTJP4YlNEpx6y1qrQtg49kdYaCDsNJN/ZED+lhiCaXJcsEM5HXPcQzFCe+wsNIhlJU7df3idGgicZT93fgt++0KQkHsD4wzi0hYe/0eLfd6HZ4ilcmXAm5IkJJL148M39pwsyQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(376002)(396003)(136003)(346002)(36840700001)(46966006)(6636002)(5660300002)(4326008)(70586007)(478600001)(8936002)(7696005)(70206006)(2906002)(316002)(54906003)(110136005)(6286002)(1076003)(82740400003)(47076005)(356005)(86362001)(7636003)(36860700001)(36756003)(83380400001)(82310400003)(55016002)(26005)(2616005)(16526019)(186003)(6666004)(426003)(336012)(8676002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jun 2021 12:47:00.8341 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9c43cfca-fce2-4517-1449-08d93bc527b5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT045.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB2488 Subject: [dpdk-dev] [PATCH v2 15/22] common/mlx5: allocate cache list memory individually X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, the list's local cache instance memory is allocated with the list. As the local cache instance array size is RTE_MAX_LCORE, most of the cases the system will only have very limited cores. allocate the instance memory individually per core will be more economic to the memory. This commit changes the instance array to pointer array, allocate the local cache memory only when the core is to be used. Signed-off-by: Suanming Mou Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common_utils.c | 62 ++++++++++++++++++------- drivers/common/mlx5/mlx5_common_utils.h | 2 +- 2 files changed, 45 insertions(+), 19 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common_utils.c b/drivers/common/mlx5/mlx5_common_utils.c index 4e385c616a..f75b1cb0da 100644 --- a/drivers/common/mlx5/mlx5_common_utils.c +++ b/drivers/common/mlx5/mlx5_common_utils.c @@ -15,14 +15,13 @@ static int mlx5_list_init(struct mlx5_list *list, const char *name, void *ctx, - bool lcores_share, mlx5_list_create_cb cb_create, + bool lcores_share, struct mlx5_list_cache *gc, + mlx5_list_create_cb cb_create, mlx5_list_match_cb cb_match, mlx5_list_remove_cb cb_remove, mlx5_list_clone_cb cb_clone, mlx5_list_clone_free_cb cb_clone_free) { - int i; - if (!cb_match || !cb_create || !cb_remove || !cb_clone || !cb_clone_free) { rte_errno = EINVAL; @@ -38,9 +37,11 @@ mlx5_list_init(struct mlx5_list *list, const char *name, void *ctx, list->cb_clone = cb_clone; list->cb_clone_free = cb_clone_free; rte_rwlock_init(&list->lock); + if (lcores_share) { + list->cache[RTE_MAX_LCORE] = gc; + LIST_INIT(&list->cache[RTE_MAX_LCORE]->h); + } DRV_LOG(DEBUG, "mlx5 list %s initialized.", list->name); - for (i = 0; i <= RTE_MAX_LCORE; i++) - LIST_INIT(&list->cache[i].h); return 0; } @@ -53,11 +54,16 @@ mlx5_list_create(const char *name, void *ctx, bool lcores_share, mlx5_list_clone_free_cb cb_clone_free) { struct mlx5_list *list; + struct mlx5_list_cache *gc = NULL; - list = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*list), 0, SOCKET_ID_ANY); + list = mlx5_malloc(MLX5_MEM_ZERO, + sizeof(*list) + (lcores_share ? sizeof(*gc) : 0), + 0, SOCKET_ID_ANY); if (!list) return NULL; - if (mlx5_list_init(list, name, ctx, lcores_share, + if (lcores_share) + gc = (struct mlx5_list_cache *)(list + 1); + if (mlx5_list_init(list, name, ctx, lcores_share, gc, cb_create, cb_match, cb_remove, cb_clone, cb_clone_free) != 0) { mlx5_free(list); @@ -69,7 +75,8 @@ mlx5_list_create(const char *name, void *ctx, bool lcores_share, static struct mlx5_list_entry * __list_lookup(struct mlx5_list *list, int lcore_index, void *ctx, bool reuse) { - struct mlx5_list_entry *entry = LIST_FIRST(&list->cache[lcore_index].h); + struct mlx5_list_entry *entry = + LIST_FIRST(&list->cache[lcore_index]->h); uint32_t ret; while (entry != NULL) { @@ -121,14 +128,14 @@ mlx5_list_cache_insert(struct mlx5_list *list, int lcore_index, lentry->ref_cnt = 1u; lentry->gentry = gentry; lentry->lcore_idx = (uint32_t)lcore_index; - LIST_INSERT_HEAD(&list->cache[lcore_index].h, lentry, next); + LIST_INSERT_HEAD(&list->cache[lcore_index]->h, lentry, next); return lentry; } static void __list_cache_clean(struct mlx5_list *list, int lcore_index) { - struct mlx5_list_cache *c = &list->cache[lcore_index]; + struct mlx5_list_cache *c = list->cache[lcore_index]; struct mlx5_list_entry *entry = LIST_FIRST(&c->h); uint32_t inv_cnt = __atomic_exchange_n(&c->inv_cnt, 0, __ATOMIC_RELAXED); @@ -161,6 +168,17 @@ mlx5_list_register(struct mlx5_list *list, void *ctx) rte_errno = ENOTSUP; return NULL; } + if (unlikely(!list->cache[lcore_index])) { + list->cache[lcore_index] = mlx5_malloc(0, + sizeof(struct mlx5_list_cache), + RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (!list->cache[lcore_index]) { + rte_errno = ENOMEM; + return NULL; + } + list->cache[lcore_index]->inv_cnt = 0; + LIST_INIT(&list->cache[lcore_index]->h); + } /* 0. Free entries that was invalidated by other lcores. */ __list_cache_clean(list, lcore_index); /* 1. Lookup in local cache. */ @@ -186,7 +204,7 @@ mlx5_list_register(struct mlx5_list *list, void *ctx) entry->ref_cnt = 1u; if (!list->lcores_share) { entry->lcore_idx = (uint32_t)lcore_index; - LIST_INSERT_HEAD(&list->cache[lcore_index].h, entry, next); + LIST_INSERT_HEAD(&list->cache[lcore_index]->h, entry, next); __atomic_add_fetch(&list->count, 1, __ATOMIC_RELAXED); DRV_LOG(DEBUG, "MLX5 list %s c%d entry %p new: %u.", list->name, lcore_index, (void *)entry, entry->ref_cnt); @@ -217,10 +235,10 @@ mlx5_list_register(struct mlx5_list *list, void *ctx) } } /* 5. Update lists. */ - LIST_INSERT_HEAD(&list->cache[RTE_MAX_LCORE].h, entry, next); + LIST_INSERT_HEAD(&list->cache[RTE_MAX_LCORE]->h, entry, next); list->gen_cnt++; rte_rwlock_write_unlock(&list->lock); - LIST_INSERT_HEAD(&list->cache[lcore_index].h, local_entry, next); + LIST_INSERT_HEAD(&list->cache[lcore_index]->h, local_entry, next); __atomic_add_fetch(&list->count, 1, __ATOMIC_RELAXED); DRV_LOG(DEBUG, "mlx5 list %s entry %p new: %u.", list->name, (void *)entry, entry->ref_cnt); @@ -245,7 +263,7 @@ mlx5_list_unregister(struct mlx5_list *list, else list->cb_remove(list->ctx, entry); } else if (likely(lcore_idx != -1)) { - __atomic_add_fetch(&list->cache[entry->lcore_idx].inv_cnt, 1, + __atomic_add_fetch(&list->cache[entry->lcore_idx]->inv_cnt, 1, __ATOMIC_RELAXED); } else { return 0; @@ -280,8 +298,10 @@ mlx5_list_uninit(struct mlx5_list *list) MLX5_ASSERT(list); for (i = 0; i <= RTE_MAX_LCORE; i++) { - while (!LIST_EMPTY(&list->cache[i].h)) { - entry = LIST_FIRST(&list->cache[i].h); + if (!list->cache[i]) + continue; + while (!LIST_EMPTY(&list->cache[i]->h)) { + entry = LIST_FIRST(&list->cache[i]->h); LIST_REMOVE(entry, next); if (i == RTE_MAX_LCORE) { list->cb_remove(list->ctx, entry); @@ -292,6 +312,8 @@ mlx5_list_uninit(struct mlx5_list *list) list->cb_clone_free(list->ctx, entry); } } + if (i != RTE_MAX_LCORE) + mlx5_free(list->cache[i]); } } @@ -320,6 +342,7 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, mlx5_list_clone_free_cb cb_clone_free) { struct mlx5_hlist *h; + struct mlx5_list_cache *gc; uint32_t act_size; uint32_t alloc_size; uint32_t i; @@ -333,7 +356,9 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, act_size = size; } alloc_size = sizeof(struct mlx5_hlist) + - sizeof(struct mlx5_hlist_bucket) * act_size; + sizeof(struct mlx5_hlist_bucket) * act_size; + if (lcores_share) + alloc_size += sizeof(struct mlx5_list_cache) * act_size; /* Using zmalloc, then no need to initialize the heads. */ h = mlx5_malloc(MLX5_MEM_ZERO, alloc_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); @@ -345,8 +370,10 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, h->mask = act_size - 1; h->lcores_share = lcores_share; h->direct_key = direct_key; + gc = (struct mlx5_list_cache *)&h->buckets[act_size]; for (i = 0; i < act_size; i++) { if (mlx5_list_init(&h->buckets[i].l, name, ctx, lcores_share, + lcores_share ? &gc[i] : NULL, cb_create, cb_match, cb_remove, cb_clone, cb_clone_free) != 0) { mlx5_free(h); @@ -358,7 +385,6 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, return h; } - struct mlx5_list_entry * mlx5_hlist_lookup(struct mlx5_hlist *h, uint64_t key, void *ctx) { diff --git a/drivers/common/mlx5/mlx5_common_utils.h b/drivers/common/mlx5/mlx5_common_utils.h index 79d0bc1a50..42ca3cbc1e 100644 --- a/drivers/common/mlx5/mlx5_common_utils.h +++ b/drivers/common/mlx5/mlx5_common_utils.h @@ -104,7 +104,7 @@ struct mlx5_list { mlx5_list_remove_cb cb_remove; /**< entry remove callback. */ mlx5_list_clone_cb cb_clone; /**< entry clone callback. */ mlx5_list_clone_free_cb cb_clone_free; - struct mlx5_list_cache cache[RTE_MAX_LCORE + 1]; + struct mlx5_list_cache *cache[RTE_MAX_LCORE + 1]; /* Lcore cache, last index is the global cache. */ volatile uint32_t gen_cnt; /* List modification may update it. */ volatile uint32_t count; /* number of entries in list. */