From patchwork Tue Oct 10 14:38:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aaron Conole X-Patchwork-Id: 132463 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 842024234D; Tue, 10 Oct 2023 16:38:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2D8A7402C2; Tue, 10 Oct 2023 16:38:13 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 595C740297 for ; Tue, 10 Oct 2023 16:38:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696948691; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ishg4r6KT3dribqef+UH55fqDgvv9yOl9+P4nqFzBis=; b=GgKvpqDuE94Xf7VcWIIURSkH11fX35BYMnH2Lq19C4UgFpKXH9MqaWIep0G8fmDpRllYUh Y8aikeLdnbyObr86+3BzdBqTcUNhKSjwo1cOuzFCE8QsgqM/76LLUruyJ1fBtrMQf7iDJs XAx8t+8TyLClgREoU6Gm+NnJw6b1EDw= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-104-YDxMWaZCP9mGU6Ef6T7F6w-1; Tue, 10 Oct 2023 10:38:02 -0400 X-MC-Unique: YDxMWaZCP9mGU6Ef6T7F6w-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B060F293248F; Tue, 10 Oct 2023 14:38:01 +0000 (UTC) Received: from RHTPC1VM0NT.lan (unknown [10.22.34.140]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2B4989CC; Tue, 10 Oct 2023 14:38:01 +0000 (UTC) From: Aaron Conole To: dev@dpdk.org Cc: John Romein , Raslan Darawsheh , Elena Agostini , Dmitry Kozlyuk , Matan Azrad , Viacheslav Ovsiienko , Ori Kam , Suanming Mou Subject: [PATCH v2] common/mlx5: Optimize mlx5 mempool get extmem Date: Tue, 10 Oct 2023 10:38:00 -0400 Message-ID: <20231010143800.102459-1-aconole@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: John Romein This patch reduces the time to allocate and register tens of gigabytes of GPU memory from hours to seconds, by sorting the heap only once instead of for each object in the mempool. Fixes: 690b2a88c2f7 ("common/mlx5: add mempool registration facilities") Signed-off-by: John Romein --- drivers/common/mlx5/mlx5_common_mr.c | 69 ++++++++-------------------- 1 file changed, 20 insertions(+), 49 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c index 40ff9153bd..77b66e444b 100644 --- a/drivers/common/mlx5/mlx5_common_mr.c +++ b/drivers/common/mlx5/mlx5_common_mr.c @@ -1389,63 +1389,23 @@ mlx5_mempool_get_chunks(struct rte_mempool *mp, struct mlx5_range **out, return 0; } -struct mlx5_mempool_get_extmem_data { - struct mlx5_range *heap; - unsigned int heap_size; - int ret; -}; - static void mlx5_mempool_get_extmem_cb(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx) { - struct mlx5_mempool_get_extmem_data *data = opaque; + struct mlx5_range *heap = opaque; struct rte_mbuf *mbuf = obj; uintptr_t addr = (uintptr_t)mbuf->buf_addr; - struct mlx5_range *seg, *heap; struct rte_memseg_list *msl; size_t page_size; uintptr_t page_start; - unsigned int pos = 0, len = data->heap_size, delta; RTE_SET_USED(mp); - RTE_SET_USED(obj_idx); - if (data->ret < 0) - return; - /* Binary search for an already visited page. */ - while (len > 1) { - delta = len / 2; - if (addr < data->heap[pos + delta].start) { - len = delta; - } else { - pos += delta; - len -= delta; - } - } - if (data->heap != NULL) { - seg = &data->heap[pos]; - if (seg->start <= addr && addr < seg->end) - return; - } - /* Determine the page boundaries and remember them. */ - heap = realloc(data->heap, sizeof(heap[0]) * (data->heap_size + 1)); - if (heap == NULL) { - free(data->heap); - data->heap = NULL; - data->ret = -1; - return; - } - data->heap = heap; - data->heap_size++; - seg = &heap[data->heap_size - 1]; msl = rte_mem_virt2memseg_list((void *)addr); page_size = msl != NULL ? msl->page_sz : rte_mem_page_size(); page_start = RTE_PTR_ALIGN_FLOOR(addr, page_size); - seg->start = page_start; - seg->end = page_start + page_size; - /* Maintain the heap order. */ - qsort(data->heap, data->heap_size, sizeof(heap[0]), - mlx5_range_compare_start); + heap[obj_idx].start = page_start; + heap[obj_idx].end = page_start + page_size; } /** @@ -1457,15 +1417,26 @@ static int mlx5_mempool_get_extmem(struct rte_mempool *mp, struct mlx5_range **out, unsigned int *out_n) { - struct mlx5_mempool_get_extmem_data data; + unsigned int out_size = 1; + struct mlx5_range *heap; DRV_LOG(DEBUG, "Recovering external pinned pages of mempool %s", mp->name); - memset(&data, 0, sizeof(data)); - rte_mempool_obj_iter(mp, mlx5_mempool_get_extmem_cb, &data); - *out = data.heap; - *out_n = data.heap_size; - return data.ret; + heap = malloc(mp->size * sizeof(struct mlx5_range)); + if (heap == NULL) + return -1; + rte_mempool_obj_iter(mp, mlx5_mempool_get_extmem_cb, heap); + qsort(heap, mp->size, sizeof(heap[0]), mlx5_range_compare_start); + /* remove duplicates */ + for (unsigned int i = 1; i < mp->size; i++) + if (heap[out_size - 1].start != heap[i].start) + heap[out_size++] = heap[i]; + heap = realloc(heap, out_size * sizeof(struct mlx5_range)); + if (heap == NULL) + return -1; + *out = heap; + *out_n = out_size; + return 0; } /**