From patchwork Thu Feb 29 13:48:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gowrishankar Muthukrishnan X-Patchwork-Id: 137497 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6C94B43C3B; Thu, 29 Feb 2024 14:48:25 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 56A5142D2A; Thu, 29 Feb 2024 14:48:25 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9CDF44025C for ; Thu, 29 Feb 2024 14:48:23 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 41TBkMw1021316; Thu, 29 Feb 2024 05:48:23 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=YZQzgJBh+scUnA9oKcyEszUsnWBk5sDY4XsqzdFVZy0=; b=FzM B2Y2QVH3G7EU3yhKit2NT4khU/hk5AObCpmZWCD1gZnnEXamUK7Ihz+W3+NyKr8B KcdZKrtKsYLfTJo8rbOaSa6nTXPBiYmGYf6jckBHu1gzDoG5Z9qz3h+TPkVOvt5N Xl0C01ZoN2AURpMlq5qno8U7GWLcw4XQKwnqp8v6pjZ2uQzUVbWtAiY960zsmhlD KoLpkGS8WKVfn+wQtVTs0Lvf826wiDziNpJO6lSdavnNF1yii+O9hRohewW3GRqC pE64p3kyc36Tl8fHEFar5o+psznA0RiTniX9cqh7rkCgb85oOOkvqu/KqnRlTyVp ahPGNecpNmNqEumJhwg== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3wjfay2xd6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 29 Feb 2024 05:48:22 -0800 (PST) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12; Thu, 29 Feb 2024 05:48:22 -0800 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1258.12 via Frontend Transport; Thu, 29 Feb 2024 05:48:22 -0800 Received: from BG-LT91401.marvell.com (unknown [10.193.66.108]) by maili.marvell.com (Postfix) with ESMTP id 496393F724F; Thu, 29 Feb 2024 05:48:17 -0800 (PST) From: Gowrishankar Muthukrishnan To: CC: , Cheng Jiang , Kevin Laatz , Bruce Richardson , "Pavan Nikhilesh" , Amit Prakash Shukla , Chengwen Feng , Jerin , Gowrishankar Muthukrishnan Subject: [v11 1/4] app/dma-perf: add skip support Date: Thu, 29 Feb 2024 19:18:07 +0530 Message-ID: <3b9ac4436f577f5203931e9f3ab31e8548709ba2.1709210551.git.gmuthukrishn@marvell.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: CCOQzxq0JvxiAl8ZrEdOmUrPR_LwyM7- X-Proofpoint-GUID: CCOQzxq0JvxiAl8ZrEdOmUrPR_LwyM7- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-29_02,2024-02-29_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support to skip running a dma-perf test-case. Signed-off-by: Amit Prakash Shukla Acked-by: Anoob Joseph Acked-by: Chengwen Feng --- v11: - config file formatting app/test-dma-perf/config.ini | 2 ++ app/test-dma-perf/main.c | 48 ++++++++++++++++++++++-------------- app/test-dma-perf/main.h | 1 + 3 files changed, 32 insertions(+), 19 deletions(-) diff --git a/app/test-dma-perf/config.ini b/app/test-dma-perf/config.ini index b550f4b23f..bb0b1aa11a 100644 --- a/app/test-dma-perf/config.ini +++ b/app/test-dma-perf/config.ini @@ -30,6 +30,8 @@ ; If you have already set the "-l" and "-a" parameters using EAL, ; make sure that the value of "lcore" falls within their range of values. +; "skip" To skip a test-case set skip to 1. + ; To specify a configuration file, use the "--config" flag followed by the path to the file. ; To specify a result file, use the "--result" flag followed by the path to the file. diff --git a/app/test-dma-perf/main.c b/app/test-dma-perf/main.c index 544784df50..e9e40e72e7 100644 --- a/app/test-dma-perf/main.c +++ b/app/test-dma-perf/main.c @@ -86,6 +86,19 @@ output_header(uint32_t case_id, struct test_configure *case_cfg) output_csv(true); } +static int +open_output_csv(const char *rst_path_ptr) +{ + fd = fopen(rst_path_ptr, "a"); + if (!fd) { + printf("Open output CSV file error.\n"); + return 1; + } + output_csv(true); + fclose(fd); + return 0; +} + static void run_test_case(struct test_configure *case_cfg) { @@ -322,6 +335,7 @@ load_configs(const char *path) const char *case_type; const char *lcore_dma; const char *mem_size_str, *buf_size_str, *ring_size_str, *kick_batch_str; + const char *skip; int args_nr, nb_vp; bool is_dma; @@ -341,6 +355,13 @@ load_configs(const char *path) for (i = 0; i < nb_sections; i++) { snprintf(section_name, CFG_NAME_LEN, "case%d", i + 1); test_case = &test_cases[i]; + + skip = rte_cfgfile_get_entry(cfgfile, section_name, "skip"); + if (skip && (atoi(skip) == 1)) { + test_case->is_skip = true; + continue; + } + case_type = rte_cfgfile_get_entry(cfgfile, section_name, "type"); if (case_type == NULL) { printf("Error: No case type in case %d, the test will be finished here.\n", @@ -525,31 +546,20 @@ main(int argc, char *argv[]) printf("Running cases...\n"); for (i = 0; i < case_nb; i++) { - if (!test_cases[i].is_valid) { - printf("Invalid test case %d.\n\n", i + 1); - snprintf(output_str[0], MAX_OUTPUT_STR_LEN, "Invalid case %d\n", i + 1); - - fd = fopen(rst_path_ptr, "a"); - if (!fd) { - printf("Open output CSV file error.\n"); + if (test_cases[i].is_skip) { + printf("Test case %d configured to be skipped.\n\n", i + 1); + snprintf(output_str[0], MAX_OUTPUT_STR_LEN, "Skip the test-case %d\n", + i + 1); + if (open_output_csv(rst_path_ptr)) return 0; - } - output_csv(true); - fclose(fd); continue; } - if (test_cases[i].test_type == TEST_TYPE_NONE) { - printf("No valid test type in test case %d.\n\n", i + 1); + if (!test_cases[i].is_valid) { + printf("Invalid test case %d.\n\n", i + 1); snprintf(output_str[0], MAX_OUTPUT_STR_LEN, "Invalid case %d\n", i + 1); - - fd = fopen(rst_path_ptr, "a"); - if (!fd) { - printf("Open output CSV file error.\n"); + if (open_output_csv(rst_path_ptr)) return 0; - } - output_csv(true); - fclose(fd); continue; } diff --git a/app/test-dma-perf/main.h b/app/test-dma-perf/main.h index 62085e6e8f..32670151af 100644 --- a/app/test-dma-perf/main.h +++ b/app/test-dma-perf/main.h @@ -40,6 +40,7 @@ struct lcore_dma_map_t { struct test_configure { bool is_valid; + bool is_skip; uint8_t test_type; const char *test_type_str; uint16_t src_numa_node; From patchwork Thu Feb 29 13:48:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gowrishankar Muthukrishnan X-Patchwork-Id: 137498 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A766C43C3B; Thu, 29 Feb 2024 14:48:30 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 98D5642DC5; Thu, 29 Feb 2024 14:48:29 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id F387142DBE for ; Thu, 29 Feb 2024 14:48:27 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 41T7OAiJ029089; Thu, 29 Feb 2024 05:48:27 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=3a/cNvegTB5vgFGSstz4CVZxfHin/RcVXxAMhSNQRJI=; b=IsY yU+H+IYzhuXujmTg8JZXulG9M7HGYM/k5z3k6CUIZ4B/3jrLgp4WtKvujonw+9PP K+BWogNhETtc2ZJJ1yTlkSXDQqdggUC/0kmN7mGPgsEH7jz4lvGY13UJzhtz4YnQ 10l6cA0zZYVgRRrIey3/zi2BH67ZF7wf9NnuJP6JLo261Uazp7vs/gE4NBEJM/rf iE9VpqmN8UI8V3qWQhrHhjShEGD6CIzTPXMk7Dzuz2Rtcj4TK88PKF2tT4W0eoHo H6Xc1M+jnHNwzpojoVW8AsrG4dIpNbZeQaVD70tGDdxzavLyWiTcz2HogD/wqFIt kG0fEfeDS8ERvb/GUKw== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3wjfay2xdh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 29 Feb 2024 05:48:27 -0800 (PST) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12; Thu, 29 Feb 2024 05:48:26 -0800 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1258.12 via Frontend Transport; Thu, 29 Feb 2024 05:48:26 -0800 Received: from BG-LT91401.marvell.com (unknown [10.193.66.108]) by maili.marvell.com (Postfix) with ESMTP id 92D123F724F; Thu, 29 Feb 2024 05:48:22 -0800 (PST) From: Gowrishankar Muthukrishnan To: CC: , Cheng Jiang , Kevin Laatz , Bruce Richardson , "Pavan Nikhilesh" , Amit Prakash Shukla , Chengwen Feng , Jerin , Gowrishankar Muthukrishnan Subject: [v11 2/4] app/dma-perf: add PCI device support Date: Thu, 29 Feb 2024 19:18:08 +0530 Message-ID: <6a36448d09a70c2041c64f7e850a6fb0a9c0945e.1709210551.git.gmuthukrishn@marvell.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: PfsmSf-bjAs2rt0hqelqRPRHVfv80sau X-Proofpoint-GUID: PfsmSf-bjAs2rt0hqelqRPRHVfv80sau X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-29_02,2024-02-29_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support to test performance for "device to memory" and "memory to device" data transfer. Signed-off-by: Amit Prakash Shukla Acked-by: Anoob Joseph Acked-by: Chengwen Feng --- v11: - config file formatting. app/test-dma-perf/benchmark.c | 119 ++++++++++++++++++++++++++++++---- app/test-dma-perf/config.ini | 31 +++++++++ app/test-dma-perf/main.c | 77 ++++++++++++++++++++++ app/test-dma-perf/main.h | 7 ++ 4 files changed, 222 insertions(+), 12 deletions(-) diff --git a/app/test-dma-perf/benchmark.c b/app/test-dma-perf/benchmark.c index 9b1f58c78c..3c4fddb138 100644 --- a/app/test-dma-perf/benchmark.c +++ b/app/test-dma-perf/benchmark.c @@ -127,17 +127,54 @@ cache_flush_buf(__rte_unused struct rte_mbuf **array, #endif } +static int +vchan_data_populate(uint32_t dev_id, struct rte_dma_vchan_conf *qconf, + struct test_configure *cfg) +{ + struct rte_dma_info info; + + qconf->direction = cfg->transfer_dir; + + rte_dma_info_get(dev_id, &info); + if (!(RTE_BIT64(qconf->direction) & info.dev_capa)) + return -1; + + qconf->nb_desc = cfg->ring_size.cur; + + switch (qconf->direction) { + case RTE_DMA_DIR_MEM_TO_DEV: + qconf->dst_port.pcie.vfen = 1; + qconf->dst_port.port_type = RTE_DMA_PORT_PCIE; + qconf->dst_port.pcie.coreid = cfg->vchan_dev.port.pcie.coreid; + qconf->dst_port.pcie.vfid = cfg->vchan_dev.port.pcie.vfid; + qconf->dst_port.pcie.pfid = cfg->vchan_dev.port.pcie.pfid; + break; + case RTE_DMA_DIR_DEV_TO_MEM: + qconf->src_port.pcie.vfen = 1; + qconf->src_port.port_type = RTE_DMA_PORT_PCIE; + qconf->src_port.pcie.coreid = cfg->vchan_dev.port.pcie.coreid; + qconf->src_port.pcie.vfid = cfg->vchan_dev.port.pcie.vfid; + qconf->src_port.pcie.pfid = cfg->vchan_dev.port.pcie.pfid; + break; + case RTE_DMA_DIR_MEM_TO_MEM: + case RTE_DMA_DIR_DEV_TO_DEV: + break; + } + + return 0; +} + /* Configuration of device. */ static void -configure_dmadev_queue(uint32_t dev_id, uint32_t ring_size) +configure_dmadev_queue(uint32_t dev_id, struct test_configure *cfg) { uint16_t vchan = 0; struct rte_dma_info info; struct rte_dma_conf dev_config = { .nb_vchans = 1 }; - struct rte_dma_vchan_conf qconf = { - .direction = RTE_DMA_DIR_MEM_TO_MEM, - .nb_desc = ring_size - }; + struct rte_dma_vchan_conf qconf = { 0 }; + + if (vchan_data_populate(dev_id, &qconf, cfg) != 0) + rte_exit(EXIT_FAILURE, "Error with vchan data populate.\n"); if (rte_dma_configure(dev_id, &dev_config) != 0) rte_exit(EXIT_FAILURE, "Error with dma configure.\n"); @@ -159,7 +196,6 @@ configure_dmadev_queue(uint32_t dev_id, uint32_t ring_size) static int config_dmadevs(struct test_configure *cfg) { - uint32_t ring_size = cfg->ring_size.cur; struct lcore_dma_map_t *ldm = &cfg->lcore_dma_map; uint32_t nb_workers = ldm->cnt; uint32_t i; @@ -176,7 +212,7 @@ config_dmadevs(struct test_configure *cfg) } ldm->dma_ids[i] = dev_id; - configure_dmadev_queue(dev_id, ring_size); + configure_dmadev_queue(dev_id, cfg); ++nb_dmadevs; } @@ -302,13 +338,23 @@ do_cpu_mem_copy(void *p) return 0; } +static void +dummy_free_ext_buf(void *addr, void *opaque) +{ + RTE_SET_USED(addr); + RTE_SET_USED(opaque); +} + static int setup_memory_env(struct test_configure *cfg, struct rte_mbuf ***srcs, struct rte_mbuf ***dsts) { - unsigned int buf_size = cfg->buf_size.cur; + static struct rte_mbuf_ext_shared_info *ext_buf_info; + unsigned int cur_buf_size = cfg->buf_size.cur; + unsigned int buf_size = cur_buf_size + RTE_PKTMBUF_HEADROOM; unsigned int nr_sockets; uint32_t nr_buf = cfg->nr_buf; + uint32_t i; nr_sockets = rte_socket_count(); if (cfg->src_numa_node >= nr_sockets || @@ -321,7 +367,7 @@ setup_memory_env(struct test_configure *cfg, struct rte_mbuf ***srcs, nr_buf, 0, 0, - buf_size + RTE_PKTMBUF_HEADROOM, + buf_size, cfg->src_numa_node); if (src_pool == NULL) { PRINT_ERR("Error with source mempool creation.\n"); @@ -332,7 +378,7 @@ setup_memory_env(struct test_configure *cfg, struct rte_mbuf ***srcs, nr_buf, 0, 0, - buf_size + RTE_PKTMBUF_HEADROOM, + buf_size, cfg->dst_numa_node); if (dst_pool == NULL) { PRINT_ERR("Error with destination mempool creation.\n"); @@ -361,16 +407,51 @@ setup_memory_env(struct test_configure *cfg, struct rte_mbuf ***srcs, return -1; } + if (cfg->transfer_dir == RTE_DMA_DIR_DEV_TO_MEM || + cfg->transfer_dir == RTE_DMA_DIR_MEM_TO_DEV) { + ext_buf_info = rte_malloc(NULL, sizeof(struct rte_mbuf_ext_shared_info), 0); + if (ext_buf_info == NULL) { + printf("Error: ext_buf_info malloc failed.\n"); + return -1; + } + } + + if (cfg->transfer_dir == RTE_DMA_DIR_DEV_TO_MEM) { + ext_buf_info->free_cb = dummy_free_ext_buf; + ext_buf_info->fcb_opaque = NULL; + for (i = 0; i < nr_buf; i++) { + /* Using mbuf structure to hold remote iova address. */ + rte_pktmbuf_attach_extbuf((*srcs)[i], + (void *)(cfg->vchan_dev.raddr + (i * buf_size)), + (rte_iova_t)(cfg->vchan_dev.raddr + (i * buf_size)), + 0, ext_buf_info); + rte_mbuf_ext_refcnt_update(ext_buf_info, 1); + } + } + + if (cfg->transfer_dir == RTE_DMA_DIR_MEM_TO_DEV) { + ext_buf_info->free_cb = dummy_free_ext_buf; + ext_buf_info->fcb_opaque = NULL; + for (i = 0; i < nr_buf; i++) { + /* Using mbuf structure to hold remote iova address. */ + rte_pktmbuf_attach_extbuf((*dsts)[i], + (void *)(cfg->vchan_dev.raddr + (i * buf_size)), + (rte_iova_t)(cfg->vchan_dev.raddr + (i * buf_size)), + 0, ext_buf_info); + rte_mbuf_ext_refcnt_update(ext_buf_info, 1); + } + } + return 0; } void mem_copy_benchmark(struct test_configure *cfg, bool is_dma) { - uint16_t i; + uint32_t i; uint32_t offset; unsigned int lcore_id = 0; - struct rte_mbuf **srcs = NULL, **dsts = NULL; + struct rte_mbuf **srcs = NULL, **dsts = NULL, **m = NULL; struct lcore_dma_map_t *ldm = &cfg->lcore_dma_map; unsigned int buf_size = cfg->buf_size.cur; uint16_t kick_batch = cfg->kick_batch.cur; @@ -476,6 +557,20 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) avg_cycles_total / nb_workers, bandwidth_total, mops_total); out: + + if (cfg->transfer_dir == RTE_DMA_DIR_DEV_TO_MEM) + m = srcs; + else if (cfg->transfer_dir == RTE_DMA_DIR_MEM_TO_DEV) + m = dsts; + + if (m) { + for (i = 0; i < nr_buf; i++) + rte_pktmbuf_detach_extbuf(m[i]); + + if (m[0]->shinfo && rte_mbuf_ext_refcnt_read(m[0]->shinfo) == 0) + rte_free(m[0]->shinfo); + } + /* free mbufs used in the test */ if (srcs != NULL) rte_pktmbuf_free_bulk(srcs, nr_buf); diff --git a/app/test-dma-perf/config.ini b/app/test-dma-perf/config.ini index bb0b1aa11a..ada0146b92 100644 --- a/app/test-dma-perf/config.ini +++ b/app/test-dma-perf/config.ini @@ -32,6 +32,21 @@ ; "skip" To skip a test-case set skip to 1. +; Parameters for data transfers from "mem to dev" and "dev to mem": +; +; "direction" denotes the direction of data transfer. It can take 3 values: +; mem2mem - mem to mem transfer +; mem2dev - mem to dev transfer +; dev2mem - dev to mem transfer +; If not specified the default value is mem2mem transfer. + +; "vchan_dev" denotes below comma separated bus related parameters for mem2dev and dev2mem dma transfer. +; "raddr" remote iova address for mem2dev and dev2mem transfer. +; "coreid" denotes PCIe core index. +; "pfid" denotes PF-id to be used for data transfer +; "vfid" denotes VF-id of PF-id to be used for data transfer. +; Example: vchan_dev=raddr=0x400000,coreid=1,pfid=2,vfid=3 + ; To specify a configuration file, use the "--config" flag followed by the path to the file. ; To specify a result file, use the "--result" flag followed by the path to the file. @@ -52,6 +67,22 @@ lcore_dma=lcore10@0000:00:04.2, lcore11@0000:00:04.3 eal_args=--in-memory --file-prefix=test [case2] +skip=1 +type=DMA_MEM_COPY +direction=dev2mem +vchan_dev=raddr=0x200000000,coreid=1,pfid=2,vfid=3 +mem_size=10 +buf_size=64,4096,2,MUL +dma_ring_size=1024 +kick_batch=32 +src_numa_node=0 +dst_numa_node=0 +cache_flush=0 +test_seconds=2 +lcore_dma=lcore10@0000:00:04.2, lcore11@0000:00:04.3 +eal_args=--in-memory --file-prefix=test + +[case3] type=CPU_MEM_COPY mem_size=10 buf_size=64,8192,2,MUL diff --git a/app/test-dma-perf/main.c b/app/test-dma-perf/main.c index e9e40e72e7..051f76a6f9 100644 --- a/app/test-dma-perf/main.c +++ b/app/test-dma-perf/main.c @@ -16,6 +16,8 @@ #include #include #include +#include +#include #include "main.h" @@ -325,6 +327,28 @@ parse_entry(const char *value, struct test_configure_entry *entry) return args_nr; } +static int populate_pcie_config(const char *key, const char *value, void *test) +{ + struct test_configure *test_case = (struct test_configure *)test; + char *endptr; + int ret = 0; + + if (strcmp(key, "raddr") == 0) + test_case->vchan_dev.raddr = strtoull(value, &endptr, 16); + else if (strcmp(key, "coreid") == 0) + test_case->vchan_dev.port.pcie.coreid = (uint8_t)atoi(value); + else if (strcmp(key, "vfid") == 0) + test_case->vchan_dev.port.pcie.vfid = (uint16_t)atoi(value); + else if (strcmp(key, "pfid") == 0) + test_case->vchan_dev.port.pcie.pfid = (uint16_t)atoi(value); + else { + printf("Invalid config param: %s\n", key); + ret = -1; + } + + return ret; +} + static uint16_t load_configs(const char *path) { @@ -333,9 +357,12 @@ load_configs(const char *path) struct test_configure *test_case; char section_name[CFG_NAME_LEN]; const char *case_type; + const char *transfer_dir; const char *lcore_dma; const char *mem_size_str, *buf_size_str, *ring_size_str, *kick_batch_str; const char *skip; + struct rte_kvargs *kvlist; + const char *vchan_dev; int args_nr, nb_vp; bool is_dma; @@ -373,6 +400,22 @@ load_configs(const char *path) if (strcmp(case_type, DMA_MEM_COPY) == 0) { test_case->test_type = TEST_TYPE_DMA_MEM_COPY; test_case->test_type_str = DMA_MEM_COPY; + + transfer_dir = rte_cfgfile_get_entry(cfgfile, section_name, "direction"); + if (transfer_dir == NULL) { + printf("Transfer direction not configured." + " Defaulting it to MEM to MEM transfer.\n"); + test_case->transfer_dir = RTE_DMA_DIR_MEM_TO_MEM; + } else { + if (strcmp(transfer_dir, "mem2dev") == 0) + test_case->transfer_dir = RTE_DMA_DIR_MEM_TO_DEV; + else if (strcmp(transfer_dir, "dev2mem") == 0) + test_case->transfer_dir = RTE_DMA_DIR_DEV_TO_MEM; + else { + printf("Defaulting the test to MEM to MEM transfer\n"); + test_case->transfer_dir = RTE_DMA_DIR_MEM_TO_MEM; + } + } is_dma = true; } else if (strcmp(case_type, CPU_MEM_COPY) == 0) { test_case->test_type = TEST_TYPE_CPU_MEM_COPY; @@ -384,6 +427,40 @@ load_configs(const char *path) continue; } + if (test_case->transfer_dir == RTE_DMA_DIR_MEM_TO_DEV || + test_case->transfer_dir == RTE_DMA_DIR_DEV_TO_MEM) { + vchan_dev = rte_cfgfile_get_entry(cfgfile, section_name, "vchan_dev"); + if (vchan_dev == NULL) { + printf("Transfer direction mem2dev and dev2mem" + " vhcan_dev shall be configured.\n"); + test_case->is_valid = false; + continue; + } + + kvlist = rte_kvargs_parse(vchan_dev, NULL); + if (kvlist == NULL) { + printf("rte_kvargs_parse() error"); + test_case->is_valid = false; + continue; + } + + if (rte_kvargs_process(kvlist, NULL, populate_pcie_config, + (void *)test_case) < 0) { + printf("rte_kvargs_process() error\n"); + rte_kvargs_free(kvlist); + test_case->is_valid = false; + continue; + } + + if (!test_case->vchan_dev.raddr) { + printf("For mem2dev and dev2mem configure raddr\n"); + rte_kvargs_free(kvlist); + test_case->is_valid = false; + continue; + } + rte_kvargs_free(kvlist); + } + test_case->src_numa_node = (int)atoi(rte_cfgfile_get_entry(cfgfile, section_name, "src_numa_node")); test_case->dst_numa_node = (int)atoi(rte_cfgfile_get_entry(cfgfile, diff --git a/app/test-dma-perf/main.h b/app/test-dma-perf/main.h index 32670151af..745c24b7fe 100644 --- a/app/test-dma-perf/main.h +++ b/app/test-dma-perf/main.h @@ -38,10 +38,16 @@ struct lcore_dma_map_t { uint16_t cnt; }; +struct test_vchan_dev_config { + struct rte_dma_port_param port; + uintptr_t raddr; +}; + struct test_configure { bool is_valid; bool is_skip; uint8_t test_type; + uint8_t transfer_dir; const char *test_type_str; uint16_t src_numa_node; uint16_t dst_numa_node; @@ -57,6 +63,7 @@ struct test_configure { uint16_t test_secs; const char *eal_args; uint8_t scenario_id; + struct test_vchan_dev_config vchan_dev; }; void mem_copy_benchmark(struct test_configure *cfg, bool is_dma); From patchwork Thu Feb 29 13:48:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gowrishankar Muthukrishnan X-Patchwork-Id: 137499 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D8E6043C3B; Thu, 29 Feb 2024 14:48:36 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E4FBC42E3B; Thu, 29 Feb 2024 14:48:34 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id A8CA742DEF for ; Thu, 29 Feb 2024 14:48:32 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 41TBjJ5M019949; Thu, 29 Feb 2024 05:48:32 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=MTykTjGPGS8e8YMDqw7LTyqo5meRGbxb6U/ZB7Rf9CI=; b=iaC qqo7y6o7pILmjvrCPUrr0gUdNWue8c7TVcGmaOTii2uNQK2pXq5VoR81ZT/y8OE+ 6XkLzbv1NIfH7St64KbJFIbvvZSvsovHwJnsk/KpHDlGY/5wPfBp3gFylyGIoKrm 0jwICxZ9zVbHYiQ2alMV63Aeu+QNJdjXnJ8MPqfHVQ0/fFvekCUXBAR/2aoizEMp VBdmaimwmoSAJYwCuo7AARfoKJx1fgohWREDNRfT/HRqCMyR3gqjetQpqUauh2gC ymcEMF3GQGqVjawCJq3c+2U43InmreAkVRTETtGSDC1/QI+dmw3M/NfO0E5TYTiM XH++uPmC5P5fedfgZew== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3wjfay2xdr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 29 Feb 2024 05:48:31 -0800 (PST) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12; Thu, 29 Feb 2024 05:48:30 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1258.12 via Frontend Transport; Thu, 29 Feb 2024 05:48:30 -0800 Received: from BG-LT91401.marvell.com (unknown [10.193.66.108]) by maili.marvell.com (Postfix) with ESMTP id DB6403F724F; Thu, 29 Feb 2024 05:48:26 -0800 (PST) From: Gowrishankar Muthukrishnan To: CC: , Cheng Jiang , Kevin Laatz , Bruce Richardson , "Pavan Nikhilesh" , Amit Prakash Shukla , Chengwen Feng , Jerin , Gowrishankar Muthukrishnan Subject: [v11 3/4] app/dma-perf: validate copied memory Date: Thu, 29 Feb 2024 19:18:09 +0530 Message-ID: <1ecdf211518dd1da9bcc771d64e82e259cc0681c.1709210551.git.gmuthukrishn@marvell.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: yokFP4L5jhz6ai5NLtnqWnXngXRtqnoI X-Proofpoint-GUID: yokFP4L5jhz6ai5NLtnqWnXngXRtqnoI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-29_02,2024-02-29_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Validate copied memory to ensure DMA copy did not fail. Signed-off-by: Gowrishankar Muthukrishnan Acked-by: Anoob Joseph Acked-by: Chengwen Feng --- app/test-dma-perf/benchmark.c | 23 ++++++++++++++++++++++- app/test-dma-perf/main.c | 16 +++++++++++----- app/test-dma-perf/main.h | 2 +- 3 files changed, 34 insertions(+), 7 deletions(-) diff --git a/app/test-dma-perf/benchmark.c b/app/test-dma-perf/benchmark.c index 3c4fddb138..9c155a58cc 100644 --- a/app/test-dma-perf/benchmark.c +++ b/app/test-dma-perf/benchmark.c @@ -12,6 +12,7 @@ #include #include #include +#include #include "main.h" @@ -407,6 +408,11 @@ setup_memory_env(struct test_configure *cfg, struct rte_mbuf ***srcs, return -1; } + for (i = 0; i < nr_buf; i++) { + memset(rte_pktmbuf_mtod((*srcs)[i], void *), rte_rand(), buf_size); + memset(rte_pktmbuf_mtod((*dsts)[i], void *), 0, buf_size); + } + if (cfg->transfer_dir == RTE_DMA_DIR_DEV_TO_MEM || cfg->transfer_dir == RTE_DMA_DIR_MEM_TO_DEV) { ext_buf_info = rte_malloc(NULL, sizeof(struct rte_mbuf_ext_shared_info), 0); @@ -445,7 +451,7 @@ setup_memory_env(struct test_configure *cfg, struct rte_mbuf ***srcs, return 0; } -void +int mem_copy_benchmark(struct test_configure *cfg, bool is_dma) { uint32_t i; @@ -463,6 +469,7 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) uint32_t avg_cycles_total; float mops, mops_total; float bandwidth, bandwidth_total; + int ret = 0; if (setup_memory_env(cfg, &srcs, &dsts) < 0) goto out; @@ -536,6 +543,18 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) rte_eal_mp_wait_lcore(); + if (cfg->transfer_dir == RTE_DMA_DIR_MEM_TO_MEM) { + for (i = 0; i < (nr_buf / nb_workers) * nb_workers; i++) { + if (memcmp(rte_pktmbuf_mtod(srcs[i], void *), + rte_pktmbuf_mtod(dsts[i], void *), + cfg->buf_size.cur) != 0) { + printf("Copy validation fails for buffer number %d\n", i); + ret = -1; + goto out; + } + } + } + mops_total = 0; bandwidth_total = 0; avg_cycles_total = 0; @@ -601,4 +620,6 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) rte_dma_stop(ldm->dma_ids[i]); } } + + return ret; } diff --git a/app/test-dma-perf/main.c b/app/test-dma-perf/main.c index 051f76a6f9..df05bcd7df 100644 --- a/app/test-dma-perf/main.c +++ b/app/test-dma-perf/main.c @@ -101,20 +101,24 @@ open_output_csv(const char *rst_path_ptr) return 0; } -static void +static int run_test_case(struct test_configure *case_cfg) { + int ret = 0; + switch (case_cfg->test_type) { case TEST_TYPE_DMA_MEM_COPY: - mem_copy_benchmark(case_cfg, true); + ret = mem_copy_benchmark(case_cfg, true); break; case TEST_TYPE_CPU_MEM_COPY: - mem_copy_benchmark(case_cfg, false); + ret = mem_copy_benchmark(case_cfg, false); break; default: printf("Unknown test type. %s\n", case_cfg->test_type_str); break; } + + return ret; } static void @@ -159,8 +163,10 @@ run_test(uint32_t case_id, struct test_configure *case_cfg) case_cfg->scenario_id++; printf("\nRunning scenario %d\n", case_cfg->scenario_id); - run_test_case(case_cfg); - output_csv(false); + if (run_test_case(case_cfg) < 0) + printf("\nTest fails! skipping this scenario.\n"); + else + output_csv(false); if (var_entry->op == OP_ADD) var_entry->cur += var_entry->incr; diff --git a/app/test-dma-perf/main.h b/app/test-dma-perf/main.h index 745c24b7fe..1123e7524a 100644 --- a/app/test-dma-perf/main.h +++ b/app/test-dma-perf/main.h @@ -66,6 +66,6 @@ struct test_configure { struct test_vchan_dev_config vchan_dev; }; -void mem_copy_benchmark(struct test_configure *cfg, bool is_dma); +int mem_copy_benchmark(struct test_configure *cfg, bool is_dma); #endif /* MAIN_H */ From patchwork Thu Feb 29 13:48:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gowrishankar Muthukrishnan X-Patchwork-Id: 137500 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D80E043C3B; Thu, 29 Feb 2024 14:48:45 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9D9E142E44; Thu, 29 Feb 2024 14:48:38 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id D0A9F42EF4 for ; Thu, 29 Feb 2024 14:48:36 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 41T83AVD022285; Thu, 29 Feb 2024 05:48:36 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=rWhtH2/BDVbaIngEoJ7o7jU0jiFyG7XGN1EnmVugj2Y=; b=juG PN/ADeDhUyfw6LGHw0BSFcX57xdbAJDV903jqXwugx2fpHVM73iL378PuQ9zlfKv JdWPGni40T9e1DbCdqF3z07JQpv/5dZOXv/AnWmA31AtPRNGosIUGOtTQLO/+/m/ RPXH21LQjukFB16G7zimB+KounXPjq7XDLS6mvx6i2gERtuBdl8lO+TRAgod6Q6v AbJz1faI/8Xs/v0EdzrPPHnFlKNNMUFmSTqyMFMb20128k8DS7Oti59EPIIKkucf Z6zqExOlWNjQJCaOd041ufV5EinEcQetQL2QzghzASnT0TyttVYarUmxKVCSD+Kw FC4dwIn1fr0/EFESSZw== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3wjp2y93cu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 29 Feb 2024 05:48:35 -0800 (PST) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12; Thu, 29 Feb 2024 05:48:34 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1258.12 via Frontend Transport; Thu, 29 Feb 2024 05:48:34 -0800 Received: from BG-LT91401.marvell.com (unknown [10.193.66.108]) by maili.marvell.com (Postfix) with ESMTP id 372BB3F7253; Thu, 29 Feb 2024 05:48:30 -0800 (PST) From: Gowrishankar Muthukrishnan To: CC: , Cheng Jiang , Kevin Laatz , Bruce Richardson , "Pavan Nikhilesh" , Amit Prakash Shukla , Chengwen Feng , Jerin , Gowrishankar Muthukrishnan Subject: [v11 4/4] app/dma-perf: add SG copy support Date: Thu, 29 Feb 2024 19:18:10 +0530 Message-ID: X-Mailer: git-send-email 2.21.0 In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-GUID: H1UelaUjePI9r9mSk9qslaOC93F9Z9au X-Proofpoint-ORIG-GUID: H1UelaUjePI9r9mSk9qslaOC93F9Z9au X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-29_02,2024-02-29_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add SG copy support. Signed-off-by: Gowrishankar Muthukrishnan Acked-by: Anoob Joseph Acked-by: Chengwen Feng --- v11: - using struct for SGE config. app/test-dma-perf/benchmark.c | 283 ++++++++++++++++++++++++++++++---- app/test-dma-perf/config.ini | 25 ++- app/test-dma-perf/main.c | 41 ++++- app/test-dma-perf/main.h | 5 +- 4 files changed, 317 insertions(+), 37 deletions(-) diff --git a/app/test-dma-perf/benchmark.c b/app/test-dma-perf/benchmark.c index 9c155a58cc..d821af8532 100644 --- a/app/test-dma-perf/benchmark.c +++ b/app/test-dma-perf/benchmark.c @@ -34,6 +34,13 @@ struct worker_info { uint32_t test_cpl; }; +struct sge_info { + struct rte_dma_sge *srcs; + struct rte_dma_sge *dsts; + uint8_t nb_srcs; + uint8_t nb_dsts; +}; + struct lcore_params { uint8_t scenario_id; unsigned int lcore_id; @@ -46,6 +53,7 @@ struct lcore_params { uint16_t test_secs; struct rte_mbuf **srcs; struct rte_mbuf **dsts; + struct sge_info sge; volatile struct worker_info worker_info; }; @@ -86,21 +94,31 @@ calc_result(uint32_t buf_size, uint32_t nr_buf, uint16_t nb_workers, uint16_t te } static void -output_result(uint8_t scenario_id, uint32_t lcore_id, char *dma_name, uint16_t ring_size, - uint16_t kick_batch, uint64_t ave_cycle, uint32_t buf_size, uint32_t nr_buf, - float memory, float bandwidth, float mops, bool is_dma) +output_result(struct test_configure *cfg, struct lcore_params *para, + uint16_t kick_batch, uint64_t ave_cycle, uint32_t buf_size, + uint32_t nr_buf, float memory, float bandwidth, float mops) { - if (is_dma) - printf("lcore %u, DMA %s, DMA Ring Size: %u, Kick Batch Size: %u.\n", - lcore_id, dma_name, ring_size, kick_batch); - else + uint16_t ring_size = cfg->ring_size.cur; + uint8_t scenario_id = cfg->scenario_id; + uint32_t lcore_id = para->lcore_id; + char *dma_name = para->dma_name; + + if (cfg->is_dma) { + printf("lcore %u, DMA %s, DMA Ring Size: %u, Kick Batch Size: %u", lcore_id, + dma_name, ring_size, kick_batch); + if (cfg->is_sg) + printf(" DMA src sges: %u, dst sges: %u", + para->sge.nb_srcs, para->sge.nb_dsts); + printf(".\n"); + } else { printf("lcore %u\n", lcore_id); + } printf("Average Cycles/op: %" PRIu64 ", Buffer Size: %u B, Buffer Number: %u, Memory: %.2lf MB, Frequency: %.3lf Ghz.\n", ave_cycle, buf_size, nr_buf, memory, rte_get_timer_hz()/1000000000.0); printf("Average Bandwidth: %.3lf Gbps, MOps: %.3lf\n", bandwidth, mops); - if (is_dma) + if (cfg->is_dma) snprintf(output_str[lcore_id], MAX_OUTPUT_STR_LEN, CSV_LINE_DMA_FMT, scenario_id, lcore_id, dma_name, ring_size, kick_batch, buf_size, nr_buf, memory, ave_cycle, bandwidth, mops); @@ -167,7 +185,7 @@ vchan_data_populate(uint32_t dev_id, struct rte_dma_vchan_conf *qconf, /* Configuration of device. */ static void -configure_dmadev_queue(uint32_t dev_id, struct test_configure *cfg) +configure_dmadev_queue(uint32_t dev_id, struct test_configure *cfg, uint8_t sges_max) { uint16_t vchan = 0; struct rte_dma_info info; @@ -190,6 +208,10 @@ configure_dmadev_queue(uint32_t dev_id, struct test_configure *cfg) rte_exit(EXIT_FAILURE, "Error, no configured queues reported on device id. %u\n", dev_id); + if (info.max_sges < sges_max) + rte_exit(EXIT_FAILURE, "Error with unsupported max_sges on device id %u.\n", + dev_id); + if (rte_dma_start(dev_id) != 0) rte_exit(EXIT_FAILURE, "Error with dma start.\n"); } @@ -202,8 +224,12 @@ config_dmadevs(struct test_configure *cfg) uint32_t i; int dev_id; uint16_t nb_dmadevs = 0; + uint8_t nb_sges = 0; char *dma_name; + if (cfg->is_sg) + nb_sges = RTE_MAX(cfg->nb_src_sges, cfg->nb_dst_sges); + for (i = 0; i < ldm->cnt; i++) { dma_name = ldm->dma_names[i]; dev_id = rte_dma_get_dev_id_by_name(dma_name); @@ -213,7 +239,7 @@ config_dmadevs(struct test_configure *cfg) } ldm->dma_ids[i] = dev_id; - configure_dmadev_queue(dev_id, cfg); + configure_dmadev_queue(dev_id, cfg, nb_sges); ++nb_dmadevs; } @@ -253,7 +279,7 @@ do_dma_submit_and_poll(uint16_t dev_id, uint64_t *async_cnt, } static inline int -do_dma_mem_copy(void *p) +do_dma_plain_mem_copy(void *p) { struct lcore_params *para = (struct lcore_params *)p; volatile struct worker_info *worker_info = &(para->worker_info); @@ -306,6 +332,65 @@ do_dma_mem_copy(void *p) return 0; } +static inline int +do_dma_sg_mem_copy(void *p) +{ + struct lcore_params *para = (struct lcore_params *)p; + volatile struct worker_info *worker_info = &(para->worker_info); + struct rte_dma_sge *src_sges = para->sge.srcs; + struct rte_dma_sge *dst_sges = para->sge.dsts; + const uint8_t nb_src_sges = para->sge.nb_srcs; + const uint8_t nb_dst_sges = para->sge.nb_dsts; + const uint16_t kick_batch = para->kick_batch; + const uint16_t dev_id = para->dev_id; + uint32_t nr_buf = para->nr_buf; + uint64_t async_cnt = 0; + uint32_t poll_cnt = 0; + uint16_t nr_cpl; + uint32_t i, j; + int ret; + + nr_buf /= RTE_MAX(nb_src_sges, nb_dst_sges); + worker_info->stop_flag = false; + worker_info->ready_flag = true; + + while (!worker_info->start_flag) + ; + + while (1) { + j = 0; + for (i = 0; i < nr_buf; i++) { +dma_copy: + ret = rte_dma_copy_sg(dev_id, 0, + &src_sges[i * nb_src_sges], &dst_sges[j * nb_dst_sges], + nb_src_sges, nb_dst_sges, 0); + if (unlikely(ret < 0)) { + if (ret == -ENOSPC) { + do_dma_submit_and_poll(dev_id, &async_cnt, worker_info); + goto dma_copy; + } else + error_exit(dev_id); + } + async_cnt++; + j++; + + if ((async_cnt % kick_batch) == 0) + do_dma_submit_and_poll(dev_id, &async_cnt, worker_info); + } + + if (worker_info->stop_flag) + break; + } + + rte_dma_submit(dev_id, 0); + while ((async_cnt > 0) && (poll_cnt++ < POLL_MAX)) { + nr_cpl = rte_dma_completed(dev_id, 0, MAX_DMA_CPL_NB, NULL, NULL); + async_cnt -= nr_cpl; + } + + return 0; +} + static inline int do_cpu_mem_copy(void *p) { @@ -347,8 +432,9 @@ dummy_free_ext_buf(void *addr, void *opaque) } static int -setup_memory_env(struct test_configure *cfg, struct rte_mbuf ***srcs, - struct rte_mbuf ***dsts) +setup_memory_env(struct test_configure *cfg, + struct rte_mbuf ***srcs, struct rte_mbuf ***dsts, + struct rte_dma_sge **src_sges, struct rte_dma_sge **dst_sges) { static struct rte_mbuf_ext_shared_info *ext_buf_info; unsigned int cur_buf_size = cfg->buf_size.cur; @@ -409,8 +495,8 @@ setup_memory_env(struct test_configure *cfg, struct rte_mbuf ***srcs, } for (i = 0; i < nr_buf; i++) { - memset(rte_pktmbuf_mtod((*srcs)[i], void *), rte_rand(), buf_size); - memset(rte_pktmbuf_mtod((*dsts)[i], void *), 0, buf_size); + memset(rte_pktmbuf_mtod((*srcs)[i], void *), rte_rand(), cur_buf_size); + memset(rte_pktmbuf_mtod((*dsts)[i], void *), 0, cur_buf_size); } if (cfg->transfer_dir == RTE_DMA_DIR_DEV_TO_MEM || @@ -448,20 +534,102 @@ setup_memory_env(struct test_configure *cfg, struct rte_mbuf ***srcs, } } + if (cfg->is_sg) { + uint8_t nb_src_sges = cfg->nb_src_sges; + uint8_t nb_dst_sges = cfg->nb_dst_sges; + uint32_t sglen_src, sglen_dst; + + *src_sges = rte_zmalloc(NULL, nr_buf * sizeof(struct rte_dma_sge), + RTE_CACHE_LINE_SIZE); + if (*src_sges == NULL) { + printf("Error: src_sges array malloc failed.\n"); + return -1; + } + + *dst_sges = rte_zmalloc(NULL, nr_buf * sizeof(struct rte_dma_sge), + RTE_CACHE_LINE_SIZE); + if (*dst_sges == NULL) { + printf("Error: dst_sges array malloc failed.\n"); + return -1; + } + + sglen_src = cur_buf_size / nb_src_sges; + sglen_dst = cur_buf_size / nb_dst_sges; + + for (i = 0; i < nr_buf; i++) { + (*src_sges)[i].addr = rte_pktmbuf_iova((*srcs)[i]); + (*src_sges)[i].length = sglen_src; + if (!((i+1) % nb_src_sges)) + (*src_sges)[i].length += (cur_buf_size % nb_src_sges); + + (*dst_sges)[i].addr = rte_pktmbuf_iova((*dsts)[i]); + (*dst_sges)[i].length = sglen_dst; + if (!((i+1) % nb_dst_sges)) + (*dst_sges)[i].length += (cur_buf_size % nb_dst_sges); + } + } + return 0; } +static uint32_t +align_buffer_count(struct test_configure *cfg, uint32_t *nr_sgsrc, uint32_t *nr_sgdst) +{ + struct lcore_dma_map_t *ldm = &cfg->lcore_dma_map; + uint16_t nb_workers = ldm->cnt; + uint32_t nr_buf; + + nr_buf = (cfg->mem_size.cur * 1024 * 1024) / (cfg->buf_size.cur * 2); + nr_buf -= (nr_buf % nb_workers); + + if (nr_sgsrc == NULL || nr_sgdst == NULL) + return nr_buf; + + if (cfg->is_sg) { + nr_buf /= nb_workers; + nr_buf -= nr_buf % (cfg->nb_src_sges * cfg->nb_dst_sges); + nr_buf *= nb_workers; + + if (cfg->nb_dst_sges > cfg->nb_src_sges) { + *nr_sgsrc = (nr_buf / cfg->nb_dst_sges * cfg->nb_src_sges); + *nr_sgdst = nr_buf; + } else { + *nr_sgsrc = nr_buf; + *nr_sgdst = (nr_buf / cfg->nb_src_sges * cfg->nb_dst_sges); + } + } + + return nr_buf; +} + +static lcore_function_t * +get_work_function(struct test_configure *cfg) +{ + lcore_function_t *fn; + + if (cfg->is_dma) { + if (!cfg->is_sg) + fn = do_dma_plain_mem_copy; + else + fn = do_dma_sg_mem_copy; + } else { + fn = do_cpu_mem_copy; + } + + return fn; +} + int -mem_copy_benchmark(struct test_configure *cfg, bool is_dma) +mem_copy_benchmark(struct test_configure *cfg) { - uint32_t i; + uint32_t i, j; uint32_t offset; unsigned int lcore_id = 0; struct rte_mbuf **srcs = NULL, **dsts = NULL, **m = NULL; + struct rte_dma_sge *src_sges = NULL, *dst_sges = NULL; struct lcore_dma_map_t *ldm = &cfg->lcore_dma_map; unsigned int buf_size = cfg->buf_size.cur; uint16_t kick_batch = cfg->kick_batch.cur; - uint32_t nr_buf = cfg->nr_buf = (cfg->mem_size.cur * 1024 * 1024) / (cfg->buf_size.cur * 2); uint16_t nb_workers = ldm->cnt; uint16_t test_secs = cfg->test_secs; float memory = 0; @@ -469,12 +637,17 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) uint32_t avg_cycles_total; float mops, mops_total; float bandwidth, bandwidth_total; + uint32_t nr_sgsrc = 0, nr_sgdst = 0; + uint32_t nr_buf; int ret = 0; - if (setup_memory_env(cfg, &srcs, &dsts) < 0) + nr_buf = align_buffer_count(cfg, &nr_sgsrc, &nr_sgdst); + cfg->nr_buf = nr_buf; + + if (setup_memory_env(cfg, &srcs, &dsts, &src_sges, &dst_sges) < 0) goto out; - if (is_dma) + if (cfg->is_dma) if (config_dmadevs(cfg) < 0) goto out; @@ -494,7 +667,7 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) printf("lcore parameters malloc failure for lcore %d\n", lcore_id); break; } - if (is_dma) { + if (cfg->is_dma) { lcores[i]->dma_name = ldm->dma_names[i]; lcores[i]->dev_id = ldm->dma_ids[i]; lcores[i]->kick_batch = kick_batch; @@ -508,10 +681,15 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) lcores[i]->scenario_id = cfg->scenario_id; lcores[i]->lcore_id = lcore_id; - if (is_dma) - rte_eal_remote_launch(do_dma_mem_copy, (void *)(lcores[i]), lcore_id); - else - rte_eal_remote_launch(do_cpu_mem_copy, (void *)(lcores[i]), lcore_id); + if (cfg->is_sg) { + lcores[i]->sge.nb_srcs = cfg->nb_src_sges; + lcores[i]->sge.nb_dsts = cfg->nb_dst_sges; + lcores[i]->sge.srcs = src_sges + (nr_sgsrc / nb_workers * i); + lcores[i]->sge.dsts = dst_sges + (nr_sgdst / nb_workers * i); + } + + rte_eal_remote_launch(get_work_function(cfg), (void *)(lcores[i]), + lcore_id); } while (1) { @@ -543,7 +721,7 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) rte_eal_mp_wait_lcore(); - if (cfg->transfer_dir == RTE_DMA_DIR_MEM_TO_MEM) { + if (cfg->transfer_dir == RTE_DMA_DIR_MEM_TO_MEM && !cfg->is_sg) { for (i = 0; i < (nr_buf / nb_workers) * nb_workers; i++) { if (memcmp(rte_pktmbuf_mtod(srcs[i], void *), rte_pktmbuf_mtod(dsts[i], void *), @@ -553,6 +731,44 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) goto out; } } + } else if (cfg->transfer_dir == RTE_DMA_DIR_MEM_TO_MEM && cfg->is_sg) { + size_t src_remsz = buf_size % cfg->nb_src_sges; + size_t dst_remsz = buf_size % cfg->nb_dst_sges; + size_t src_sz = buf_size / cfg->nb_src_sges; + size_t dst_sz = buf_size / cfg->nb_dst_sges; + uint8_t src[buf_size], dst[buf_size]; + uint8_t *sbuf, *dbuf, *ptr; + + for (i = 0; i < (nr_buf / RTE_MAX(cfg->nb_src_sges, cfg->nb_dst_sges)); i++) { + sbuf = src; + dbuf = dst; + ptr = NULL; + + for (j = 0; j < cfg->nb_src_sges; j++) { + ptr = rte_pktmbuf_mtod(srcs[i * cfg->nb_src_sges + j], uint8_t *); + memcpy(sbuf, ptr, src_sz); + sbuf += src_sz; + } + + if (src_remsz) + memcpy(sbuf, ptr + src_sz, src_remsz); + + for (j = 0; j < cfg->nb_dst_sges; j++) { + ptr = rte_pktmbuf_mtod(dsts[i * cfg->nb_dst_sges + j], uint8_t *); + memcpy(dbuf, ptr, dst_sz); + dbuf += dst_sz; + } + + if (dst_remsz) + memcpy(dbuf, ptr + dst_sz, dst_remsz); + + if (memcmp(src, dst, buf_size) != 0) { + printf("SG Copy validation fails for buffer number %d\n", + i * cfg->nb_src_sges); + ret = -1; + goto out; + } + } } mops_total = 0; @@ -562,10 +778,8 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) calc_result(buf_size, nr_buf, nb_workers, test_secs, lcores[i]->worker_info.test_cpl, &memory, &avg_cycles, &bandwidth, &mops); - output_result(cfg->scenario_id, lcores[i]->lcore_id, - lcores[i]->dma_name, cfg->ring_size.cur, kick_batch, - avg_cycles, buf_size, nr_buf / nb_workers, memory, - bandwidth, mops, is_dma); + output_result(cfg, lcores[i], kick_batch, avg_cycles, buf_size, + nr_buf / nb_workers, memory, bandwidth, mops); mops_total += mops; bandwidth_total += bandwidth; avg_cycles_total += avg_cycles; @@ -608,13 +822,20 @@ mem_copy_benchmark(struct test_configure *cfg, bool is_dma) rte_mempool_free(dst_pool); dst_pool = NULL; + /* free sges for mbufs */ + rte_free(src_sges); + src_sges = NULL; + + rte_free(dst_sges); + dst_sges = NULL; + /* free the worker parameters */ for (i = 0; i < nb_workers; i++) { rte_free(lcores[i]); lcores[i] = NULL; } - if (is_dma) { + if (cfg->is_dma) { for (i = 0; i < nb_workers; i++) { printf("Stopping dmadev %d\n", ldm->dma_ids[i]); rte_dma_stop(ldm->dma_ids[i]); diff --git a/app/test-dma-perf/config.ini b/app/test-dma-perf/config.ini index ada0146b92..2a7e2567d3 100644 --- a/app/test-dma-perf/config.ini +++ b/app/test-dma-perf/config.ini @@ -47,6 +47,14 @@ ; "vfid" denotes VF-id of PF-id to be used for data transfer. ; Example: vchan_dev=raddr=0x400000,coreid=1,pfid=2,vfid=3 +; Parameters for DMA scatter-gather memory copy: +; +; "dma_src_sge" denotes number of source segments. +; "dma_dst_sge" denotes number of destination segments. +; +; For DMA scatter-gather memory copy, the parameters need to be configured +; and they are valid only when type is DMA_MEM_COPY. + ; To specify a configuration file, use the "--config" flag followed by the path to the file. ; To specify a result file, use the "--result" flag followed by the path to the file. @@ -67,6 +75,21 @@ lcore_dma=lcore10@0000:00:04.2, lcore11@0000:00:04.3 eal_args=--in-memory --file-prefix=test [case2] +type=DMA_MEM_COPY +mem_size=10 +buf_size=64,8192,2,MUL +dma_ring_size=1024 +dma_src_sge=4 +dma_dst_sge=1 +kick_batch=32 +src_numa_node=0 +dst_numa_node=0 +cache_flush=0 +test_seconds=2 +lcore_dma=lcore10@0000:00:04.2, lcore11@0000:00:04.3 +eal_args=--in-memory --file-prefix=test + +[case3] skip=1 type=DMA_MEM_COPY direction=dev2mem @@ -82,7 +105,7 @@ test_seconds=2 lcore_dma=lcore10@0000:00:04.2, lcore11@0000:00:04.3 eal_args=--in-memory --file-prefix=test -[case3] +[case4] type=CPU_MEM_COPY mem_size=10 buf_size=64,8192,2,MUL diff --git a/app/test-dma-perf/main.c b/app/test-dma-perf/main.c index df05bcd7df..18219918cc 100644 --- a/app/test-dma-perf/main.c +++ b/app/test-dma-perf/main.c @@ -108,10 +108,8 @@ run_test_case(struct test_configure *case_cfg) switch (case_cfg->test_type) { case TEST_TYPE_DMA_MEM_COPY: - ret = mem_copy_benchmark(case_cfg, true); - break; case TEST_TYPE_CPU_MEM_COPY: - ret = mem_copy_benchmark(case_cfg, false); + ret = mem_copy_benchmark(case_cfg); break; default: printf("Unknown test type. %s\n", case_cfg->test_type_str); @@ -365,7 +363,8 @@ load_configs(const char *path) const char *case_type; const char *transfer_dir; const char *lcore_dma; - const char *mem_size_str, *buf_size_str, *ring_size_str, *kick_batch_str; + const char *mem_size_str, *buf_size_str, *ring_size_str, *kick_batch_str, + *src_sges_str, *dst_sges_str; const char *skip; struct rte_kvargs *kvlist; const char *vchan_dev; @@ -467,6 +466,7 @@ load_configs(const char *path) rte_kvargs_free(kvlist); } + test_case->is_dma = is_dma; test_case->src_numa_node = (int)atoi(rte_cfgfile_get_entry(cfgfile, section_name, "src_numa_node")); test_case->dst_numa_node = (int)atoi(rte_cfgfile_get_entry(cfgfile, @@ -501,6 +501,39 @@ load_configs(const char *path) } else if (args_nr == 4) nb_vp++; + src_sges_str = rte_cfgfile_get_entry(cfgfile, section_name, + "dma_src_sge"); + if (src_sges_str != NULL) { + test_case->nb_src_sges = (int)atoi(rte_cfgfile_get_entry(cfgfile, + section_name, "dma_src_sge")); + } + + dst_sges_str = rte_cfgfile_get_entry(cfgfile, section_name, + "dma_dst_sge"); + if (dst_sges_str != NULL) { + test_case->nb_dst_sges = (int)atoi(rte_cfgfile_get_entry(cfgfile, + section_name, "dma_dst_sge")); + } + + if ((src_sges_str != NULL && dst_sges_str == NULL) || + (src_sges_str == NULL && dst_sges_str != NULL)) { + printf("parse dma_src_sge, dma_dst_sge error in case %d.\n", + i + 1); + test_case->is_valid = false; + continue; + } else if (src_sges_str != NULL && dst_sges_str != NULL) { + test_case->is_sg = true; + + if (test_case->nb_src_sges == 0 || test_case->nb_dst_sges == 0) { + printf("dma_src_sge and dma_dst_sge can not be 0 in case %d.\n", + i + 1); + test_case->is_valid = false; + continue; + } + } else { + test_case->is_sg = false; + } + kick_batch_str = rte_cfgfile_get_entry(cfgfile, section_name, "kick_batch"); args_nr = parse_entry(kick_batch_str, &test_case->kick_batch); if (args_nr < 0) { diff --git a/app/test-dma-perf/main.h b/app/test-dma-perf/main.h index 1123e7524a..e88d72f54f 100644 --- a/app/test-dma-perf/main.h +++ b/app/test-dma-perf/main.h @@ -53,11 +53,14 @@ struct test_configure { uint16_t dst_numa_node; uint16_t opcode; bool is_dma; + bool is_sg; struct lcore_dma_map_t lcore_dma_map; struct test_configure_entry mem_size; struct test_configure_entry buf_size; struct test_configure_entry ring_size; struct test_configure_entry kick_batch; + uint8_t nb_src_sges; + uint8_t nb_dst_sges; uint8_t cache_flush; uint32_t nr_buf; uint16_t test_secs; @@ -66,6 +69,6 @@ struct test_configure { struct test_vchan_dev_config vchan_dev; }; -int mem_copy_benchmark(struct test_configure *cfg, bool is_dma); +int mem_copy_benchmark(struct test_configure *cfg); #endif /* MAIN_H */