From patchwork Thu May 30 08:06:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Jozwiak X-Patchwork-Id: 53884 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2C55A1B94A; Thu, 30 May 2019 10:06:41 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id B799C1B94A for ; Thu, 30 May 2019 10:06:38 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 01:06:38 -0700 X-ExtLoop1: 1 Received: from tjozwiax-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.103.104.46]) by fmsmga001.fm.intel.com with ESMTP; 30 May 2019 01:06:36 -0700 From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm.gmail.com@dpdk.org, shallyv@marvell.com Date: Thu, 30 May 2019 10:06:22 +0200 Message-Id: <1559203588-24046-2-git-send-email-tomaszx.jozwiak@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> References: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> Subject: [dpdk-dev] [PATCH v1 1/7] app/test-compress-perf: add weak functions for multi-cores test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds a template functions for multi-cores performance version of compress-perf-tool. Signed-off-by: Tomasz Jozwiak --- app/test-compress-perf/Makefile | 3 +- app/test-compress-perf/comp_perf.h | 61 +++ app/test-compress-perf/comp_perf_options.h | 45 +- app/test-compress-perf/comp_perf_options_parse.c | 24 +- app/test-compress-perf/comp_perf_test_common.c | 285 +++++++++++ app/test-compress-perf/comp_perf_test_common.h | 41 ++ app/test-compress-perf/main.c | 624 ++++++++++------------- app/test-compress-perf/meson.build | 3 +- 8 files changed, 685 insertions(+), 401 deletions(-) create mode 100644 app/test-compress-perf/comp_perf.h create mode 100644 app/test-compress-perf/comp_perf_test_common.c create mode 100644 app/test-compress-perf/comp_perf_test_common.h diff --git a/app/test-compress-perf/Makefile b/app/test-compress-perf/Makefile index d20e17e..de74129 100644 --- a/app/test-compress-perf/Makefile +++ b/app/test-compress-perf/Makefile @@ -12,7 +12,6 @@ CFLAGS += -O3 # all source are stored in SRCS-y SRCS-y := main.c SRCS-y += comp_perf_options_parse.c -SRCS-y += comp_perf_test_verify.c -SRCS-y += comp_perf_test_benchmark.c +SRCS-y += comp_perf_test_common.c include $(RTE_SDK)/mk/rte.app.mk diff --git a/app/test-compress-perf/comp_perf.h b/app/test-compress-perf/comp_perf.h new file mode 100644 index 0000000..144ad8a --- /dev/null +++ b/app/test-compress-perf/comp_perf.h @@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#ifndef _COMP_PERF_ +#define _COMP_PERF_ + +#include + +struct comp_test_data; + +typedef void *(*cperf_constructor_t)( + uint8_t dev_id, + uint16_t qp_id, + struct comp_test_data *options); + +typedef int (*cperf_runner_t)(void *test_ctx); +typedef void (*cperf_destructor_t)(void *test_ctx); + +struct cperf_test { + cperf_constructor_t constructor; + cperf_runner_t runner; + cperf_destructor_t destructor; +}; + +/* Needed for weak functions*/ + +void * +cperf_benchmark_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused); + +void +cperf_benchmark_test_destructor(void *arg __rte_unused); + +int +cperf_benchmark_test_runner(void *test_ctx __rte_unused); + +void * +cperf_verify_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused); + +void +cperf_verify_test_destructor(void *arg __rte_unused); + +int +cperf_verify_test_runner(void *test_ctx __rte_unused); + +void * +cperf_pmd_cyclecount_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused); + +void +cperf_pmd_cyclecount_test_destructor(void *arg __rte_unused); + +int +cperf_pmd_cyclecount_test_runner(void *test_ctx __rte_unused); + +#endif /* _COMP_PERF_ */ diff --git a/app/test-compress-perf/comp_perf_options.h b/app/test-compress-perf/comp_perf_options.h index f87751d..79e63d5 100644 --- a/app/test-compress-perf/comp_perf_options.h +++ b/app/test-compress-perf/comp_perf_options.h @@ -13,6 +13,24 @@ #define MAX_MBUF_DATA_SIZE (UINT16_MAX - RTE_PKTMBUF_HEADROOM) #define MAX_SEG_SIZE ((int)(MAX_MBUF_DATA_SIZE / EXPANSE_RATIO)) +extern const char *cperf_test_type_strs[]; + +/* Cleanup state machine */ +enum cleanup_st { + ST_CLEAR = 0, + ST_TEST_DATA, + ST_COMPDEV, + ST_INPUT_DATA, + ST_MEMORY_ALLOC, + ST_DURING_TEST +}; + +enum cperf_perf_test_type { + CPERF_TEST_TYPE_BENCHMARK, + CPERF_TEST_TYPE_VERIFY, + CPERF_TEST_TYPE_PMDCC +}; + enum comp_operation { COMPRESS_ONLY, DECOMPRESS_ONLY, @@ -30,37 +48,26 @@ struct range_list { struct comp_test_data { char driver_name[64]; char input_file[64]; - struct rte_mbuf **comp_bufs; - struct rte_mbuf **decomp_bufs; - uint32_t total_bufs; + enum cperf_perf_test_type test; + uint8_t *input_data; size_t input_data_sz; - uint8_t *compressed_data; - uint8_t *decompressed_data; - struct rte_mempool *comp_buf_pool; - struct rte_mempool *decomp_buf_pool; - struct rte_mempool *op_pool; - int8_t cdev_id; + uint16_t nb_qps; uint16_t seg_sz; uint16_t out_seg_sz; uint16_t burst_sz; uint32_t pool_sz; uint32_t num_iter; uint16_t max_sgl_segs; + enum rte_comp_huffman huffman_enc; enum comp_operation test_op; int window_sz; - struct range_list level; - /* Store TSC duration for all levels (including level 0) */ - uint64_t comp_tsc_duration[RTE_COMP_LEVEL_MAX + 1]; - uint64_t decomp_tsc_duration[RTE_COMP_LEVEL_MAX + 1]; - size_t comp_data_sz; - size_t decomp_data_sz; + struct range_list level_lst; + uint8_t level; + double ratio; - double comp_gbps; - double decomp_gbps; - double comp_tsc_byte; - double decomp_tsc_byte; + enum cleanup_st cleanup; }; int diff --git a/app/test-compress-perf/comp_perf_options_parse.c b/app/test-compress-perf/comp_perf_options_parse.c index 2fb6fb4..bc4b98a 100644 --- a/app/test-compress-perf/comp_perf_options_parse.c +++ b/app/test-compress-perf/comp_perf_options_parse.c @@ -466,19 +466,20 @@ parse_level(struct comp_test_data *test_data, const char *arg) * Try parsing the argument as a range, if it fails, * arse it as a list */ - if (parse_range(arg, &test_data->level.min, &test_data->level.max, - &test_data->level.inc) < 0) { - ret = parse_list(arg, test_data->level.list, - &test_data->level.min, - &test_data->level.max); + if (parse_range(arg, &test_data->level_lst.min, + &test_data->level_lst.max, + &test_data->level_lst.inc) < 0) { + ret = parse_list(arg, test_data->level_lst.list, + &test_data->level_lst.min, + &test_data->level_lst.max); if (ret < 0) { RTE_LOG(ERR, USER1, "Failed to parse compression level/s\n"); return -1; } - test_data->level.count = ret; + test_data->level_lst.count = ret; - if (test_data->level.max > RTE_COMP_LEVEL_MAX) { + if (test_data->level_lst.max > RTE_COMP_LEVEL_MAX) { RTE_LOG(ERR, USER1, "Level cannot be higher than %u\n", RTE_COMP_LEVEL_MAX); return -1; @@ -498,7 +499,6 @@ struct long_opt_parser { }; static struct option lgopts[] = { - { CPERF_DRIVER_NAME, required_argument, 0, 0 }, { CPERF_TEST_FILE, required_argument, 0, 0 }, { CPERF_SEG_SIZE, required_argument, 0, 0 }, @@ -572,7 +572,6 @@ comp_perf_options_parse(struct comp_test_data *test_data, int argc, char **argv) void comp_perf_options_default(struct comp_test_data *test_data) { - test_data->cdev_id = -1; test_data->seg_sz = 2048; test_data->burst_sz = 32; test_data->pool_sz = 8192; @@ -581,9 +580,10 @@ comp_perf_options_default(struct comp_test_data *test_data) test_data->huffman_enc = RTE_COMP_HUFFMAN_DYNAMIC; test_data->test_op = COMPRESS_DECOMPRESS; test_data->window_sz = -1; - test_data->level.min = 1; - test_data->level.max = 9; - test_data->level.inc = 1; + test_data->level_lst.min = 1; + test_data->level_lst.max = 9; + test_data->level_lst.inc = 1; + test_data->test = CPERF_TEST_TYPE_BENCHMARK; } int diff --git a/app/test-compress-perf/comp_perf_test_common.c b/app/test-compress-perf/comp_perf_test_common.c new file mode 100644 index 0000000..1d2b25e --- /dev/null +++ b/app/test-compress-perf/comp_perf_test_common.c @@ -0,0 +1,285 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#include +#include +#include +#include + +#include "comp_perf_options.h" +#include "comp_perf_test_verify.h" +#include "comp_perf_test_benchmark.h" +#include "comp_perf.h" +#include "comp_perf_test_common.h" + +#define DIV_CEIL(a, b) ((a) / (b) + ((a) % (b) != 0)) + +int +param_range_check(uint16_t size, const struct rte_param_log2_range *range) +{ + unsigned int next_size; + + /* Check lower/upper bounds */ + if (size < range->min) + return -1; + + if (size > range->max) + return -1; + + /* If range is actually only one value, size is correct */ + if (range->increment == 0) + return 0; + + /* Check if value is one of the supported sizes */ + for (next_size = range->min; next_size <= range->max; + next_size += range->increment) + if (size == next_size) + return 0; + + return -1; +} + +static uint32_t +find_buf_size(uint32_t input_size) +{ + uint32_t i; + + /* From performance point of view the buffer size should be a + * power of 2 but also should be enough to store incompressible data + */ + + /* We're looking for nearest power of 2 buffer size, which is greather + * than input_size + */ + uint32_t size = + !input_size ? MIN_COMPRESSED_BUF_SIZE : (input_size << 1); + + for (i = UINT16_MAX + 1; !(i & size); i >>= 1) + ; + + return i > ((UINT16_MAX + 1) >> 1) + ? (uint32_t)((float)input_size * EXPANSE_RATIO) + : i; +} + +void +comp_perf_free_memory(struct cperf_mem_resources *mem) +{ + uint32_t i; + + for (i = 0; i < mem->total_bufs; i++) { + rte_pktmbuf_free(mem->comp_bufs[i]); + rte_pktmbuf_free(mem->decomp_bufs[i]); + } + + rte_free(mem->decomp_bufs); + rte_free(mem->comp_bufs); + rte_free(mem->decompressed_data); + rte_free(mem->compressed_data); + rte_mempool_free(mem->op_pool); + rte_mempool_free(mem->decomp_buf_pool); + rte_mempool_free(mem->comp_buf_pool); +} + +int +comp_perf_allocate_memory(struct comp_test_data *test_data, + struct cperf_mem_resources *mem) +{ + test_data->out_seg_sz = find_buf_size(test_data->seg_sz); + /* Number of segments for input and output + * (compression and decompression) + */ + uint32_t total_segs = DIV_CEIL(test_data->input_data_sz, + test_data->seg_sz); + char pool_name[32] = ""; + + snprintf(pool_name, sizeof(pool_name), "comp_buf_pool_%u_qp_%u", + mem->dev_id, mem->qp_id); + mem->comp_buf_pool = rte_pktmbuf_pool_create(pool_name, + total_segs, + 0, 0, + test_data->out_seg_sz + RTE_PKTMBUF_HEADROOM, + rte_socket_id()); + if (mem->comp_buf_pool == NULL) { + RTE_LOG(ERR, USER1, "Mbuf mempool could not be created\n"); + return -1; + } + + snprintf(pool_name, sizeof(pool_name), "decomp_buf_pool_%u_qp_%u", + mem->dev_id, mem->qp_id); + mem->decomp_buf_pool = rte_pktmbuf_pool_create(pool_name, + total_segs, + 0, 0, test_data->seg_sz + RTE_PKTMBUF_HEADROOM, + rte_socket_id()); + if (mem->decomp_buf_pool == NULL) { + RTE_LOG(ERR, USER1, "Mbuf mempool could not be created\n"); + return -1; + } + + mem->total_bufs = DIV_CEIL(total_segs, test_data->max_sgl_segs); + + snprintf(pool_name, sizeof(pool_name), "op_pool_%u_qp_%u", + mem->dev_id, mem->qp_id); + mem->op_pool = rte_comp_op_pool_create(pool_name, + mem->total_bufs, + 0, 0, rte_socket_id()); + if (mem->op_pool == NULL) { + RTE_LOG(ERR, USER1, "Comp op mempool could not be created\n"); + return -1; + } + + /* + * Compressed data might be a bit larger than input data, + * if data cannot be compressed + */ + mem->compressed_data = rte_zmalloc_socket(NULL, + test_data->input_data_sz * EXPANSE_RATIO + + MIN_COMPRESSED_BUF_SIZE, 0, + rte_socket_id()); + if (mem->compressed_data == NULL) { + RTE_LOG(ERR, USER1, "Memory to hold the data from the input " + "file could not be allocated\n"); + return -1; + } + + mem->decompressed_data = rte_zmalloc_socket(NULL, + test_data->input_data_sz, 0, + rte_socket_id()); + if (mem->decompressed_data == NULL) { + RTE_LOG(ERR, USER1, "Memory to hold the data from the input " + "file could not be allocated\n"); + return -1; + } + + mem->comp_bufs = rte_zmalloc_socket(NULL, + mem->total_bufs * sizeof(struct rte_mbuf *), + 0, rte_socket_id()); + if (mem->comp_bufs == NULL) { + RTE_LOG(ERR, USER1, "Memory to hold the compression mbufs" + " could not be allocated\n"); + return -1; + } + + mem->decomp_bufs = rte_zmalloc_socket(NULL, + mem->total_bufs * sizeof(struct rte_mbuf *), + 0, rte_socket_id()); + if (mem->decomp_bufs == NULL) { + RTE_LOG(ERR, USER1, "Memory to hold the decompression mbufs" + " could not be allocated\n"); + return -1; + } + return 0; +} + +int +prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem) +{ + uint32_t remaining_data = test_data->input_data_sz; + uint8_t *input_data_ptr = test_data->input_data; + size_t data_sz; + uint8_t *data_addr; + uint32_t i, j; + + for (i = 0; i < mem->total_bufs; i++) { + /* Allocate data in input mbuf and copy data from input file */ + mem->decomp_bufs[i] = + rte_pktmbuf_alloc(mem->decomp_buf_pool); + if (mem->decomp_bufs[i] == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"); + return -1; + } + + data_sz = RTE_MIN(remaining_data, test_data->seg_sz); + data_addr = (uint8_t *) rte_pktmbuf_append( + mem->decomp_bufs[i], data_sz); + if (data_addr == NULL) { + RTE_LOG(ERR, USER1, "Could not append data\n"); + return -1; + } + rte_memcpy(data_addr, input_data_ptr, data_sz); + + input_data_ptr += data_sz; + remaining_data -= data_sz; + + /* Already one segment in the mbuf */ + uint16_t segs_per_mbuf = 1; + + /* Chain mbufs if needed for input mbufs */ + while (segs_per_mbuf < test_data->max_sgl_segs + && remaining_data > 0) { + struct rte_mbuf *next_seg = + rte_pktmbuf_alloc(mem->decomp_buf_pool); + + if (next_seg == NULL) { + RTE_LOG(ERR, USER1, + "Could not allocate mbuf\n"); + return -1; + } + + data_sz = RTE_MIN(remaining_data, test_data->seg_sz); + data_addr = (uint8_t *)rte_pktmbuf_append(next_seg, + data_sz); + + if (data_addr == NULL) { + RTE_LOG(ERR, USER1, "Could not append data\n"); + return -1; + } + + rte_memcpy(data_addr, input_data_ptr, data_sz); + input_data_ptr += data_sz; + remaining_data -= data_sz; + + if (rte_pktmbuf_chain(mem->decomp_bufs[i], + next_seg) < 0) { + RTE_LOG(ERR, USER1, "Could not chain mbufs\n"); + return -1; + } + segs_per_mbuf++; + } + + /* Allocate data in output mbuf */ + mem->comp_bufs[i] = + rte_pktmbuf_alloc(mem->comp_buf_pool); + if (mem->comp_bufs[i] == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"); + return -1; + } + data_addr = (uint8_t *) rte_pktmbuf_append( + mem->comp_bufs[i], + test_data->out_seg_sz); + if (data_addr == NULL) { + RTE_LOG(ERR, USER1, "Could not append data\n"); + return -1; + } + + /* Chain mbufs if needed for output mbufs */ + for (j = 1; j < segs_per_mbuf; j++) { + struct rte_mbuf *next_seg = + rte_pktmbuf_alloc(mem->comp_buf_pool); + + if (next_seg == NULL) { + RTE_LOG(ERR, USER1, + "Could not allocate mbuf\n"); + return -1; + } + + data_addr = (uint8_t *)rte_pktmbuf_append(next_seg, + test_data->out_seg_sz); + + if (data_addr == NULL) { + RTE_LOG(ERR, USER1, "Could not append data\n"); + return -1; + } + + if (rte_pktmbuf_chain(mem->comp_bufs[i], + next_seg) < 0) { + RTE_LOG(ERR, USER1, "Could not chain mbufs\n"); + return -1; + } + } + } + + return 0; +} + diff --git a/app/test-compress-perf/comp_perf_test_common.h b/app/test-compress-perf/comp_perf_test_common.h new file mode 100644 index 0000000..9c11e3a --- /dev/null +++ b/app/test-compress-perf/comp_perf_test_common.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#ifndef _COMP_PERF_TEST_COMMON_H_ +#define _COMP_PERF_TEST_COMMON_H_ + +#include + +#include + +struct cperf_mem_resources { + uint8_t dev_id; + uint16_t qp_id; + uint8_t lcore_id; + uint32_t total_bufs; + uint8_t *compressed_data; + uint8_t *decompressed_data; + + struct rte_mbuf **comp_bufs; + struct rte_mbuf **decomp_bufs; + + struct rte_mempool *comp_buf_pool; + struct rte_mempool *decomp_buf_pool; + struct rte_mempool *op_pool; +}; + +int +param_range_check(uint16_t size, const struct rte_param_log2_range *range); + +void +comp_perf_free_memory(struct cperf_mem_resources *mem); + +int +comp_perf_allocate_memory(struct comp_test_data *test_data, + struct cperf_mem_resources *mem); + +int +prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem); + +#endif /* _COMP_PERF_TEST_COMMON_H_ */ diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index c2a45d1..f393c1b 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -8,56 +8,44 @@ #include #include "comp_perf_options.h" -#include "comp_perf_test_verify.h" -#include "comp_perf_test_benchmark.h" +#include "comp_perf.h" +#include "comp_perf_test_common.h" #define NUM_MAX_XFORMS 16 #define NUM_MAX_INFLIGHT_OPS 512 -#define DIV_CEIL(a, b) ((a) / (b) + ((a) % (b) != 0)) - -/* Cleanup state machine */ -static enum cleanup_st { - ST_CLEAR = 0, - ST_TEST_DATA, - ST_COMPDEV, - ST_INPUT_DATA, - ST_MEMORY_ALLOC, - ST_PREPARE_BUF, - ST_DURING_TEST -} cleanup = ST_CLEAR; - -static int -param_range_check(uint16_t size, const struct rte_param_log2_range *range) -{ - unsigned int next_size; - - /* Check lower/upper bounds */ - if (size < range->min) - return -1; - - if (size > range->max) - return -1; - - /* If range is actually only one value, size is correct */ - if (range->increment == 0) - return 0; - - /* Check if value is one of the supported sizes */ - for (next_size = range->min; next_size <= range->max; - next_size += range->increment) - if (size == next_size) - return 0; - - return -1; -} +__extension__ +const char *cperf_test_type_strs[] = { + [CPERF_TEST_TYPE_BENCHMARK] = "benchmark", + [CPERF_TEST_TYPE_VERIFY] = "verify", + [CPERF_TEST_TYPE_PMDCC] = "pmd-cyclecount" +}; + +__extension__ +static const struct cperf_test cperf_testmap[] = { + [CPERF_TEST_TYPE_BENCHMARK] = { + cperf_benchmark_test_constructor, + cperf_benchmark_test_runner, + cperf_benchmark_test_destructor + }, + [CPERF_TEST_TYPE_VERIFY] = { + cperf_verify_test_constructor, + cperf_verify_test_runner, + cperf_verify_test_destructor + }, + [CPERF_TEST_TYPE_PMDCC] = { + cperf_pmd_cyclecount_test_constructor, + cperf_pmd_cyclecount_test_runner, + cperf_pmd_cyclecount_test_destructor + } +}; static int -comp_perf_check_capabilities(struct comp_test_data *test_data) +comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id) { const struct rte_compressdev_capabilities *cap; - cap = rte_compressdev_capability_get(test_data->cdev_id, + cap = rte_compressdev_capability_get(cdev_id, RTE_COMP_ALGO_DEFLATE); if (cap == NULL) { @@ -105,7 +93,7 @@ comp_perf_check_capabilities(struct comp_test_data *test_data) } /* Level 0 support */ - if (test_data->level.min == 0 && + if (test_data->level_lst.min == 0 && (comp_flags & RTE_COMP_FF_NONCOMPRESSED_BLOCKS) == 0) { RTE_LOG(ERR, USER1, "Compress device does not support " "level 0 (no compression)\n"); @@ -115,110 +103,105 @@ comp_perf_check_capabilities(struct comp_test_data *test_data) return 0; } -static uint32_t -find_buf_size(uint32_t input_size) +static int +comp_perf_initialize_compressdev(struct comp_test_data *test_data, + uint8_t *enabled_cdevs) { - uint32_t i; - - /* From performance point of view the buffer size should be a - * power of 2 but also should be enough to store incompressible data - */ + uint8_t enabled_cdev_count, nb_lcores, cdev_id; + unsigned int i, j; + int ret; - /* We're looking for nearest power of 2 buffer size, which is greather - * than input_size - */ - uint32_t size = - !input_size ? MIN_COMPRESSED_BUF_SIZE : (input_size << 1); + enabled_cdev_count = rte_compressdev_devices_get(test_data->driver_name, + enabled_cdevs, RTE_COMPRESS_MAX_DEVS); + if (enabled_cdev_count == 0) { + RTE_LOG(ERR, USER1, "No compress devices type %s available\n", + test_data->driver_name); + return -EINVAL; + } - for (i = UINT16_MAX + 1; !(i & size); i >>= 1) - ; + nb_lcores = rte_lcore_count() - 1; + /* + * Use fewer devices, + * if there are more available than cores. + */ + if (enabled_cdev_count > nb_lcores) { + enabled_cdev_count = nb_lcores; + RTE_LOG(INFO, USER1, + " There's more available devices than cores!" + " The number of devices has been aligned to %d cores\n", + nb_lcores); + } - return i > ((UINT16_MAX + 1) >> 1) - ? (uint32_t)((float)input_size * EXPANSE_RATIO) - : i; -} + /* + * Calculate number of needed queue pairs, based on the amount + * of available number of logical cores and compression devices. + * For instance, if there are 4 cores and 2 compression devices, + * 2 queue pairs will be set up per device. + * One queue pair per one core. + * if e.g.: there're 3 cores and 2 compression devices, + * 2 queue pairs will be set up per device but one queue pair + * will left unused in the last one device + */ + test_data->nb_qps = (nb_lcores % enabled_cdev_count) ? + (nb_lcores / enabled_cdev_count) + 1 : + nb_lcores / enabled_cdev_count; -static int -comp_perf_allocate_memory(struct comp_test_data *test_data) -{ + for (i = 0; i < enabled_cdev_count && + i < RTE_COMPRESS_MAX_DEVS; i++) { + cdev_id = enabled_cdevs[i]; - test_data->out_seg_sz = find_buf_size(test_data->seg_sz); - /* Number of segments for input and output - * (compression and decompression) - */ - uint32_t total_segs = DIV_CEIL(test_data->input_data_sz, - test_data->seg_sz); - test_data->comp_buf_pool = rte_pktmbuf_pool_create("comp_buf_pool", - total_segs, - 0, 0, - test_data->out_seg_sz + RTE_PKTMBUF_HEADROOM, - rte_socket_id()); - if (test_data->comp_buf_pool == NULL) { - RTE_LOG(ERR, USER1, "Mbuf mempool could not be created\n"); - return -1; - } + struct rte_compressdev_info cdev_info; + uint8_t socket_id = rte_compressdev_socket_id(cdev_id); - cleanup = ST_MEMORY_ALLOC; - test_data->decomp_buf_pool = rte_pktmbuf_pool_create("decomp_buf_pool", - total_segs, - 0, 0, test_data->seg_sz + RTE_PKTMBUF_HEADROOM, - rte_socket_id()); - if (test_data->decomp_buf_pool == NULL) { - RTE_LOG(ERR, USER1, "Mbuf mempool could not be created\n"); - return -1; - } + rte_compressdev_info_get(cdev_id, &cdev_info); + if (test_data->nb_qps > cdev_info.max_nb_queue_pairs) { + RTE_LOG(ERR, USER1, + "Number of needed queue pairs is higher " + "than the maximum number of queue pairs " + "per device.\n"); + RTE_LOG(ERR, USER1, + "Lower the number of cores or increase " + "the number of crypto devices\n"); + return -EINVAL; + } - test_data->total_bufs = DIV_CEIL(total_segs, test_data->max_sgl_segs); + if (comp_perf_check_capabilities(test_data, cdev_id) < 0) + return -EINVAL; - test_data->op_pool = rte_comp_op_pool_create("op_pool", - test_data->total_bufs, - 0, 0, rte_socket_id()); - if (test_data->op_pool == NULL) { - RTE_LOG(ERR, USER1, "Comp op mempool could not be created\n"); - return -1; - } + /* Configure compressdev */ + struct rte_compressdev_config config = { + .socket_id = socket_id, + .nb_queue_pairs = test_data->nb_qps, + .max_nb_priv_xforms = NUM_MAX_XFORMS, + .max_nb_streams = 0 + }; - /* - * Compressed data might be a bit larger than input data, - * if data cannot be compressed - */ - test_data->compressed_data = rte_zmalloc_socket(NULL, - test_data->input_data_sz * EXPANSE_RATIO - + MIN_COMPRESSED_BUF_SIZE, 0, - rte_socket_id()); - if (test_data->compressed_data == NULL) { - RTE_LOG(ERR, USER1, "Memory to hold the data from the input " - "file could not be allocated\n"); - return -1; - } + if (rte_compressdev_configure(cdev_id, &config) < 0) { + RTE_LOG(ERR, USER1, "Device configuration failed\n"); + return -EINVAL; + } - test_data->decompressed_data = rte_zmalloc_socket(NULL, - test_data->input_data_sz, 0, - rte_socket_id()); - if (test_data->decompressed_data == NULL) { - RTE_LOG(ERR, USER1, "Memory to hold the data from the input " - "file could not be allocated\n"); - return -1; - } + for (j = 0; j < test_data->nb_qps; j++) { + ret = rte_compressdev_queue_pair_setup(cdev_id, j, + NUM_MAX_INFLIGHT_OPS, socket_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, + "Failed to setup queue pair %u on compressdev %u", + j, cdev_id); + return -EINVAL; + } + } - test_data->comp_bufs = rte_zmalloc_socket(NULL, - test_data->total_bufs * sizeof(struct rte_mbuf *), - 0, rte_socket_id()); - if (test_data->comp_bufs == NULL) { - RTE_LOG(ERR, USER1, "Memory to hold the compression mbufs" - " could not be allocated\n"); - return -1; + ret = rte_compressdev_start(cdev_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, + "Failed to start device %u: error %d\n", + cdev_id, ret); + return -EPERM; + } } - test_data->decomp_bufs = rte_zmalloc_socket(NULL, - test_data->total_bufs * sizeof(struct rte_mbuf *), - 0, rte_socket_id()); - if (test_data->decomp_bufs == NULL) { - RTE_LOG(ERR, USER1, "Memory to hold the decompression mbufs" - " could not be allocated\n"); - return -1; - } - return 0; + return enabled_cdev_count; } static int @@ -244,7 +227,8 @@ comp_perf_dump_input_data(struct comp_test_data *test_data) if (test_data->input_data_sz == 0) test_data->input_data_sz = actual_file_sz; - if (fseek(f, 0, SEEK_SET) != 0) { + if (test_data->input_data_sz <= 0 || actual_file_sz <= 0 || + fseek(f, 0, SEEK_SET) != 0) { RTE_LOG(ERR, USER1, "Size of input could not be calculated\n"); goto end; } @@ -294,187 +278,18 @@ comp_perf_dump_input_data(struct comp_test_data *test_data) return ret; } -static int -comp_perf_initialize_compressdev(struct comp_test_data *test_data) -{ - uint8_t enabled_cdev_count; - uint8_t enabled_cdevs[RTE_COMPRESS_MAX_DEVS]; - - enabled_cdev_count = rte_compressdev_devices_get(test_data->driver_name, - enabled_cdevs, RTE_COMPRESS_MAX_DEVS); - if (enabled_cdev_count == 0) { - RTE_LOG(ERR, USER1, "No compress devices type %s available\n", - test_data->driver_name); - return -EINVAL; - } - - if (enabled_cdev_count > 1) - RTE_LOG(INFO, USER1, - "Only the first compress device will be used\n"); - - test_data->cdev_id = enabled_cdevs[0]; - - if (comp_perf_check_capabilities(test_data) < 0) - return -1; - - /* Configure compressdev (one device, one queue pair) */ - struct rte_compressdev_config config = { - .socket_id = rte_socket_id(), - .nb_queue_pairs = 1, - .max_nb_priv_xforms = NUM_MAX_XFORMS, - .max_nb_streams = 0 - }; - - if (rte_compressdev_configure(test_data->cdev_id, &config) < 0) { - RTE_LOG(ERR, USER1, "Device configuration failed\n"); - return -1; - } - - if (rte_compressdev_queue_pair_setup(test_data->cdev_id, 0, - NUM_MAX_INFLIGHT_OPS, rte_socket_id()) < 0) { - RTE_LOG(ERR, USER1, "Queue pair setup failed\n"); - return -1; - } - - if (rte_compressdev_start(test_data->cdev_id) < 0) { - RTE_LOG(ERR, USER1, "Device could not be started\n"); - return -1; - } - - return 0; -} - -static int -prepare_bufs(struct comp_test_data *test_data) -{ - uint32_t remaining_data = test_data->input_data_sz; - uint8_t *input_data_ptr = test_data->input_data; - size_t data_sz; - uint8_t *data_addr; - uint32_t i, j; - - for (i = 0; i < test_data->total_bufs; i++) { - /* Allocate data in input mbuf and copy data from input file */ - test_data->decomp_bufs[i] = - rte_pktmbuf_alloc(test_data->decomp_buf_pool); - if (test_data->decomp_bufs[i] == NULL) { - RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"); - return -1; - } - - cleanup = ST_PREPARE_BUF; - data_sz = RTE_MIN(remaining_data, test_data->seg_sz); - data_addr = (uint8_t *) rte_pktmbuf_append( - test_data->decomp_bufs[i], data_sz); - if (data_addr == NULL) { - RTE_LOG(ERR, USER1, "Could not append data\n"); - return -1; - } - rte_memcpy(data_addr, input_data_ptr, data_sz); - - input_data_ptr += data_sz; - remaining_data -= data_sz; - - /* Already one segment in the mbuf */ - uint16_t segs_per_mbuf = 1; - - /* Chain mbufs if needed for input mbufs */ - while (segs_per_mbuf < test_data->max_sgl_segs - && remaining_data > 0) { - struct rte_mbuf *next_seg = - rte_pktmbuf_alloc(test_data->decomp_buf_pool); - - if (next_seg == NULL) { - RTE_LOG(ERR, USER1, - "Could not allocate mbuf\n"); - return -1; - } - - data_sz = RTE_MIN(remaining_data, test_data->seg_sz); - data_addr = (uint8_t *)rte_pktmbuf_append(next_seg, - data_sz); - - if (data_addr == NULL) { - RTE_LOG(ERR, USER1, "Could not append data\n"); - return -1; - } - - rte_memcpy(data_addr, input_data_ptr, data_sz); - input_data_ptr += data_sz; - remaining_data -= data_sz; - - if (rte_pktmbuf_chain(test_data->decomp_bufs[i], - next_seg) < 0) { - RTE_LOG(ERR, USER1, "Could not chain mbufs\n"); - return -1; - } - segs_per_mbuf++; - } - - /* Allocate data in output mbuf */ - test_data->comp_bufs[i] = - rte_pktmbuf_alloc(test_data->comp_buf_pool); - if (test_data->comp_bufs[i] == NULL) { - RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"); - return -1; - } - data_addr = (uint8_t *) rte_pktmbuf_append( - test_data->comp_bufs[i], - test_data->out_seg_sz); - if (data_addr == NULL) { - RTE_LOG(ERR, USER1, "Could not append data\n"); - return -1; - } - - /* Chain mbufs if needed for output mbufs */ - for (j = 1; j < segs_per_mbuf; j++) { - struct rte_mbuf *next_seg = - rte_pktmbuf_alloc(test_data->comp_buf_pool); - - if (next_seg == NULL) { - RTE_LOG(ERR, USER1, - "Could not allocate mbuf\n"); - return -1; - } - - data_addr = (uint8_t *)rte_pktmbuf_append(next_seg, - test_data->out_seg_sz); - - if (data_addr == NULL) { - RTE_LOG(ERR, USER1, "Could not append data\n"); - return -1; - } - - if (rte_pktmbuf_chain(test_data->comp_bufs[i], - next_seg) < 0) { - RTE_LOG(ERR, USER1, "Could not chain mbufs\n"); - return -1; - } - } - } - - return 0; -} - -static void -free_bufs(struct comp_test_data *test_data) -{ - uint32_t i; - - for (i = 0; i < test_data->total_bufs; i++) { - rte_pktmbuf_free(test_data->comp_bufs[i]); - rte_pktmbuf_free(test_data->decomp_bufs[i]); - } -} - - - int main(int argc, char **argv) { - uint8_t level, level_idx = 0; + uint8_t level_idx = 0; int ret, i; struct comp_test_data *test_data; + void *ctx[RTE_MAX_LCORE] = {}; + uint8_t enabled_cdevs[RTE_COMPRESS_MAX_DEVS]; + int nb_compressdevs = 0; + uint16_t total_nb_qps = 0; + uint8_t cdev_id; + uint32_t lcore_id; /* Initialise DPDK EAL */ ret = rte_eal_init(argc, argv); @@ -491,7 +306,7 @@ main(int argc, char **argv) rte_socket_id()); ret = EXIT_SUCCESS; - cleanup = ST_TEST_DATA; + test_data->cleanup = ST_TEST_DATA; comp_perf_options_default(test_data); if (comp_perf_options_parse(test_data, argc, argv) < 0) { @@ -506,96 +321,111 @@ main(int argc, char **argv) goto end; } - if (comp_perf_initialize_compressdev(test_data) < 0) { - ret = EXIT_FAILURE; - goto end; - } + nb_compressdevs = + comp_perf_initialize_compressdev(test_data, enabled_cdevs); - cleanup = ST_COMPDEV; - if (comp_perf_dump_input_data(test_data) < 0) { + if (nb_compressdevs < 1) { ret = EXIT_FAILURE; goto end; } - cleanup = ST_INPUT_DATA; - if (comp_perf_allocate_memory(test_data) < 0) { + test_data->cleanup = ST_COMPDEV; + if (comp_perf_dump_input_data(test_data) < 0) { ret = EXIT_FAILURE; goto end; } - if (prepare_bufs(test_data) < 0) { - ret = EXIT_FAILURE; - goto end; - } + test_data->cleanup = ST_INPUT_DATA; - if (test_data->level.inc != 0) - level = test_data->level.min; + if (test_data->level_lst.inc != 0) + test_data->level = test_data->level_lst.min; else - level = test_data->level.list[0]; + test_data->level = test_data->level_lst.list[0]; printf("Burst size = %u\n", test_data->burst_sz); printf("File size = %zu\n", test_data->input_data_sz); - printf("%6s%12s%17s%19s%21s%15s%21s%23s%16s\n", - "Level", "Comp size", "Comp ratio [%]", - "Comp [Cycles/it]", "Comp [Cycles/Byte]", "Comp [Gbps]", - "Decomp [Cycles/it]", "Decomp [Cycles/Byte]", "Decomp [Gbps]"); + test_data->cleanup = ST_DURING_TEST; + total_nb_qps = nb_compressdevs * test_data->nb_qps; - cleanup = ST_DURING_TEST; - while (level <= test_data->level.max) { + i = 0; + uint8_t qp_id = 0, cdev_index = 0; - /* - * Run a first iteration, to verify compression and - * get the compression ratio for the level - */ - if (cperf_verification(test_data, level) != EXIT_SUCCESS) - break; + RTE_LCORE_FOREACH_SLAVE(lcore_id) { - /* - * Run benchmarking test - */ - if (cperf_benchmark(test_data, level) != EXIT_SUCCESS) + if (i == total_nb_qps) break; - printf("%6u%12zu%17.2f%19"PRIu64"%21.2f" - "%15.2f%21"PRIu64"%23.2f%16.2f\n", - level, test_data->comp_data_sz, test_data->ratio, - test_data->comp_tsc_duration[level], - test_data->comp_tsc_byte, test_data->comp_gbps, - test_data->decomp_tsc_duration[level], - test_data->decomp_tsc_byte, test_data->decomp_gbps); + cdev_id = enabled_cdevs[cdev_index]; + ctx[i] = cperf_testmap[test_data->test].constructor( + cdev_id, qp_id, + test_data); + if (ctx[i] == NULL) { + RTE_LOG(ERR, USER1, "Test run constructor failed\n"); + goto end; + } + qp_id = (qp_id + 1) % test_data->nb_qps; + if (qp_id == 0) + cdev_index++; + i++; + } + + while (test_data->level <= test_data->level_lst.max) { + + i = 0; + RTE_LCORE_FOREACH_SLAVE(lcore_id) { + + if (i == total_nb_qps) + break; - if (test_data->level.inc != 0) - level += test_data->level.inc; + rte_eal_remote_launch( + cperf_testmap[test_data->test].runner, + ctx[i], lcore_id); + i++; + } + i = 0; + RTE_LCORE_FOREACH_SLAVE(lcore_id) { + + if (i == total_nb_qps) + break; + ret |= rte_eal_wait_lcore(lcore_id); + i++; + } + + if (ret != EXIT_SUCCESS) + break; + + if (test_data->level_lst.inc != 0) + test_data->level += test_data->level_lst.inc; else { - if (++level_idx == test_data->level.count) + if (++level_idx == test_data->level_lst.count) break; - level = test_data->level.list[level_idx]; + test_data->level = test_data->level_lst.list[level_idx]; } } end: - switch (cleanup) { + switch (test_data->cleanup) { case ST_DURING_TEST: - case ST_PREPARE_BUF: - free_bufs(test_data); - /* fallthrough */ - case ST_MEMORY_ALLOC: - rte_free(test_data->decomp_bufs); - rte_free(test_data->comp_bufs); - rte_free(test_data->decompressed_data); - rte_free(test_data->compressed_data); - rte_mempool_free(test_data->op_pool); - rte_mempool_free(test_data->decomp_buf_pool); - rte_mempool_free(test_data->comp_buf_pool); + i = 0; + RTE_LCORE_FOREACH_SLAVE(lcore_id) { + if (i == total_nb_qps) + break; + + if (ctx[i] && cperf_testmap[test_data->test].destructor) + cperf_testmap[test_data->test].destructor( + ctx[i]); + i++; + } /* fallthrough */ case ST_INPUT_DATA: rte_free(test_data->input_data); /* fallthrough */ case ST_COMPDEV: - if (test_data->cdev_id != -1) - rte_compressdev_stop(test_data->cdev_id); + for (i = 0; i < nb_compressdevs && + i < RTE_COMPRESS_MAX_DEVS; i++) + rte_compressdev_stop(enabled_cdevs[i]); /* fallthrough */ case ST_TEST_DATA: rte_free(test_data); @@ -612,3 +442,65 @@ main(int argc, char **argv) } return ret; } + +__rte_weak void * +cperf_benchmark_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused) +{ + RTE_LOG(INFO, USER1, "Benchmark test is not supported yet\n"); + return NULL; +} + +__rte_weak void +cperf_benchmark_test_destructor(void *arg __rte_unused) +{ + +} + +__rte_weak int +cperf_benchmark_test_runner(void *test_ctx __rte_unused) +{ + return 0; +} +__rte_weak void * +cperf_verify_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused) +{ + RTE_LOG(INFO, USER1, "Verify test is not supported yet\n"); + return NULL; +} + +__rte_weak void +cperf_verify_test_destructor(void *arg __rte_unused) +{ + +} + +__rte_weak int +cperf_verify_test_runner(void *test_ctx __rte_unused) +{ + return 0; +} + +__rte_weak void * +cperf_pmd_cyclecount_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused) +{ + RTE_LOG(INFO, USER1, "Pmd_cyclecount test is not supported yet\n"); + return NULL; +} + +__rte_weak void +cperf_pmd_cyclecount_test_destructor(void *arg __rte_unused) +{ + +} + +__rte_weak int +cperf_pmd_cyclecount_test_runner(void *test_ctx __rte_unused) +{ + return 0; +} diff --git a/app/test-compress-perf/meson.build b/app/test-compress-perf/meson.build index ec73e5e..00413c6 100644 --- a/app/test-compress-perf/meson.build +++ b/app/test-compress-perf/meson.build @@ -4,6 +4,5 @@ allow_experimental_apis = true sources = files('comp_perf_options_parse.c', 'main.c', - 'comp_perf_test_verify.c', - 'comp_perf_test_benchmark.c') + 'comp_perf_test_common.c') deps = ['compressdev'] From patchwork Thu May 30 08:06:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Jozwiak X-Patchwork-Id: 53885 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 621171B955; Thu, 30 May 2019 10:06:43 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id F11BC1B94A for ; Thu, 30 May 2019 10:06:39 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 01:06:39 -0700 X-ExtLoop1: 1 Received: from tjozwiax-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.103.104.46]) by fmsmga001.fm.intel.com with ESMTP; 30 May 2019 01:06:38 -0700 From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm.gmail.com@dpdk.org, shallyv@marvell.com Date: Thu, 30 May 2019 10:06:23 +0200 Message-Id: <1559203588-24046-3-git-send-email-tomaszx.jozwiak@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> References: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> Subject: [dpdk-dev] [PATCH v1 2/7] app/test-compress-perf: add ptest command line option X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds --ptest option to make possible a choose of test case from command line. Signed-off-by: Tomasz Jozwiak --- app/test-compress-perf/comp_perf_options_parse.c | 36 ++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/app/test-compress-perf/comp_perf_options_parse.c b/app/test-compress-perf/comp_perf_options_parse.c index bc4b98a..07672b2 100644 --- a/app/test-compress-perf/comp_perf_options_parse.c +++ b/app/test-compress-perf/comp_perf_options_parse.c @@ -15,6 +15,7 @@ #include "comp_perf_options.h" +#define CPERF_PTEST_TYPE ("ptest") #define CPERF_DRIVER_NAME ("driver-name") #define CPERF_TEST_FILE ("input-file") #define CPERF_SEG_SIZE ("seg-sz") @@ -37,6 +38,7 @@ static void usage(char *progname) { printf("%s [EAL options] --\n" + " --ptest benchmark / verify :" " --driver-name NAME: compress driver to use\n" " --input-file NAME: file to compress and decompress\n" " --extended-input-sz N: extend file data up to this size (default: no extension)\n" @@ -76,6 +78,37 @@ get_str_key_id_mapping(struct name_id_map *map, unsigned int map_len, } static int +parse_cperf_test_type(struct comp_test_data *test_data, const char *arg) +{ + struct name_id_map cperftest_namemap[] = { + { + cperf_test_type_strs[CPERF_TEST_TYPE_BENCHMARK], + CPERF_TEST_TYPE_BENCHMARK + }, + { + cperf_test_type_strs[CPERF_TEST_TYPE_VERIFY], + CPERF_TEST_TYPE_VERIFY + }, + { + cperf_test_type_strs[CPERF_TEST_TYPE_PMDCC], + CPERF_TEST_TYPE_PMDCC + } + }; + + int id = get_str_key_id_mapping( + (struct name_id_map *)cperftest_namemap, + RTE_DIM(cperftest_namemap), arg); + if (id < 0) { + RTE_LOG(ERR, USER1, "failed to parse test type"); + return -1; + } + + test_data->test = (enum cperf_perf_test_type)id; + + return 0; +} + +static int parse_uint32_t(uint32_t *value, const char *arg) { char *end = NULL; @@ -499,6 +532,8 @@ struct long_opt_parser { }; static struct option lgopts[] = { + + { CPERF_PTEST_TYPE, required_argument, 0, 0 }, { CPERF_DRIVER_NAME, required_argument, 0, 0 }, { CPERF_TEST_FILE, required_argument, 0, 0 }, { CPERF_SEG_SIZE, required_argument, 0, 0 }, @@ -517,6 +552,7 @@ static int comp_perf_opts_parse_long(int opt_idx, struct comp_test_data *test_data) { struct long_opt_parser parsermap[] = { + { CPERF_PTEST_TYPE, parse_cperf_test_type }, { CPERF_DRIVER_NAME, parse_driver_name }, { CPERF_TEST_FILE, parse_test_file }, { CPERF_SEG_SIZE, parse_seg_sz }, From patchwork Thu May 30 08:06:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Jozwiak X-Patchwork-Id: 53886 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 11AA61B965; Thu, 30 May 2019 10:06:46 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id C80741B950 for ; Thu, 30 May 2019 10:06:41 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 01:06:41 -0700 X-ExtLoop1: 1 Received: from tjozwiax-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.103.104.46]) by fmsmga001.fm.intel.com with ESMTP; 30 May 2019 01:06:39 -0700 From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm.gmail.com@dpdk.org, shallyv@marvell.com Date: Thu, 30 May 2019 10:06:24 +0200 Message-Id: <1559203588-24046-4-git-send-email-tomaszx.jozwiak@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> References: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> Subject: [dpdk-dev] [PATCH v1 3/7] app/test-compress-perf: add verification test case X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds a verification part to compression-perf-tool as a separate test case, which can be executed multi-threaded. Signed-off-by: Tomasz Jozwiak --- app/test-compress-perf/Makefile | 1 + app/test-compress-perf/comp_perf_test_verify.c | 122 ++++++++++++++++++------- app/test-compress-perf/comp_perf_test_verify.h | 24 ++++- app/test-compress-perf/main.c | 1 + app/test-compress-perf/meson.build | 1 + 5 files changed, 112 insertions(+), 37 deletions(-) diff --git a/app/test-compress-perf/Makefile b/app/test-compress-perf/Makefile index de74129..f54d9a4 100644 --- a/app/test-compress-perf/Makefile +++ b/app/test-compress-perf/Makefile @@ -12,6 +12,7 @@ CFLAGS += -O3 # all source are stored in SRCS-y SRCS-y := main.c SRCS-y += comp_perf_options_parse.c +SRCS-y += comp_perf_test_verify.c SRCS-y += comp_perf_test_common.c include $(RTE_SDK)/mk/rte.app.mk diff --git a/app/test-compress-perf/comp_perf_test_verify.c b/app/test-compress-perf/comp_perf_test_verify.c index 28a0fe8..c2aab70 100644 --- a/app/test-compress-perf/comp_perf_test_verify.c +++ b/app/test-compress-perf/comp_perf_test_verify.c @@ -8,14 +8,48 @@ #include #include "comp_perf_test_verify.h" +#include "comp_perf_test_common.h" + +void +cperf_verify_test_destructor(void *arg) +{ + if (arg) { + comp_perf_free_memory(&((struct cperf_verify_ctx *)arg)->mem); + rte_free(arg); + } +} + +void * +cperf_verify_test_constructor(uint8_t dev_id, uint16_t qp_id, + struct comp_test_data *options) +{ + struct cperf_verify_ctx *ctx = NULL; + + ctx = rte_malloc(NULL, sizeof(struct cperf_verify_ctx), 0); + + if (ctx != NULL) { + ctx->mem.dev_id = dev_id; + ctx->mem.qp_id = qp_id; + ctx->options = options; + + if (!comp_perf_allocate_memory(ctx->options, &ctx->mem) && + !prepare_bufs(ctx->options, &ctx->mem)) + return ctx; + } + + cperf_verify_test_destructor(ctx); + return NULL; +} static int -main_loop(struct comp_test_data *test_data, uint8_t level, - enum rte_comp_xform_type type, - uint8_t *output_data_ptr, - size_t *output_data_sz) +main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) { - uint8_t dev_id = test_data->cdev_id; + struct comp_test_data *test_data = ctx->options; + uint8_t *output_data_ptr; + size_t *output_data_sz; + struct cperf_mem_resources *mem = &ctx->mem; + + uint8_t dev_id = mem->dev_id; uint32_t i, iter, num_iter; struct rte_comp_op **ops, **deq_ops; void *priv_xform = NULL; @@ -33,7 +67,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } ops = rte_zmalloc_socket(NULL, - 2 * test_data->total_bufs * sizeof(struct rte_comp_op *), + 2 * mem->total_bufs * sizeof(struct rte_comp_op *), 0, rte_socket_id()); if (ops == NULL) { @@ -42,7 +76,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, return -1; } - deq_ops = &ops[test_data->total_bufs]; + deq_ops = &ops[mem->total_bufs]; if (type == RTE_COMP_COMPRESS) { xform = (struct rte_comp_xform) { @@ -50,14 +84,16 @@ main_loop(struct comp_test_data *test_data, uint8_t level, .compress = { .algo = RTE_COMP_ALGO_DEFLATE, .deflate.huffman = test_data->huffman_enc, - .level = level, + .level = test_data->level, .window_size = test_data->window_sz, .chksum = RTE_COMP_CHECKSUM_NONE, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; - input_bufs = test_data->decomp_bufs; - output_bufs = test_data->comp_bufs; + output_data_ptr = ctx->mem.compressed_data; + output_data_sz = &ctx->comp_data_sz; + input_bufs = mem->decomp_bufs; + output_bufs = mem->comp_bufs; out_seg_sz = test_data->out_seg_sz; } else { xform = (struct rte_comp_xform) { @@ -69,8 +105,10 @@ main_loop(struct comp_test_data *test_data, uint8_t level, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; - input_bufs = test_data->comp_bufs; - output_bufs = test_data->decomp_bufs; + output_data_ptr = ctx->mem.decompressed_data; + output_data_sz = &ctx->decomp_data_sz; + input_bufs = mem->comp_bufs; + output_bufs = mem->decomp_bufs; out_seg_sz = test_data->seg_sz; } @@ -85,8 +123,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, num_iter = 1; for (iter = 0; iter < num_iter; iter++) { - uint32_t total_ops = test_data->total_bufs; - uint32_t remaining_ops = test_data->total_bufs; + uint32_t total_ops = mem->total_bufs; + uint32_t remaining_ops = mem->total_bufs; uint32_t total_deq_ops = 0; uint32_t total_enq_ops = 0; uint16_t ops_unused = 0; @@ -113,7 +151,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, /* Allocate compression operations */ if (ops_needed && !rte_comp_op_bulk_alloc( - test_data->op_pool, + mem->op_pool, &ops[ops_unused], ops_needed)) { RTE_LOG(ERR, USER1, @@ -149,7 +187,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, ops[op_id]->private_xform = priv_xform; } - num_enq = rte_compressdev_enqueue_burst(dev_id, 0, ops, + num_enq = rte_compressdev_enqueue_burst(dev_id, + mem->qp_id, ops, num_ops); if (num_enq == 0) { struct rte_compressdev_stats stats; @@ -165,7 +204,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, remaining_ops -= num_enq; total_enq_ops += num_enq; - num_deq = rte_compressdev_dequeue_burst(dev_id, 0, + num_deq = rte_compressdev_dequeue_burst(dev_id, + mem->qp_id, deq_ops, test_data->burst_sz); total_deq_ops += num_deq; @@ -220,15 +260,17 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } } } - rte_mempool_put_bulk(test_data->op_pool, + rte_mempool_put_bulk(mem->op_pool, (void **)deq_ops, num_deq); allocated -= num_deq; } /* Dequeue the last operations */ while (total_deq_ops < total_ops) { - num_deq = rte_compressdev_dequeue_burst(dev_id, 0, - deq_ops, test_data->burst_sz); + num_deq = rte_compressdev_dequeue_burst(dev_id, + mem->qp_id, + deq_ops, + test_data->burst_sz); if (num_deq == 0) { struct rte_compressdev_stats stats; @@ -291,7 +333,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } } } - rte_mempool_put_bulk(test_data->op_pool, + rte_mempool_put_bulk(mem->op_pool, (void **)deq_ops, num_deq); allocated -= num_deq; } @@ -300,45 +342,45 @@ main_loop(struct comp_test_data *test_data, uint8_t level, if (output_data_sz) *output_data_sz = output_size; end: - rte_mempool_put_bulk(test_data->op_pool, (void **)ops, allocated); + rte_mempool_put_bulk(mem->op_pool, (void **)ops, allocated); rte_compressdev_private_xform_free(dev_id, priv_xform); rte_free(ops); return res; } - - int -cperf_verification(struct comp_test_data *test_data, uint8_t level) +cperf_verify_test_runner(void *test_ctx) { + struct cperf_verify_ctx *ctx = test_ctx; + struct comp_test_data *test_data = ctx->options; int ret = EXIT_SUCCESS; + static rte_atomic16_t display_once = RTE_ATOMIC16_INIT(0); + uint32_t lcore = rte_lcore_id(); + + ctx->mem.lcore_id = lcore; test_data->ratio = 0; - if (main_loop(test_data, level, RTE_COMP_COMPRESS, - test_data->compressed_data, - &test_data->comp_data_sz) < 0) { + if (main_loop(ctx, RTE_COMP_COMPRESS) < 0) { ret = EXIT_FAILURE; goto end; } - if (main_loop(test_data, level, RTE_COMP_DECOMPRESS, - test_data->decompressed_data, - &test_data->decomp_data_sz) < 0) { + if (main_loop(ctx, RTE_COMP_DECOMPRESS) < 0) { ret = EXIT_FAILURE; goto end; } - if (test_data->decomp_data_sz != test_data->input_data_sz) { + if (ctx->decomp_data_sz != test_data->input_data_sz) { RTE_LOG(ERR, USER1, "Decompressed data length not equal to input data length\n"); RTE_LOG(ERR, USER1, "Decompressed size = %zu, expected = %zu\n", - test_data->decomp_data_sz, test_data->input_data_sz); + ctx->decomp_data_sz, test_data->input_data_sz); ret = EXIT_FAILURE; goto end; } else { - if (memcmp(test_data->decompressed_data, + if (memcmp(ctx->mem.decompressed_data, test_data->input_data, test_data->input_data_sz) != 0) { RTE_LOG(ERR, USER1, @@ -348,9 +390,19 @@ cperf_verification(struct comp_test_data *test_data, uint8_t level) } } - test_data->ratio = (double) test_data->comp_data_sz / + ctx->ratio = (double) ctx->comp_data_sz / test_data->input_data_sz * 100; + if (!ctx->silent) { + if (rte_atomic16_test_and_set(&display_once)) { + printf("%12s%6s%12s%17s\n", + "lcore id", "Level", "Comp size", "Comp ratio [%]"); + } + printf("%12u%6u%12zu%17.2f\n", + ctx->mem.lcore_id, + test_data->level, ctx->comp_data_sz, ctx->ratio); + } + end: return ret; } diff --git a/app/test-compress-perf/comp_perf_test_verify.h b/app/test-compress-perf/comp_perf_test_verify.h index 67c6b49..ae8b742 100644 --- a/app/test-compress-perf/comp_perf_test_verify.h +++ b/app/test-compress-perf/comp_perf_test_verify.h @@ -1,13 +1,33 @@ /* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation + * Copyright(c) 2018-2019 Intel Corporation */ #ifndef _COMP_PERF_TEST_VERIFY_ #define _COMP_PERF_TEST_VERIFY_ +#include + #include "comp_perf_options.h" +#include "comp_perf_test_common.h" + +struct cperf_verify_ctx { + struct cperf_mem_resources mem; + struct comp_test_data *options; + + int silent; + size_t comp_data_sz; + size_t decomp_data_sz; + double ratio; +}; + +void +cperf_verify_test_destructor(void *arg); int -cperf_verification(struct comp_test_data *test_data, uint8_t level); +cperf_verify_test_runner(void *test_ctx); + +void * +cperf_verify_test_constructor(uint8_t dev_id, uint16_t qp_id, + struct comp_test_data *options); #endif diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index f393c1b..813b205 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -8,6 +8,7 @@ #include #include "comp_perf_options.h" +#include "comp_perf_test_verify.h" #include "comp_perf.h" #include "comp_perf_test_common.h" diff --git a/app/test-compress-perf/meson.build b/app/test-compress-perf/meson.build index 00413c6..c6246e5 100644 --- a/app/test-compress-perf/meson.build +++ b/app/test-compress-perf/meson.build @@ -4,5 +4,6 @@ allow_experimental_apis = true sources = files('comp_perf_options_parse.c', 'main.c', + 'comp_perf_test_verify.c', 'comp_perf_test_common.c') deps = ['compressdev'] From patchwork Thu May 30 08:06:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Jozwiak X-Patchwork-Id: 53887 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 12F1F1B995; Thu, 30 May 2019 10:06:48 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 9C51B1B958 for ; Thu, 30 May 2019 10:06:43 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 01:06:43 -0700 X-ExtLoop1: 1 Received: from tjozwiax-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.103.104.46]) by fmsmga001.fm.intel.com with ESMTP; 30 May 2019 01:06:41 -0700 From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm.gmail.com@dpdk.org, shallyv@marvell.com Date: Thu, 30 May 2019 10:06:25 +0200 Message-Id: <1559203588-24046-5-git-send-email-tomaszx.jozwiak@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> References: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> Subject: [dpdk-dev] [PATCH v1 4/7] app/test-compress-perf: add benchmark test case X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds a benchmark part to compression-perf-tool as a separate test case, which can be executed multi-threaded. Signed-off-by: Tomasz Jozwiak --- app/test-compress-perf/Makefile | 1 + app/test-compress-perf/comp_perf_test_benchmark.c | 139 ++++++++++++++++------ app/test-compress-perf/comp_perf_test_benchmark.h | 25 +++- app/test-compress-perf/main.c | 1 + app/test-compress-perf/meson.build | 1 + 5 files changed, 129 insertions(+), 38 deletions(-) diff --git a/app/test-compress-perf/Makefile b/app/test-compress-perf/Makefile index f54d9a4..d1a6820 100644 --- a/app/test-compress-perf/Makefile +++ b/app/test-compress-perf/Makefile @@ -13,6 +13,7 @@ CFLAGS += -O3 SRCS-y := main.c SRCS-y += comp_perf_options_parse.c SRCS-y += comp_perf_test_verify.c +SRCS-y += comp_perf_test_benchmark.c SRCS-y += comp_perf_test_common.c include $(RTE_SDK)/mk/rte.app.mk diff --git a/app/test-compress-perf/comp_perf_test_benchmark.c b/app/test-compress-perf/comp_perf_test_benchmark.c index 5752906..9b0b146 100644 --- a/app/test-compress-perf/comp_perf_test_benchmark.c +++ b/app/test-compress-perf/comp_perf_test_benchmark.c @@ -10,11 +10,45 @@ #include "comp_perf_test_benchmark.h" +void +cperf_benchmark_test_destructor(void *arg) +{ + if (arg) { + comp_perf_free_memory( + &((struct cperf_benchmark_ctx *)arg)->ver.mem); + rte_free(arg); + } +} + +void * +cperf_benchmark_test_constructor(uint8_t dev_id, uint16_t qp_id, + struct comp_test_data *options) +{ + struct cperf_benchmark_ctx *ctx = NULL; + + ctx = rte_malloc(NULL, sizeof(struct cperf_benchmark_ctx), 0); + + if (ctx != NULL) { + ctx->ver.mem.dev_id = dev_id; + ctx->ver.mem.qp_id = qp_id; + ctx->ver.options = options; + ctx->ver.silent = 1; /* ver. part will be silent */ + + if (!comp_perf_allocate_memory(ctx->ver.options, &ctx->ver.mem) + && !prepare_bufs(ctx->ver.options, &ctx->ver.mem)) + return ctx; + } + + cperf_benchmark_test_destructor(ctx); + return NULL; +} + static int -main_loop(struct comp_test_data *test_data, uint8_t level, - enum rte_comp_xform_type type) +main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) { - uint8_t dev_id = test_data->cdev_id; + struct comp_test_data *test_data = ctx->ver.options; + struct cperf_mem_resources *mem = &ctx->ver.mem; + uint8_t dev_id = mem->dev_id; uint32_t i, iter, num_iter; struct rte_comp_op **ops, **deq_ops; void *priv_xform = NULL; @@ -31,7 +65,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } ops = rte_zmalloc_socket(NULL, - 2 * test_data->total_bufs * sizeof(struct rte_comp_op *), + 2 * mem->total_bufs * sizeof(struct rte_comp_op *), 0, rte_socket_id()); if (ops == NULL) { @@ -40,7 +74,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, return -1; } - deq_ops = &ops[test_data->total_bufs]; + deq_ops = &ops[mem->total_bufs]; if (type == RTE_COMP_COMPRESS) { xform = (struct rte_comp_xform) { @@ -48,14 +82,14 @@ main_loop(struct comp_test_data *test_data, uint8_t level, .compress = { .algo = RTE_COMP_ALGO_DEFLATE, .deflate.huffman = test_data->huffman_enc, - .level = level, + .level = test_data->level, .window_size = test_data->window_sz, .chksum = RTE_COMP_CHECKSUM_NONE, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; - input_bufs = test_data->decomp_bufs; - output_bufs = test_data->comp_bufs; + input_bufs = mem->decomp_bufs; + output_bufs = mem->comp_bufs; out_seg_sz = test_data->out_seg_sz; } else { xform = (struct rte_comp_xform) { @@ -67,8 +101,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; - input_bufs = test_data->comp_bufs; - output_bufs = test_data->decomp_bufs; + input_bufs = mem->comp_bufs; + output_bufs = mem->decomp_bufs; out_seg_sz = test_data->seg_sz; } @@ -82,13 +116,13 @@ main_loop(struct comp_test_data *test_data, uint8_t level, uint64_t tsc_start, tsc_end, tsc_duration; - tsc_start = tsc_end = tsc_duration = 0; - tsc_start = rte_rdtsc(); num_iter = test_data->num_iter; + tsc_start = tsc_end = tsc_duration = 0; + tsc_start = rte_rdtsc_precise(); for (iter = 0; iter < num_iter; iter++) { - uint32_t total_ops = test_data->total_bufs; - uint32_t remaining_ops = test_data->total_bufs; + uint32_t total_ops = mem->total_bufs; + uint32_t remaining_ops = mem->total_bufs; uint32_t total_deq_ops = 0; uint32_t total_enq_ops = 0; uint16_t ops_unused = 0; @@ -113,7 +147,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, /* Allocate compression operations */ if (ops_needed && !rte_comp_op_bulk_alloc( - test_data->op_pool, + mem->op_pool, &ops[ops_unused], ops_needed)) { RTE_LOG(ERR, USER1, @@ -149,7 +183,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, ops[op_id]->private_xform = priv_xform; } - num_enq = rte_compressdev_enqueue_burst(dev_id, 0, ops, + num_enq = rte_compressdev_enqueue_burst(dev_id, + mem->qp_id, ops, num_ops); if (num_enq == 0) { struct rte_compressdev_stats stats; @@ -165,7 +200,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, remaining_ops -= num_enq; total_enq_ops += num_enq; - num_deq = rte_compressdev_dequeue_burst(dev_id, 0, + num_deq = rte_compressdev_dequeue_burst(dev_id, + mem->qp_id, deq_ops, test_data->burst_sz); total_deq_ops += num_deq; @@ -177,7 +213,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, if (op->status != RTE_COMP_OP_STATUS_SUCCESS) { RTE_LOG(ERR, USER1, - "Some operations were not successful\n"); + "Some operations were not successful\n"); goto end; } @@ -198,15 +234,17 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } } } - rte_mempool_put_bulk(test_data->op_pool, + rte_mempool_put_bulk(mem->op_pool, (void **)deq_ops, num_deq); allocated -= num_deq; } /* Dequeue the last operations */ while (total_deq_ops < total_ops) { - num_deq = rte_compressdev_dequeue_burst(dev_id, 0, - deq_ops, test_data->burst_sz); + num_deq = rte_compressdev_dequeue_burst(dev_id, + mem->qp_id, + deq_ops, + test_data->burst_sz); if (num_deq == 0) { struct rte_compressdev_stats stats; @@ -226,7 +264,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, if (op->status != RTE_COMP_OP_STATUS_SUCCESS) { RTE_LOG(ERR, USER1, - "Some operations were not successful\n"); + "Some operations were not successful\n"); goto end; } @@ -247,65 +285,92 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } } } - rte_mempool_put_bulk(test_data->op_pool, + rte_mempool_put_bulk(mem->op_pool, (void **)deq_ops, num_deq); allocated -= num_deq; } } - tsc_end = rte_rdtsc(); + tsc_end = rte_rdtsc_precise(); tsc_duration = tsc_end - tsc_start; if (type == RTE_COMP_COMPRESS) - test_data->comp_tsc_duration[level] = + ctx->comp_tsc_duration[test_data->level] = tsc_duration / num_iter; else - test_data->decomp_tsc_duration[level] = + ctx->decomp_tsc_duration[test_data->level] = tsc_duration / num_iter; end: - rte_mempool_put_bulk(test_data->op_pool, (void **)ops, allocated); + rte_mempool_put_bulk(mem->op_pool, (void **)ops, allocated); rte_compressdev_private_xform_free(dev_id, priv_xform); rte_free(ops); return res; } int -cperf_benchmark(struct comp_test_data *test_data, uint8_t level) +cperf_benchmark_test_runner(void *test_ctx) { + struct cperf_benchmark_ctx *ctx = test_ctx; + struct comp_test_data *test_data = ctx->ver.options; + uint32_t lcore = rte_lcore_id(); + static rte_atomic16_t display_once = RTE_ATOMIC16_INIT(0); + + ctx->ver.mem.lcore_id = lcore; int i, ret = EXIT_SUCCESS; /* + * First the verification part is needed + */ + if (cperf_verify_test_runner(&ctx->ver)) { + ret = EXIT_FAILURE; + goto end; + } + + /* * Run the tests twice, discarding the first performance * results, before the cache is warmed up */ for (i = 0; i < 2; i++) { - if (main_loop(test_data, level, RTE_COMP_COMPRESS) < 0) { + if (main_loop(ctx, RTE_COMP_COMPRESS) < 0) { ret = EXIT_FAILURE; goto end; } } for (i = 0; i < 2; i++) { - if (main_loop(test_data, level, RTE_COMP_DECOMPRESS) < 0) { + if (main_loop(ctx, RTE_COMP_DECOMPRESS) < 0) { ret = EXIT_FAILURE; goto end; } } - test_data->comp_tsc_byte = - (double)(test_data->comp_tsc_duration[level]) / + ctx->comp_tsc_byte = + (double)(ctx->comp_tsc_duration[test_data->level]) / test_data->input_data_sz; - test_data->decomp_tsc_byte = - (double)(test_data->decomp_tsc_duration[level]) / + ctx->decomp_tsc_byte = + (double)(ctx->decomp_tsc_duration[test_data->level]) / test_data->input_data_sz; - test_data->comp_gbps = rte_get_tsc_hz() / test_data->comp_tsc_byte * 8 / + ctx->comp_gbps = rte_get_tsc_hz() / (ctx->comp_tsc_byte * 8) / + 1000000000; + + ctx->decomp_gbps = rte_get_tsc_hz() / (ctx->decomp_tsc_byte * 8) / 1000000000; - test_data->decomp_gbps = rte_get_tsc_hz() / test_data->decomp_tsc_byte - * 8 / 1000000000; + if (rte_atomic16_test_and_set(&display_once)) { + printf("%12s%6s%12s%17s%15s%16s\n", + "lcore id", "Level", "Comp size", "Comp ratio [%]", + "Comp [Gbps]", "Decomp [Gbps]"); + } + + printf("%12u%6u%12zu%17.2f%15.2f%16.2f\n", + ctx->ver.mem.lcore_id, + test_data->level, ctx->ver.comp_data_sz, ctx->ver.ratio, + ctx->comp_gbps, + ctx->decomp_gbps); + end: return ret; } diff --git a/app/test-compress-perf/comp_perf_test_benchmark.h b/app/test-compress-perf/comp_perf_test_benchmark.h index b193445..d9b2694 100644 --- a/app/test-compress-perf/comp_perf_test_benchmark.h +++ b/app/test-compress-perf/comp_perf_test_benchmark.h @@ -5,9 +5,32 @@ #ifndef _COMP_PERF_TEST_BENCHMARK_ #define _COMP_PERF_TEST_BENCHMARK_ +#include + #include "comp_perf_options.h" +#include "comp_perf_test_common.h" +#include "comp_perf_test_verify.h" + +struct cperf_benchmark_ctx { + struct cperf_verify_ctx ver; + + /* Store TSC duration for all levels (including level 0) */ + uint64_t comp_tsc_duration[RTE_COMP_LEVEL_MAX + 1]; + uint64_t decomp_tsc_duration[RTE_COMP_LEVEL_MAX + 1]; + double comp_gbps; + double decomp_gbps; + double comp_tsc_byte; + double decomp_tsc_byte; +}; + +void +cperf_benchmark_test_destructor(void *arg); int -cperf_benchmark(struct comp_test_data *test_data, uint8_t level); +cperf_benchmark_test_runner(void *test_ctx); + +void * +cperf_benchmark_test_constructor(uint8_t dev_id, uint16_t qp_id, + struct comp_test_data *options); #endif diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index 813b205..c8be84e 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -9,6 +9,7 @@ #include "comp_perf_options.h" #include "comp_perf_test_verify.h" +#include "comp_perf_test_benchmark.h" #include "comp_perf.h" #include "comp_perf_test_common.h" diff --git a/app/test-compress-perf/meson.build b/app/test-compress-perf/meson.build index c6246e5..1136f04 100644 --- a/app/test-compress-perf/meson.build +++ b/app/test-compress-perf/meson.build @@ -5,5 +5,6 @@ allow_experimental_apis = true sources = files('comp_perf_options_parse.c', 'main.c', 'comp_perf_test_verify.c', + 'comp_perf_test_benchmark.c', 'comp_perf_test_common.c') deps = ['compressdev'] From patchwork Thu May 30 08:06:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Jozwiak X-Patchwork-Id: 53888 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D83841B997; Thu, 30 May 2019 10:06:49 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 024BB5680 for ; Thu, 30 May 2019 10:06:44 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 01:06:44 -0700 X-ExtLoop1: 1 Received: from tjozwiax-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.103.104.46]) by fmsmga001.fm.intel.com with ESMTP; 30 May 2019 01:06:43 -0700 From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm.gmail.com@dpdk.org, shallyv@marvell.com Date: Thu, 30 May 2019 10:06:26 +0200 Message-Id: <1559203588-24046-6-git-send-email-tomaszx.jozwiak@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> References: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> Subject: [dpdk-dev] [PATCH v1 5/7] doc: update dpdk-test-compress-perf description X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch updates a dpdk-test-compress-perf documentation. Signed-off-by: Tomasz Jozwiak --- doc/guides/tools/comp_perf.rst | 34 +++++++++++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 3 deletions(-) diff --git a/doc/guides/tools/comp_perf.rst b/doc/guides/tools/comp_perf.rst index 52869c1..71eef18 100644 --- a/doc/guides/tools/comp_perf.rst +++ b/doc/guides/tools/comp_perf.rst @@ -6,7 +6,9 @@ dpdk-test-compress-perf Tool The ``dpdk-test-compress-perf`` tool is a Data Plane Development Kit (DPDK) utility that allows measuring performance parameters of PMDs available in the -compress tree. The tool reads the data from a file (--input-file), +compress tree. User can use multiple cores to run tests on but only +one type of compression PMD can be measured during single application +execution. The tool reads the data from a file (--input-file), dumps all the file into a buffer and fills out the data of input mbufs, which are passed to compress device with compression operations. Then, the output buffers are fed into the decompression stage, and the resulting @@ -26,9 +28,35 @@ Limitations * Stateful operation is not supported in this version. +EAL Options +~~~~~~~~~~~ + +The following are the EAL command-line options that can be used in conjunction +with the ``dpdk-test-compress-perf`` application. +See the DPDK Getting Started Guides for more information on these options. + +* ``-c `` or ``-l `` + + Set the hexadecimal bitmask of the cores to run on. The corelist is a + list cores to use. + +.. Note:: + + One lcore is needed for process admin, tests are run on all other cores. + To run tests on two lcores, three lcores must be passed to the tool. + +* ``-w `` + + Add a PCI device in white list. + +* ``--vdev `` + + Add a virtual device. + +Appication Options +~~~~~~~~~~~~~~~~~~ -Command line options --------------------- + ``--ptest [benchmark/verify]``: set test type (default: benchmark) ``--driver-name NAME``: compress driver to use From patchwork Thu May 30 08:06:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Jozwiak X-Patchwork-Id: 53889 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 216A01B9A3; Thu, 30 May 2019 10:06:52 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id A03151B952 for ; Thu, 30 May 2019 10:06:46 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 01:06:46 -0700 X-ExtLoop1: 1 Received: from tjozwiax-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.103.104.46]) by fmsmga001.fm.intel.com with ESMTP; 30 May 2019 01:06:44 -0700 From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm.gmail.com@dpdk.org, shallyv@marvell.com Date: Thu, 30 May 2019 10:06:27 +0200 Message-Id: <1559203588-24046-7-git-send-email-tomaszx.jozwiak@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> References: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> Subject: [dpdk-dev] [PATCH v1 6/7] app/test-compress-perf: add force process termination X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds a possibility to force controlled process termination as a result of two signals: SIGTERM and SIGINT Signed-off-by: Tomasz Jozwiak --- app/test-compress-perf/comp_perf_options.h | 1 + app/test-compress-perf/comp_perf_test_benchmark.c | 13 ++++++++++++ app/test-compress-perf/comp_perf_test_verify.c | 14 ++++++++++++ app/test-compress-perf/main.c | 26 +++++++++++++++++++++-- 4 files changed, 52 insertions(+), 2 deletions(-) diff --git a/app/test-compress-perf/comp_perf_options.h b/app/test-compress-perf/comp_perf_options.h index 79e63d5..534212d 100644 --- a/app/test-compress-perf/comp_perf_options.h +++ b/app/test-compress-perf/comp_perf_options.h @@ -68,6 +68,7 @@ struct comp_test_data { double ratio; enum cleanup_st cleanup; + int perf_comp_force_stop; }; int diff --git a/app/test-compress-perf/comp_perf_test_benchmark.c b/app/test-compress-perf/comp_perf_test_benchmark.c index 9b0b146..b38b33c 100644 --- a/app/test-compress-perf/comp_perf_test_benchmark.c +++ b/app/test-compress-perf/comp_perf_test_benchmark.c @@ -183,6 +183,9 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) ops[op_id]->private_xform = priv_xform; } + if (unlikely(test_data->perf_comp_force_stop)) + goto end; + num_enq = rte_compressdev_enqueue_burst(dev_id, mem->qp_id, ops, num_ops); @@ -241,6 +244,9 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) /* Dequeue the last operations */ while (total_deq_ops < total_ops) { + if (unlikely(test_data->perf_comp_force_stop)) + goto end; + num_deq = rte_compressdev_dequeue_burst(dev_id, mem->qp_id, deq_ops, @@ -305,6 +311,13 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) rte_mempool_put_bulk(mem->op_pool, (void **)ops, allocated); rte_compressdev_private_xform_free(dev_id, priv_xform); rte_free(ops); + + if (test_data->perf_comp_force_stop) { + RTE_LOG(ERR, USER1, + "lcore: %d Perf. test has been aborted by user\n", + mem->lcore_id); + res = -1; + } return res; } diff --git a/app/test-compress-perf/comp_perf_test_verify.c b/app/test-compress-perf/comp_perf_test_verify.c index c2aab70..b2cd7a0 100644 --- a/app/test-compress-perf/comp_perf_test_verify.c +++ b/app/test-compress-perf/comp_perf_test_verify.c @@ -187,6 +187,9 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) ops[op_id]->private_xform = priv_xform; } + if (unlikely(test_data->perf_comp_force_stop)) + goto end; + num_enq = rte_compressdev_enqueue_burst(dev_id, mem->qp_id, ops, num_ops); @@ -267,6 +270,9 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) /* Dequeue the last operations */ while (total_deq_ops < total_ops) { + if (unlikely(test_data->perf_comp_force_stop)) + goto end; + num_deq = rte_compressdev_dequeue_burst(dev_id, mem->qp_id, deq_ops, @@ -345,6 +351,14 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) rte_mempool_put_bulk(mem->op_pool, (void **)ops, allocated); rte_compressdev_private_xform_free(dev_id, priv_xform); rte_free(ops); + + if (test_data->perf_comp_force_stop) { + RTE_LOG(ERR, USER1, + "lcore: %d Perf. test has been aborted by user\n", + mem->lcore_id); + res = -1; + } + return res; } diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index c8be84e..98acd02 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -2,6 +2,10 @@ * Copyright(c) 2018 Intel Corporation */ +#include +#include +#include + #include #include #include @@ -42,6 +46,8 @@ static const struct cperf_test cperf_testmap[] = { } }; +static struct comp_test_data *test_data; + static int comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id) { @@ -280,12 +286,24 @@ comp_perf_dump_input_data(struct comp_test_data *test_data) return ret; } +static void +comp_perf_cleanup_on_signal(int signalNumber __rte_unused) +{ + test_data->perf_comp_force_stop = 1; +} + +static void +comp_perf_register_cleanup_on_signal(void) +{ + signal(SIGTERM, comp_perf_cleanup_on_signal); + signal(SIGINT, comp_perf_cleanup_on_signal); +} + int main(int argc, char **argv) { uint8_t level_idx = 0; int ret, i; - struct comp_test_data *test_data; void *ctx[RTE_MAX_LCORE] = {}; uint8_t enabled_cdevs[RTE_COMPRESS_MAX_DEVS]; int nb_compressdevs = 0; @@ -307,6 +325,8 @@ main(int argc, char **argv) rte_exit(EXIT_FAILURE, "Cannot reserve memory in socket %d\n", rte_socket_id()); + comp_perf_register_cleanup_on_signal(); + ret = EXIT_SUCCESS; test_data->cleanup = ST_TEST_DATA; comp_perf_options_default(test_data); @@ -426,8 +446,10 @@ main(int argc, char **argv) /* fallthrough */ case ST_COMPDEV: for (i = 0; i < nb_compressdevs && - i < RTE_COMPRESS_MAX_DEVS; i++) + i < RTE_COMPRESS_MAX_DEVS; i++) { rte_compressdev_stop(enabled_cdevs[i]); + rte_compressdev_close(enabled_cdevs[i]); + } /* fallthrough */ case ST_TEST_DATA: rte_free(test_data); From patchwork Thu May 30 08:06:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Jozwiak X-Patchwork-Id: 53890 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D81EA1B9A9; Thu, 30 May 2019 10:06:53 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 082481B993 for ; Thu, 30 May 2019 10:06:47 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 01:06:47 -0700 X-ExtLoop1: 1 Received: from tjozwiax-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.103.104.46]) by fmsmga001.fm.intel.com with ESMTP; 30 May 2019 01:06:46 -0700 From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm.gmail.com@dpdk.org, shallyv@marvell.com Date: Thu, 30 May 2019 10:06:28 +0200 Message-Id: <1559203588-24046-8-git-send-email-tomaszx.jozwiak@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> References: <1559203588-24046-1-git-send-email-tomaszx.jozwiak@intel.com> Subject: [dpdk-dev] [PATCH v1 7/7] doc: update release notes for 19.08 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Added release note entry for test-compress-perf application Signed-off-by: Tomasz Jozwiak --- doc/guides/rel_notes/release_19_08.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/doc/guides/rel_notes/release_19_08.rst b/doc/guides/rel_notes/release_19_08.rst index b9510f9..543e7d3 100644 --- a/doc/guides/rel_notes/release_19_08.rst +++ b/doc/guides/rel_notes/release_19_08.rst @@ -54,6 +54,9 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **Updated test-compress-perf tool application.** + + Added multiple cores feature to compression perf tool application. Removed Items -------------