From patchwork Wed Jun 26 16:30:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Tomasz_J=C3=B3=C5=BAwiak?= X-Patchwork-Id: 55419 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E59022B99; Wed, 26 Jun 2019 18:30:51 +0200 (CEST) Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by dpdk.org (Postfix) with ESMTP id 5A6192B98 for ; Wed, 26 Jun 2019 18:30:46 +0200 (CEST) Received: by mail-wm1-f44.google.com with SMTP id c66so2782586wmf.0 for ; Wed, 26 Jun 2019 09:30:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=dTrohSI8eGOuJ4kf/oY1jGX5A4zyEsAE9NQV0qvVIpk=; b=trhJfSgbIpDhYOU8QQTZeZGOk/twYxUsjOAz6fJ6SoKlIBXNjfhMj9WYA6oVrZx/di hvmJF9beMdWxam5COPGu1+bIPz9yhFdAmxkoSjqV5m8RIclkTjvlodENPkyuum0IeC+k quuz4q37slhZifsA5yUgdOWqGW3no7rdkSnSPE+vHrXCXtAbvj6bPwvrARYNXGDklsCG 7G+s6UomfXjE0IcCTufSew0X+v7ytFl/Yu+hy/iQuPK4yx2i231DOgLkBg9Sy6HBWiH+ lvfHh0iFPEKBrcfMZLTNAenuEB5UJc5EBNGnnJ6ZGRv6m+1PsNDInl7Vguq7/vGy4VFS 8zXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=dTrohSI8eGOuJ4kf/oY1jGX5A4zyEsAE9NQV0qvVIpk=; b=dEeKMPhQHQ8text7wvS7buYZslEynViVYGHA/eviRji1oePgBpfz1bwtkFk8FyF6ma K31cLldHBoKkqDSxP1GAULvl50sp3kyZOVoS39DtbRscch0Pew14yWRUkEx/scpJgQNJ BjKPver25TO4Kxx8AwK+3AZ8MbNh89jSOwFv2cieUl6iaNGJ1fMI1JAI0GoQJPHzpg2Y A3kfwxBV9ZzM8oJ77g78in6/0AFu78CLfSqtBfLzoah5NttkDEKZl+J+odOdACbNC6g2 /0RMg3dKDOBo9GJQiMyWOxOeTI6Lnu2ra6xlN7whdgaMIbC1g0cXZDsb+Fo40vo2UrmC +z8Q== X-Gm-Message-State: APjAAAVO1phwk7SIDDSTBZOihkyvVZOzWkw1/lxspxO75sF3U2ZHkAcs knufkQ4ZwclLn8QwCOhBePSfgjmUexRsaw== X-Google-Smtp-Source: APXvYqxtyz/869D+lt5U8wxF6vwSO1mCJf96n6O6ApslFsPI90ly2UInBTx/38FGce8JGwBPrneaqw== X-Received: by 2002:a7b:c933:: with SMTP id h19mr3470285wml.52.1561566645148; Wed, 26 Jun 2019 09:30:45 -0700 (PDT) Received: from localhost.localdomain (178.43.130.103.ipv4.supernova.orange.pl. [178.43.130.103]) by smtp.gmail.com with ESMTPSA id n10sm16989663wrw.83.2019.06.26.09.30.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 26 Jun 2019 09:30:44 -0700 (PDT) From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm@gmail.com, shallyv@marvell.com, arturx.trybula@intel.com Date: Wed, 26 Jun 2019 18:30:27 +0200 Message-Id: <1561566633-12329-2-git-send-email-tjozwiakgm@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> References: <1560032553-15528-1-git-send-email-tjozwiakgm@gmail.com> <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> Subject: [dpdk-dev] [PATCH v3 1/7] app/test-compress-perf: add weak functions for multi-cores test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tomasz Jozwiak This patch adds a template functions for multi-cores performance version of compress-perf-tool. Signed-off-by: Tomasz Jozwiak Signed-off-by: Tomasz Jozwiak --- app/test-compress-perf/Makefile | 3 +- app/test-compress-perf/comp_perf.h | 61 +++ app/test-compress-perf/comp_perf_options.h | 45 +- app/test-compress-perf/comp_perf_options_parse.c | 24 +- app/test-compress-perf/comp_perf_test_common.c | 285 +++++++++++ app/test-compress-perf/comp_perf_test_common.h | 41 ++ app/test-compress-perf/main.c | 626 ++++++++++------------- app/test-compress-perf/meson.build | 3 +- 8 files changed, 686 insertions(+), 402 deletions(-) create mode 100644 app/test-compress-perf/comp_perf.h create mode 100644 app/test-compress-perf/comp_perf_test_common.c create mode 100644 app/test-compress-perf/comp_perf_test_common.h diff --git a/app/test-compress-perf/Makefile b/app/test-compress-perf/Makefile index d20e17e..de74129 100644 --- a/app/test-compress-perf/Makefile +++ b/app/test-compress-perf/Makefile @@ -12,7 +12,6 @@ CFLAGS += -O3 # all source are stored in SRCS-y SRCS-y := main.c SRCS-y += comp_perf_options_parse.c -SRCS-y += comp_perf_test_verify.c -SRCS-y += comp_perf_test_benchmark.c +SRCS-y += comp_perf_test_common.c include $(RTE_SDK)/mk/rte.app.mk diff --git a/app/test-compress-perf/comp_perf.h b/app/test-compress-perf/comp_perf.h new file mode 100644 index 0000000..144ad8a --- /dev/null +++ b/app/test-compress-perf/comp_perf.h @@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#ifndef _COMP_PERF_ +#define _COMP_PERF_ + +#include + +struct comp_test_data; + +typedef void *(*cperf_constructor_t)( + uint8_t dev_id, + uint16_t qp_id, + struct comp_test_data *options); + +typedef int (*cperf_runner_t)(void *test_ctx); +typedef void (*cperf_destructor_t)(void *test_ctx); + +struct cperf_test { + cperf_constructor_t constructor; + cperf_runner_t runner; + cperf_destructor_t destructor; +}; + +/* Needed for weak functions*/ + +void * +cperf_benchmark_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused); + +void +cperf_benchmark_test_destructor(void *arg __rte_unused); + +int +cperf_benchmark_test_runner(void *test_ctx __rte_unused); + +void * +cperf_verify_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused); + +void +cperf_verify_test_destructor(void *arg __rte_unused); + +int +cperf_verify_test_runner(void *test_ctx __rte_unused); + +void * +cperf_pmd_cyclecount_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused); + +void +cperf_pmd_cyclecount_test_destructor(void *arg __rte_unused); + +int +cperf_pmd_cyclecount_test_runner(void *test_ctx __rte_unused); + +#endif /* _COMP_PERF_ */ diff --git a/app/test-compress-perf/comp_perf_options.h b/app/test-compress-perf/comp_perf_options.h index f87751d..79e63d5 100644 --- a/app/test-compress-perf/comp_perf_options.h +++ b/app/test-compress-perf/comp_perf_options.h @@ -13,6 +13,24 @@ #define MAX_MBUF_DATA_SIZE (UINT16_MAX - RTE_PKTMBUF_HEADROOM) #define MAX_SEG_SIZE ((int)(MAX_MBUF_DATA_SIZE / EXPANSE_RATIO)) +extern const char *cperf_test_type_strs[]; + +/* Cleanup state machine */ +enum cleanup_st { + ST_CLEAR = 0, + ST_TEST_DATA, + ST_COMPDEV, + ST_INPUT_DATA, + ST_MEMORY_ALLOC, + ST_DURING_TEST +}; + +enum cperf_perf_test_type { + CPERF_TEST_TYPE_BENCHMARK, + CPERF_TEST_TYPE_VERIFY, + CPERF_TEST_TYPE_PMDCC +}; + enum comp_operation { COMPRESS_ONLY, DECOMPRESS_ONLY, @@ -30,37 +48,26 @@ struct range_list { struct comp_test_data { char driver_name[64]; char input_file[64]; - struct rte_mbuf **comp_bufs; - struct rte_mbuf **decomp_bufs; - uint32_t total_bufs; + enum cperf_perf_test_type test; + uint8_t *input_data; size_t input_data_sz; - uint8_t *compressed_data; - uint8_t *decompressed_data; - struct rte_mempool *comp_buf_pool; - struct rte_mempool *decomp_buf_pool; - struct rte_mempool *op_pool; - int8_t cdev_id; + uint16_t nb_qps; uint16_t seg_sz; uint16_t out_seg_sz; uint16_t burst_sz; uint32_t pool_sz; uint32_t num_iter; uint16_t max_sgl_segs; + enum rte_comp_huffman huffman_enc; enum comp_operation test_op; int window_sz; - struct range_list level; - /* Store TSC duration for all levels (including level 0) */ - uint64_t comp_tsc_duration[RTE_COMP_LEVEL_MAX + 1]; - uint64_t decomp_tsc_duration[RTE_COMP_LEVEL_MAX + 1]; - size_t comp_data_sz; - size_t decomp_data_sz; + struct range_list level_lst; + uint8_t level; + double ratio; - double comp_gbps; - double decomp_gbps; - double comp_tsc_byte; - double decomp_tsc_byte; + enum cleanup_st cleanup; }; int diff --git a/app/test-compress-perf/comp_perf_options_parse.c b/app/test-compress-perf/comp_perf_options_parse.c index 2fb6fb4..bc4b98a 100644 --- a/app/test-compress-perf/comp_perf_options_parse.c +++ b/app/test-compress-perf/comp_perf_options_parse.c @@ -466,19 +466,20 @@ parse_level(struct comp_test_data *test_data, const char *arg) * Try parsing the argument as a range, if it fails, * arse it as a list */ - if (parse_range(arg, &test_data->level.min, &test_data->level.max, - &test_data->level.inc) < 0) { - ret = parse_list(arg, test_data->level.list, - &test_data->level.min, - &test_data->level.max); + if (parse_range(arg, &test_data->level_lst.min, + &test_data->level_lst.max, + &test_data->level_lst.inc) < 0) { + ret = parse_list(arg, test_data->level_lst.list, + &test_data->level_lst.min, + &test_data->level_lst.max); if (ret < 0) { RTE_LOG(ERR, USER1, "Failed to parse compression level/s\n"); return -1; } - test_data->level.count = ret; + test_data->level_lst.count = ret; - if (test_data->level.max > RTE_COMP_LEVEL_MAX) { + if (test_data->level_lst.max > RTE_COMP_LEVEL_MAX) { RTE_LOG(ERR, USER1, "Level cannot be higher than %u\n", RTE_COMP_LEVEL_MAX); return -1; @@ -498,7 +499,6 @@ struct long_opt_parser { }; static struct option lgopts[] = { - { CPERF_DRIVER_NAME, required_argument, 0, 0 }, { CPERF_TEST_FILE, required_argument, 0, 0 }, { CPERF_SEG_SIZE, required_argument, 0, 0 }, @@ -572,7 +572,6 @@ comp_perf_options_parse(struct comp_test_data *test_data, int argc, char **argv) void comp_perf_options_default(struct comp_test_data *test_data) { - test_data->cdev_id = -1; test_data->seg_sz = 2048; test_data->burst_sz = 32; test_data->pool_sz = 8192; @@ -581,9 +580,10 @@ comp_perf_options_default(struct comp_test_data *test_data) test_data->huffman_enc = RTE_COMP_HUFFMAN_DYNAMIC; test_data->test_op = COMPRESS_DECOMPRESS; test_data->window_sz = -1; - test_data->level.min = 1; - test_data->level.max = 9; - test_data->level.inc = 1; + test_data->level_lst.min = 1; + test_data->level_lst.max = 9; + test_data->level_lst.inc = 1; + test_data->test = CPERF_TEST_TYPE_BENCHMARK; } int diff --git a/app/test-compress-perf/comp_perf_test_common.c b/app/test-compress-perf/comp_perf_test_common.c new file mode 100644 index 0000000..1d2b25e --- /dev/null +++ b/app/test-compress-perf/comp_perf_test_common.c @@ -0,0 +1,285 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#include +#include +#include +#include + +#include "comp_perf_options.h" +#include "comp_perf_test_verify.h" +#include "comp_perf_test_benchmark.h" +#include "comp_perf.h" +#include "comp_perf_test_common.h" + +#define DIV_CEIL(a, b) ((a) / (b) + ((a) % (b) != 0)) + +int +param_range_check(uint16_t size, const struct rte_param_log2_range *range) +{ + unsigned int next_size; + + /* Check lower/upper bounds */ + if (size < range->min) + return -1; + + if (size > range->max) + return -1; + + /* If range is actually only one value, size is correct */ + if (range->increment == 0) + return 0; + + /* Check if value is one of the supported sizes */ + for (next_size = range->min; next_size <= range->max; + next_size += range->increment) + if (size == next_size) + return 0; + + return -1; +} + +static uint32_t +find_buf_size(uint32_t input_size) +{ + uint32_t i; + + /* From performance point of view the buffer size should be a + * power of 2 but also should be enough to store incompressible data + */ + + /* We're looking for nearest power of 2 buffer size, which is greather + * than input_size + */ + uint32_t size = + !input_size ? MIN_COMPRESSED_BUF_SIZE : (input_size << 1); + + for (i = UINT16_MAX + 1; !(i & size); i >>= 1) + ; + + return i > ((UINT16_MAX + 1) >> 1) + ? (uint32_t)((float)input_size * EXPANSE_RATIO) + : i; +} + +void +comp_perf_free_memory(struct cperf_mem_resources *mem) +{ + uint32_t i; + + for (i = 0; i < mem->total_bufs; i++) { + rte_pktmbuf_free(mem->comp_bufs[i]); + rte_pktmbuf_free(mem->decomp_bufs[i]); + } + + rte_free(mem->decomp_bufs); + rte_free(mem->comp_bufs); + rte_free(mem->decompressed_data); + rte_free(mem->compressed_data); + rte_mempool_free(mem->op_pool); + rte_mempool_free(mem->decomp_buf_pool); + rte_mempool_free(mem->comp_buf_pool); +} + +int +comp_perf_allocate_memory(struct comp_test_data *test_data, + struct cperf_mem_resources *mem) +{ + test_data->out_seg_sz = find_buf_size(test_data->seg_sz); + /* Number of segments for input and output + * (compression and decompression) + */ + uint32_t total_segs = DIV_CEIL(test_data->input_data_sz, + test_data->seg_sz); + char pool_name[32] = ""; + + snprintf(pool_name, sizeof(pool_name), "comp_buf_pool_%u_qp_%u", + mem->dev_id, mem->qp_id); + mem->comp_buf_pool = rte_pktmbuf_pool_create(pool_name, + total_segs, + 0, 0, + test_data->out_seg_sz + RTE_PKTMBUF_HEADROOM, + rte_socket_id()); + if (mem->comp_buf_pool == NULL) { + RTE_LOG(ERR, USER1, "Mbuf mempool could not be created\n"); + return -1; + } + + snprintf(pool_name, sizeof(pool_name), "decomp_buf_pool_%u_qp_%u", + mem->dev_id, mem->qp_id); + mem->decomp_buf_pool = rte_pktmbuf_pool_create(pool_name, + total_segs, + 0, 0, test_data->seg_sz + RTE_PKTMBUF_HEADROOM, + rte_socket_id()); + if (mem->decomp_buf_pool == NULL) { + RTE_LOG(ERR, USER1, "Mbuf mempool could not be created\n"); + return -1; + } + + mem->total_bufs = DIV_CEIL(total_segs, test_data->max_sgl_segs); + + snprintf(pool_name, sizeof(pool_name), "op_pool_%u_qp_%u", + mem->dev_id, mem->qp_id); + mem->op_pool = rte_comp_op_pool_create(pool_name, + mem->total_bufs, + 0, 0, rte_socket_id()); + if (mem->op_pool == NULL) { + RTE_LOG(ERR, USER1, "Comp op mempool could not be created\n"); + return -1; + } + + /* + * Compressed data might be a bit larger than input data, + * if data cannot be compressed + */ + mem->compressed_data = rte_zmalloc_socket(NULL, + test_data->input_data_sz * EXPANSE_RATIO + + MIN_COMPRESSED_BUF_SIZE, 0, + rte_socket_id()); + if (mem->compressed_data == NULL) { + RTE_LOG(ERR, USER1, "Memory to hold the data from the input " + "file could not be allocated\n"); + return -1; + } + + mem->decompressed_data = rte_zmalloc_socket(NULL, + test_data->input_data_sz, 0, + rte_socket_id()); + if (mem->decompressed_data == NULL) { + RTE_LOG(ERR, USER1, "Memory to hold the data from the input " + "file could not be allocated\n"); + return -1; + } + + mem->comp_bufs = rte_zmalloc_socket(NULL, + mem->total_bufs * sizeof(struct rte_mbuf *), + 0, rte_socket_id()); + if (mem->comp_bufs == NULL) { + RTE_LOG(ERR, USER1, "Memory to hold the compression mbufs" + " could not be allocated\n"); + return -1; + } + + mem->decomp_bufs = rte_zmalloc_socket(NULL, + mem->total_bufs * sizeof(struct rte_mbuf *), + 0, rte_socket_id()); + if (mem->decomp_bufs == NULL) { + RTE_LOG(ERR, USER1, "Memory to hold the decompression mbufs" + " could not be allocated\n"); + return -1; + } + return 0; +} + +int +prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem) +{ + uint32_t remaining_data = test_data->input_data_sz; + uint8_t *input_data_ptr = test_data->input_data; + size_t data_sz; + uint8_t *data_addr; + uint32_t i, j; + + for (i = 0; i < mem->total_bufs; i++) { + /* Allocate data in input mbuf and copy data from input file */ + mem->decomp_bufs[i] = + rte_pktmbuf_alloc(mem->decomp_buf_pool); + if (mem->decomp_bufs[i] == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"); + return -1; + } + + data_sz = RTE_MIN(remaining_data, test_data->seg_sz); + data_addr = (uint8_t *) rte_pktmbuf_append( + mem->decomp_bufs[i], data_sz); + if (data_addr == NULL) { + RTE_LOG(ERR, USER1, "Could not append data\n"); + return -1; + } + rte_memcpy(data_addr, input_data_ptr, data_sz); + + input_data_ptr += data_sz; + remaining_data -= data_sz; + + /* Already one segment in the mbuf */ + uint16_t segs_per_mbuf = 1; + + /* Chain mbufs if needed for input mbufs */ + while (segs_per_mbuf < test_data->max_sgl_segs + && remaining_data > 0) { + struct rte_mbuf *next_seg = + rte_pktmbuf_alloc(mem->decomp_buf_pool); + + if (next_seg == NULL) { + RTE_LOG(ERR, USER1, + "Could not allocate mbuf\n"); + return -1; + } + + data_sz = RTE_MIN(remaining_data, test_data->seg_sz); + data_addr = (uint8_t *)rte_pktmbuf_append(next_seg, + data_sz); + + if (data_addr == NULL) { + RTE_LOG(ERR, USER1, "Could not append data\n"); + return -1; + } + + rte_memcpy(data_addr, input_data_ptr, data_sz); + input_data_ptr += data_sz; + remaining_data -= data_sz; + + if (rte_pktmbuf_chain(mem->decomp_bufs[i], + next_seg) < 0) { + RTE_LOG(ERR, USER1, "Could not chain mbufs\n"); + return -1; + } + segs_per_mbuf++; + } + + /* Allocate data in output mbuf */ + mem->comp_bufs[i] = + rte_pktmbuf_alloc(mem->comp_buf_pool); + if (mem->comp_bufs[i] == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"); + return -1; + } + data_addr = (uint8_t *) rte_pktmbuf_append( + mem->comp_bufs[i], + test_data->out_seg_sz); + if (data_addr == NULL) { + RTE_LOG(ERR, USER1, "Could not append data\n"); + return -1; + } + + /* Chain mbufs if needed for output mbufs */ + for (j = 1; j < segs_per_mbuf; j++) { + struct rte_mbuf *next_seg = + rte_pktmbuf_alloc(mem->comp_buf_pool); + + if (next_seg == NULL) { + RTE_LOG(ERR, USER1, + "Could not allocate mbuf\n"); + return -1; + } + + data_addr = (uint8_t *)rte_pktmbuf_append(next_seg, + test_data->out_seg_sz); + + if (data_addr == NULL) { + RTE_LOG(ERR, USER1, "Could not append data\n"); + return -1; + } + + if (rte_pktmbuf_chain(mem->comp_bufs[i], + next_seg) < 0) { + RTE_LOG(ERR, USER1, "Could not chain mbufs\n"); + return -1; + } + } + } + + return 0; +} + diff --git a/app/test-compress-perf/comp_perf_test_common.h b/app/test-compress-perf/comp_perf_test_common.h new file mode 100644 index 0000000..9c11e3a --- /dev/null +++ b/app/test-compress-perf/comp_perf_test_common.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#ifndef _COMP_PERF_TEST_COMMON_H_ +#define _COMP_PERF_TEST_COMMON_H_ + +#include + +#include + +struct cperf_mem_resources { + uint8_t dev_id; + uint16_t qp_id; + uint8_t lcore_id; + uint32_t total_bufs; + uint8_t *compressed_data; + uint8_t *decompressed_data; + + struct rte_mbuf **comp_bufs; + struct rte_mbuf **decomp_bufs; + + struct rte_mempool *comp_buf_pool; + struct rte_mempool *decomp_buf_pool; + struct rte_mempool *op_pool; +}; + +int +param_range_check(uint16_t size, const struct rte_param_log2_range *range); + +void +comp_perf_free_memory(struct cperf_mem_resources *mem); + +int +comp_perf_allocate_memory(struct comp_test_data *test_data, + struct cperf_mem_resources *mem); + +int +prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem); + +#endif /* _COMP_PERF_TEST_COMMON_H_ */ diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index 7b9ee74..d142b91 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -8,56 +8,44 @@ #include #include "comp_perf_options.h" -#include "comp_perf_test_verify.h" -#include "comp_perf_test_benchmark.h" +#include "comp_perf.h" +#include "comp_perf_test_common.h" #define NUM_MAX_XFORMS 16 #define NUM_MAX_INFLIGHT_OPS 512 -#define DIV_CEIL(a, b) ((a) / (b) + ((a) % (b) != 0)) - -/* Cleanup state machine */ -static enum cleanup_st { - ST_CLEAR = 0, - ST_TEST_DATA, - ST_COMPDEV, - ST_INPUT_DATA, - ST_MEMORY_ALLOC, - ST_PREPARE_BUF, - ST_DURING_TEST -} cleanup = ST_CLEAR; - -static int -param_range_check(uint16_t size, const struct rte_param_log2_range *range) -{ - unsigned int next_size; - - /* Check lower/upper bounds */ - if (size < range->min) - return -1; - - if (size > range->max) - return -1; - - /* If range is actually only one value, size is correct */ - if (range->increment == 0) - return 0; - - /* Check if value is one of the supported sizes */ - for (next_size = range->min; next_size <= range->max; - next_size += range->increment) - if (size == next_size) - return 0; - - return -1; -} +__extension__ +const char *cperf_test_type_strs[] = { + [CPERF_TEST_TYPE_BENCHMARK] = "benchmark", + [CPERF_TEST_TYPE_VERIFY] = "verify", + [CPERF_TEST_TYPE_PMDCC] = "pmd-cyclecount" +}; + +__extension__ +static const struct cperf_test cperf_testmap[] = { + [CPERF_TEST_TYPE_BENCHMARK] = { + cperf_benchmark_test_constructor, + cperf_benchmark_test_runner, + cperf_benchmark_test_destructor + }, + [CPERF_TEST_TYPE_VERIFY] = { + cperf_verify_test_constructor, + cperf_verify_test_runner, + cperf_verify_test_destructor + }, + [CPERF_TEST_TYPE_PMDCC] = { + cperf_pmd_cyclecount_test_constructor, + cperf_pmd_cyclecount_test_runner, + cperf_pmd_cyclecount_test_destructor + } +}; static int -comp_perf_check_capabilities(struct comp_test_data *test_data) +comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id) { const struct rte_compressdev_capabilities *cap; - cap = rte_compressdev_capability_get(test_data->cdev_id, + cap = rte_compressdev_capability_get(cdev_id, RTE_COMP_ALGO_DEFLATE); if (cap == NULL) { @@ -105,7 +93,7 @@ comp_perf_check_capabilities(struct comp_test_data *test_data) } /* Level 0 support */ - if (test_data->level.min == 0 && + if (test_data->level_lst.min == 0 && (comp_flags & RTE_COMP_FF_NONCOMPRESSED_BLOCKS) == 0) { RTE_LOG(ERR, USER1, "Compress device does not support " "level 0 (no compression)\n"); @@ -115,110 +103,108 @@ comp_perf_check_capabilities(struct comp_test_data *test_data) return 0; } -static uint32_t -find_buf_size(uint32_t input_size) +static int +comp_perf_initialize_compressdev(struct comp_test_data *test_data, + uint8_t *enabled_cdevs) { - uint32_t i; + uint8_t enabled_cdev_count, nb_lcores, cdev_id; + unsigned int i, j; + int ret; - /* From performance point of view the buffer size should be a - * power of 2 but also should be enough to store incompressible data - */ + enabled_cdev_count = rte_compressdev_devices_get(test_data->driver_name, + enabled_cdevs, RTE_COMPRESS_MAX_DEVS); + if (enabled_cdev_count == 0) { + RTE_LOG(ERR, USER1, "No compress devices type %s available\n", + test_data->driver_name); + return -EINVAL; + } - /* We're looking for nearest power of 2 buffer size, which is greather - * than input_size + nb_lcores = rte_lcore_count() - 1; + /* + * Use fewer devices, + * if there are more available than cores. */ - uint32_t size = - !input_size ? MIN_COMPRESSED_BUF_SIZE : (input_size << 1); - - for (i = UINT16_MAX + 1; !(i & size); i >>= 1) - ; - - return i > ((UINT16_MAX + 1) >> 1) - ? (uint32_t)((float)input_size * EXPANSE_RATIO) - : i; -} - -static int -comp_perf_allocate_memory(struct comp_test_data *test_data) -{ + if (enabled_cdev_count > nb_lcores) { + enabled_cdev_count = nb_lcores; + RTE_LOG(INFO, USER1, + " There's more available devices than cores!" + " The number of devices has been aligned to %d cores\n", + nb_lcores); + } - test_data->out_seg_sz = find_buf_size(test_data->seg_sz); - /* Number of segments for input and output - * (compression and decompression) + /* + * Calculate number of needed queue pairs, based on the amount + * of available number of logical cores and compression devices. + * For instance, if there are 4 cores and 2 compression devices, + * 2 queue pairs will be set up per device. + * One queue pair per one core. + * if e.g.: there're 3 cores and 2 compression devices, + * 2 queue pairs will be set up per device but one queue pair + * will left unused in the last one device */ - uint32_t total_segs = DIV_CEIL(test_data->input_data_sz, - test_data->seg_sz); - test_data->comp_buf_pool = rte_pktmbuf_pool_create("comp_buf_pool", - total_segs, - 0, 0, - test_data->out_seg_sz + RTE_PKTMBUF_HEADROOM, - rte_socket_id()); - if (test_data->comp_buf_pool == NULL) { - RTE_LOG(ERR, USER1, "Mbuf mempool could not be created\n"); - return -1; - } + test_data->nb_qps = (nb_lcores % enabled_cdev_count) ? + (nb_lcores / enabled_cdev_count) + 1 : + nb_lcores / enabled_cdev_count; - cleanup = ST_MEMORY_ALLOC; - test_data->decomp_buf_pool = rte_pktmbuf_pool_create("decomp_buf_pool", - total_segs, - 0, 0, test_data->seg_sz + RTE_PKTMBUF_HEADROOM, - rte_socket_id()); - if (test_data->decomp_buf_pool == NULL) { - RTE_LOG(ERR, USER1, "Mbuf mempool could not be created\n"); - return -1; - } + for (i = 0; i < enabled_cdev_count && + i < RTE_COMPRESS_MAX_DEVS; i++, + nb_lcores -= test_data->nb_qps) { + cdev_id = enabled_cdevs[i]; - test_data->total_bufs = DIV_CEIL(total_segs, test_data->max_sgl_segs); + struct rte_compressdev_info cdev_info; + uint8_t socket_id = rte_compressdev_socket_id(cdev_id); - test_data->op_pool = rte_comp_op_pool_create("op_pool", - test_data->total_bufs, - 0, 0, rte_socket_id()); - if (test_data->op_pool == NULL) { - RTE_LOG(ERR, USER1, "Comp op mempool could not be created\n"); - return -1; - } + rte_compressdev_info_get(cdev_id, &cdev_info); + if (cdev_info.max_nb_queue_pairs && + test_data->nb_qps > cdev_info.max_nb_queue_pairs) { + RTE_LOG(ERR, USER1, + "Number of needed queue pairs is higher " + "than the maximum number of queue pairs " + "per device.\n"); + RTE_LOG(ERR, USER1, + "Lower the number of cores or increase " + "the number of crypto devices\n"); + return -EINVAL; + } - /* - * Compressed data might be a bit larger than input data, - * if data cannot be compressed - */ - test_data->compressed_data = rte_zmalloc_socket(NULL, - test_data->input_data_sz * EXPANSE_RATIO - + MIN_COMPRESSED_BUF_SIZE, 0, - rte_socket_id()); - if (test_data->compressed_data == NULL) { - RTE_LOG(ERR, USER1, "Memory to hold the data from the input " - "file could not be allocated\n"); - return -1; - } + if (comp_perf_check_capabilities(test_data, cdev_id) < 0) + return -EINVAL; + + /* Configure compressdev */ + struct rte_compressdev_config config = { + .socket_id = socket_id, + .nb_queue_pairs = nb_lcores > test_data->nb_qps + ? test_data->nb_qps : nb_lcores, + .max_nb_priv_xforms = NUM_MAX_XFORMS, + .max_nb_streams = 0 + }; + + if (rte_compressdev_configure(cdev_id, &config) < 0) { + RTE_LOG(ERR, USER1, "Device configuration failed\n"); + return -EINVAL; + } - test_data->decompressed_data = rte_zmalloc_socket(NULL, - test_data->input_data_sz, 0, - rte_socket_id()); - if (test_data->decompressed_data == NULL) { - RTE_LOG(ERR, USER1, "Memory to hold the data from the input " - "file could not be allocated\n"); - return -1; - } + for (j = 0; j < test_data->nb_qps; j++) { + ret = rte_compressdev_queue_pair_setup(cdev_id, j, + NUM_MAX_INFLIGHT_OPS, socket_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, + "Failed to setup queue pair %u on compressdev %u", + j, cdev_id); + return -EINVAL; + } + } - test_data->comp_bufs = rte_zmalloc_socket(NULL, - test_data->total_bufs * sizeof(struct rte_mbuf *), - 0, rte_socket_id()); - if (test_data->comp_bufs == NULL) { - RTE_LOG(ERR, USER1, "Memory to hold the compression mbufs" - " could not be allocated\n"); - return -1; + ret = rte_compressdev_start(cdev_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, + "Failed to start device %u: error %d\n", + cdev_id, ret); + return -EPERM; + } } - test_data->decomp_bufs = rte_zmalloc_socket(NULL, - test_data->total_bufs * sizeof(struct rte_mbuf *), - 0, rte_socket_id()); - if (test_data->decomp_bufs == NULL) { - RTE_LOG(ERR, USER1, "Memory to hold the decompression mbufs" - " could not be allocated\n"); - return -1; - } - return 0; + return enabled_cdev_count; } static int @@ -295,187 +281,18 @@ comp_perf_dump_input_data(struct comp_test_data *test_data) return ret; } -static int -comp_perf_initialize_compressdev(struct comp_test_data *test_data) -{ - uint8_t enabled_cdev_count; - uint8_t enabled_cdevs[RTE_COMPRESS_MAX_DEVS]; - - enabled_cdev_count = rte_compressdev_devices_get(test_data->driver_name, - enabled_cdevs, RTE_COMPRESS_MAX_DEVS); - if (enabled_cdev_count == 0) { - RTE_LOG(ERR, USER1, "No compress devices type %s available\n", - test_data->driver_name); - return -EINVAL; - } - - if (enabled_cdev_count > 1) - RTE_LOG(INFO, USER1, - "Only the first compress device will be used\n"); - - test_data->cdev_id = enabled_cdevs[0]; - - if (comp_perf_check_capabilities(test_data) < 0) - return -1; - - /* Configure compressdev (one device, one queue pair) */ - struct rte_compressdev_config config = { - .socket_id = rte_socket_id(), - .nb_queue_pairs = 1, - .max_nb_priv_xforms = NUM_MAX_XFORMS, - .max_nb_streams = 0 - }; - - if (rte_compressdev_configure(test_data->cdev_id, &config) < 0) { - RTE_LOG(ERR, USER1, "Device configuration failed\n"); - return -1; - } - - if (rte_compressdev_queue_pair_setup(test_data->cdev_id, 0, - NUM_MAX_INFLIGHT_OPS, rte_socket_id()) < 0) { - RTE_LOG(ERR, USER1, "Queue pair setup failed\n"); - return -1; - } - - if (rte_compressdev_start(test_data->cdev_id) < 0) { - RTE_LOG(ERR, USER1, "Device could not be started\n"); - return -1; - } - - return 0; -} - -static int -prepare_bufs(struct comp_test_data *test_data) -{ - uint32_t remaining_data = test_data->input_data_sz; - uint8_t *input_data_ptr = test_data->input_data; - size_t data_sz; - uint8_t *data_addr; - uint32_t i, j; - - for (i = 0; i < test_data->total_bufs; i++) { - /* Allocate data in input mbuf and copy data from input file */ - test_data->decomp_bufs[i] = - rte_pktmbuf_alloc(test_data->decomp_buf_pool); - if (test_data->decomp_bufs[i] == NULL) { - RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"); - return -1; - } - - cleanup = ST_PREPARE_BUF; - data_sz = RTE_MIN(remaining_data, test_data->seg_sz); - data_addr = (uint8_t *) rte_pktmbuf_append( - test_data->decomp_bufs[i], data_sz); - if (data_addr == NULL) { - RTE_LOG(ERR, USER1, "Could not append data\n"); - return -1; - } - rte_memcpy(data_addr, input_data_ptr, data_sz); - - input_data_ptr += data_sz; - remaining_data -= data_sz; - - /* Already one segment in the mbuf */ - uint16_t segs_per_mbuf = 1; - - /* Chain mbufs if needed for input mbufs */ - while (segs_per_mbuf < test_data->max_sgl_segs - && remaining_data > 0) { - struct rte_mbuf *next_seg = - rte_pktmbuf_alloc(test_data->decomp_buf_pool); - - if (next_seg == NULL) { - RTE_LOG(ERR, USER1, - "Could not allocate mbuf\n"); - return -1; - } - - data_sz = RTE_MIN(remaining_data, test_data->seg_sz); - data_addr = (uint8_t *)rte_pktmbuf_append(next_seg, - data_sz); - - if (data_addr == NULL) { - RTE_LOG(ERR, USER1, "Could not append data\n"); - return -1; - } - - rte_memcpy(data_addr, input_data_ptr, data_sz); - input_data_ptr += data_sz; - remaining_data -= data_sz; - - if (rte_pktmbuf_chain(test_data->decomp_bufs[i], - next_seg) < 0) { - RTE_LOG(ERR, USER1, "Could not chain mbufs\n"); - return -1; - } - segs_per_mbuf++; - } - - /* Allocate data in output mbuf */ - test_data->comp_bufs[i] = - rte_pktmbuf_alloc(test_data->comp_buf_pool); - if (test_data->comp_bufs[i] == NULL) { - RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"); - return -1; - } - data_addr = (uint8_t *) rte_pktmbuf_append( - test_data->comp_bufs[i], - test_data->out_seg_sz); - if (data_addr == NULL) { - RTE_LOG(ERR, USER1, "Could not append data\n"); - return -1; - } - - /* Chain mbufs if needed for output mbufs */ - for (j = 1; j < segs_per_mbuf; j++) { - struct rte_mbuf *next_seg = - rte_pktmbuf_alloc(test_data->comp_buf_pool); - - if (next_seg == NULL) { - RTE_LOG(ERR, USER1, - "Could not allocate mbuf\n"); - return -1; - } - - data_addr = (uint8_t *)rte_pktmbuf_append(next_seg, - test_data->out_seg_sz); - - if (data_addr == NULL) { - RTE_LOG(ERR, USER1, "Could not append data\n"); - return -1; - } - - if (rte_pktmbuf_chain(test_data->comp_bufs[i], - next_seg) < 0) { - RTE_LOG(ERR, USER1, "Could not chain mbufs\n"); - return -1; - } - } - } - - return 0; -} - -static void -free_bufs(struct comp_test_data *test_data) -{ - uint32_t i; - - for (i = 0; i < test_data->total_bufs; i++) { - rte_pktmbuf_free(test_data->comp_bufs[i]); - rte_pktmbuf_free(test_data->decomp_bufs[i]); - } -} - - - int main(int argc, char **argv) { - uint8_t level, level_idx = 0; + uint8_t level_idx = 0; int ret, i; struct comp_test_data *test_data; + void *ctx[RTE_MAX_LCORE] = {}; + uint8_t enabled_cdevs[RTE_COMPRESS_MAX_DEVS]; + int nb_compressdevs = 0; + uint16_t total_nb_qps = 0; + uint8_t cdev_id; + uint32_t lcore_id; /* Initialise DPDK EAL */ ret = rte_eal_init(argc, argv); @@ -492,7 +309,7 @@ main(int argc, char **argv) rte_socket_id()); ret = EXIT_SUCCESS; - cleanup = ST_TEST_DATA; + test_data->cleanup = ST_TEST_DATA; comp_perf_options_default(test_data); if (comp_perf_options_parse(test_data, argc, argv) < 0) { @@ -507,99 +324,112 @@ main(int argc, char **argv) goto end; } - if (comp_perf_initialize_compressdev(test_data) < 0) { - ret = EXIT_FAILURE; - goto end; - } + nb_compressdevs = + comp_perf_initialize_compressdev(test_data, enabled_cdevs); - cleanup = ST_COMPDEV; - if (comp_perf_dump_input_data(test_data) < 0) { + if (nb_compressdevs < 1) { ret = EXIT_FAILURE; goto end; } - cleanup = ST_INPUT_DATA; - if (comp_perf_allocate_memory(test_data) < 0) { + test_data->cleanup = ST_COMPDEV; + if (comp_perf_dump_input_data(test_data) < 0) { ret = EXIT_FAILURE; goto end; } - if (prepare_bufs(test_data) < 0) { - ret = EXIT_FAILURE; - goto end; - } + test_data->cleanup = ST_INPUT_DATA; - if (test_data->level.inc != 0) - level = test_data->level.min; + if (test_data->level_lst.inc != 0) + test_data->level = test_data->level_lst.min; else - level = test_data->level.list[0]; + test_data->level = test_data->level_lst.list[0]; printf("App uses socket: %u\n", rte_socket_id()); - printf("Driver uses socket: %u\n", - rte_compressdev_socket_id(test_data->cdev_id)); printf("Burst size = %u\n", test_data->burst_sz); printf("File size = %zu\n", test_data->input_data_sz); - printf("%6s%12s%17s%19s%21s%15s%21s%23s%16s\n", - "Level", "Comp size", "Comp ratio [%]", - "Comp [Cycles/it]", "Comp [Cycles/Byte]", "Comp [Gbps]", - "Decomp [Cycles/it]", "Decomp [Cycles/Byte]", "Decomp [Gbps]"); + test_data->cleanup = ST_DURING_TEST; + total_nb_qps = nb_compressdevs * test_data->nb_qps; - cleanup = ST_DURING_TEST; - while (level <= test_data->level.max) { + i = 0; + uint8_t qp_id = 0, cdev_index = 0; - /* - * Run a first iteration, to verify compression and - * get the compression ratio for the level - */ - if (cperf_verification(test_data, level) != EXIT_SUCCESS) - break; + RTE_LCORE_FOREACH_SLAVE(lcore_id) { - /* - * Run benchmarking test - */ - if (cperf_benchmark(test_data, level) != EXIT_SUCCESS) + if (i == total_nb_qps) break; - printf("%6u%12zu%17.2f%19"PRIu64"%21.2f" - "%15.2f%21"PRIu64"%23.2f%16.2f\n", - level, test_data->comp_data_sz, test_data->ratio, - test_data->comp_tsc_duration[level], - test_data->comp_tsc_byte, test_data->comp_gbps, - test_data->decomp_tsc_duration[level], - test_data->decomp_tsc_byte, test_data->decomp_gbps); + cdev_id = enabled_cdevs[cdev_index]; + ctx[i] = cperf_testmap[test_data->test].constructor( + cdev_id, qp_id, + test_data); + if (ctx[i] == NULL) { + RTE_LOG(ERR, USER1, "Test run constructor failed\n"); + goto end; + } + qp_id = (qp_id + 1) % test_data->nb_qps; + if (qp_id == 0) + cdev_index++; + i++; + } + + while (test_data->level <= test_data->level_lst.max) { + + i = 0; + RTE_LCORE_FOREACH_SLAVE(lcore_id) { + + if (i == total_nb_qps) + break; - if (test_data->level.inc != 0) - level += test_data->level.inc; + rte_eal_remote_launch( + cperf_testmap[test_data->test].runner, + ctx[i], lcore_id); + i++; + } + i = 0; + RTE_LCORE_FOREACH_SLAVE(lcore_id) { + + if (i == total_nb_qps) + break; + ret |= rte_eal_wait_lcore(lcore_id); + i++; + } + + if (ret != EXIT_SUCCESS) + break; + + if (test_data->level_lst.inc != 0) + test_data->level += test_data->level_lst.inc; else { - if (++level_idx == test_data->level.count) + if (++level_idx == test_data->level_lst.count) break; - level = test_data->level.list[level_idx]; + test_data->level = test_data->level_lst.list[level_idx]; } } end: - switch (cleanup) { + switch (test_data->cleanup) { case ST_DURING_TEST: - case ST_PREPARE_BUF: - free_bufs(test_data); - /* fallthrough */ - case ST_MEMORY_ALLOC: - rte_free(test_data->decomp_bufs); - rte_free(test_data->comp_bufs); - rte_free(test_data->decompressed_data); - rte_free(test_data->compressed_data); - rte_mempool_free(test_data->op_pool); - rte_mempool_free(test_data->decomp_buf_pool); - rte_mempool_free(test_data->comp_buf_pool); + i = 0; + RTE_LCORE_FOREACH_SLAVE(lcore_id) { + if (i == total_nb_qps) + break; + + if (ctx[i] && cperf_testmap[test_data->test].destructor) + cperf_testmap[test_data->test].destructor( + ctx[i]); + i++; + } /* fallthrough */ case ST_INPUT_DATA: rte_free(test_data->input_data); /* fallthrough */ case ST_COMPDEV: - if (test_data->cdev_id != -1) - rte_compressdev_stop(test_data->cdev_id); + for (i = 0; i < nb_compressdevs && + i < RTE_COMPRESS_MAX_DEVS; i++) + rte_compressdev_stop(enabled_cdevs[i]); /* fallthrough */ case ST_TEST_DATA: rte_free(test_data); @@ -616,3 +446,65 @@ main(int argc, char **argv) } return ret; } + +__rte_weak void * +cperf_benchmark_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused) +{ + RTE_LOG(INFO, USER1, "Benchmark test is not supported yet\n"); + return NULL; +} + +__rte_weak void +cperf_benchmark_test_destructor(void *arg __rte_unused) +{ + +} + +__rte_weak int +cperf_benchmark_test_runner(void *test_ctx __rte_unused) +{ + return 0; +} +__rte_weak void * +cperf_verify_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused) +{ + RTE_LOG(INFO, USER1, "Verify test is not supported yet\n"); + return NULL; +} + +__rte_weak void +cperf_verify_test_destructor(void *arg __rte_unused) +{ + +} + +__rte_weak int +cperf_verify_test_runner(void *test_ctx __rte_unused) +{ + return 0; +} + +__rte_weak void * +cperf_pmd_cyclecount_test_constructor(uint8_t dev_id __rte_unused, + uint16_t qp_id __rte_unused, + struct comp_test_data *options __rte_unused) +{ + RTE_LOG(INFO, USER1, "Pmd_cyclecount test is not supported yet\n"); + return NULL; +} + +__rte_weak void +cperf_pmd_cyclecount_test_destructor(void *arg __rte_unused) +{ + +} + +__rte_weak int +cperf_pmd_cyclecount_test_runner(void *test_ctx __rte_unused) +{ + return 0; +} diff --git a/app/test-compress-perf/meson.build b/app/test-compress-perf/meson.build index ec73e5e..00413c6 100644 --- a/app/test-compress-perf/meson.build +++ b/app/test-compress-perf/meson.build @@ -4,6 +4,5 @@ allow_experimental_apis = true sources = files('comp_perf_options_parse.c', 'main.c', - 'comp_perf_test_verify.c', - 'comp_perf_test_benchmark.c') + 'comp_perf_test_common.c') deps = ['compressdev'] From patchwork Wed Jun 26 16:30:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Tomasz_J=C3=B3=C5=BAwiak?= X-Patchwork-Id: 55420 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 62A5E2BE5; Wed, 26 Jun 2019 18:30:54 +0200 (CEST) Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by dpdk.org (Postfix) with ESMTP id E286E2B98 for ; Wed, 26 Jun 2019 18:30:46 +0200 (CEST) Received: by mail-wm1-f65.google.com with SMTP id a15so2767549wmj.5 for ; Wed, 26 Jun 2019 09:30:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=7tgU6qs64B716E7hP1GF5n2LLyCLNfi02XQLdiW5qxI=; b=ShVDJXnFqofKee/QjUF6cBEhYPWo1tb64hkQHZ1ZQyPPs61+EBxAPPwD80Du96gsCN +F6Bgbt0svqDNardR70g8MHWM3kV9QIMmUGJ2yisSgr2yEGmdj1hNecpGpcPLpVMmo3m 9GLY9bkIEHkYC5+9KC/2qjya6Nn2fyOzAs0aQp77RUvCf6b2fp4xMD47RZG4TILV60Ly VoaqWWNpVJRGQMew6Sxy5NBO5XuNfkhPici55dn1dSx5q31+ae+rdebSLBBNzag36YhR aiq9m/+2UavkEGaKdDOEDlc223KwPCnjiITxdElRQJGYvlcrJn5yXfagslqfwyCdF/+J Vfjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=7tgU6qs64B716E7hP1GF5n2LLyCLNfi02XQLdiW5qxI=; b=W21YKESmDxGeEVfUSSPC55lhz3SBZG1N3rvH+4Fa4GX6Rhwy39BQk2kpPMY6ULechd zr4IRwXN0AWAf1Io5w7/I34M2+EDirvDiTPJ/wVp21XhK9aKBhMMeCOcpcadyGXO3h+D ja2N8s9fjZi5sc+JxGrRCkq2FQ9qOmsn7wBpfITHR/gRJaR09d+8CzgqVbxNdyAY/h0/ BA13h8Y7miPPdlZZwHiQE4QOMttd/YmviQ/Rh/iIulrehUaXAaBngRT4txdKD5FW/jaB pBvWdUuNl2X8Z31y91WiXZBtQuogg2AUAmQJt9q80XLNkwXSREeHP4VJR3bHE7pJ7WVh HVHA== X-Gm-Message-State: APjAAAUBhAPN8ohJOtst/Z+pN3i4zlkM4Uw3bXfHx0nV8BUdqXPhL/Vd 5L0wiv57z7qxWJOatUNU1eOsBsKFZTGT4Q== X-Google-Smtp-Source: APXvYqzVfvOW0Tz8xI1iLeyH8khSOhPI78CrjHQrZgEShuPqgxEGHLDcqggj0XBUwdDzZRAc7sTr7w== X-Received: by 2002:a1c:ef10:: with SMTP id n16mr3084005wmh.134.1561566646457; Wed, 26 Jun 2019 09:30:46 -0700 (PDT) Received: from localhost.localdomain (178.43.130.103.ipv4.supernova.orange.pl. [178.43.130.103]) by smtp.gmail.com with ESMTPSA id n10sm16989663wrw.83.2019.06.26.09.30.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 26 Jun 2019 09:30:45 -0700 (PDT) From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm@gmail.com, shallyv@marvell.com, arturx.trybula@intel.com Date: Wed, 26 Jun 2019 18:30:28 +0200 Message-Id: <1561566633-12329-3-git-send-email-tjozwiakgm@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> References: <1560032553-15528-1-git-send-email-tjozwiakgm@gmail.com> <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> Subject: [dpdk-dev] [PATCH v3 2/7] app/test-compress-perf: add ptest command line option X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tomasz Jozwiak This patch adds --ptest option to make possible a choose of test case from command line. Signed-off-by: Tomasz Jozwiak --- app/test-compress-perf/comp_perf_options_parse.c | 36 ++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/app/test-compress-perf/comp_perf_options_parse.c b/app/test-compress-perf/comp_perf_options_parse.c index bc4b98a..07672b2 100644 --- a/app/test-compress-perf/comp_perf_options_parse.c +++ b/app/test-compress-perf/comp_perf_options_parse.c @@ -15,6 +15,7 @@ #include "comp_perf_options.h" +#define CPERF_PTEST_TYPE ("ptest") #define CPERF_DRIVER_NAME ("driver-name") #define CPERF_TEST_FILE ("input-file") #define CPERF_SEG_SIZE ("seg-sz") @@ -37,6 +38,7 @@ static void usage(char *progname) { printf("%s [EAL options] --\n" + " --ptest benchmark / verify :" " --driver-name NAME: compress driver to use\n" " --input-file NAME: file to compress and decompress\n" " --extended-input-sz N: extend file data up to this size (default: no extension)\n" @@ -76,6 +78,37 @@ get_str_key_id_mapping(struct name_id_map *map, unsigned int map_len, } static int +parse_cperf_test_type(struct comp_test_data *test_data, const char *arg) +{ + struct name_id_map cperftest_namemap[] = { + { + cperf_test_type_strs[CPERF_TEST_TYPE_BENCHMARK], + CPERF_TEST_TYPE_BENCHMARK + }, + { + cperf_test_type_strs[CPERF_TEST_TYPE_VERIFY], + CPERF_TEST_TYPE_VERIFY + }, + { + cperf_test_type_strs[CPERF_TEST_TYPE_PMDCC], + CPERF_TEST_TYPE_PMDCC + } + }; + + int id = get_str_key_id_mapping( + (struct name_id_map *)cperftest_namemap, + RTE_DIM(cperftest_namemap), arg); + if (id < 0) { + RTE_LOG(ERR, USER1, "failed to parse test type"); + return -1; + } + + test_data->test = (enum cperf_perf_test_type)id; + + return 0; +} + +static int parse_uint32_t(uint32_t *value, const char *arg) { char *end = NULL; @@ -499,6 +532,8 @@ struct long_opt_parser { }; static struct option lgopts[] = { + + { CPERF_PTEST_TYPE, required_argument, 0, 0 }, { CPERF_DRIVER_NAME, required_argument, 0, 0 }, { CPERF_TEST_FILE, required_argument, 0, 0 }, { CPERF_SEG_SIZE, required_argument, 0, 0 }, @@ -517,6 +552,7 @@ static int comp_perf_opts_parse_long(int opt_idx, struct comp_test_data *test_data) { struct long_opt_parser parsermap[] = { + { CPERF_PTEST_TYPE, parse_cperf_test_type }, { CPERF_DRIVER_NAME, parse_driver_name }, { CPERF_TEST_FILE, parse_test_file }, { CPERF_SEG_SIZE, parse_seg_sz }, From patchwork Wed Jun 26 16:30:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Tomasz_J=C3=B3=C5=BAwiak?= X-Patchwork-Id: 55421 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 65F072BF7; Wed, 26 Jun 2019 18:30:56 +0200 (CEST) Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by dpdk.org (Postfix) with ESMTP id 731112B99 for ; Wed, 26 Jun 2019 18:30:48 +0200 (CEST) Received: by mail-wr1-f48.google.com with SMTP id f9so3437874wre.12 for ; Wed, 26 Jun 2019 09:30:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=W6HVRBePOzSU+y/dOxpGpTSMxwPiM/2NiAt5phUK814=; b=jG9hq/cOCVTX7eDrKcC8qnxxs2PBWS8IUnMOYt4T2M3QDzRe5YInYLbCbUEM1oAD61 TMmw0qgDn5AcdvQdLjlCKtUaI9TbVEQyaXsSiVvEDFBJUCaMstSP3IwPPbrSZCgs1nKm OTHW/UhlW4sPVcymGc4zlai9k9FDsNW7kv+1UrR30fO5B6wXfFlz6iFjnqj+zqH4U/ms 5kwWUFTWLvnhhbCuf71Vk1In3FmNLSqmSHhXg9Nk8RVADkkHz5Ew4YVWb3dAphjS1+9K yRXDb3g29FRuAjXQmmr77REIfYOyQuP6fI/BJzTHggabjf2kppSGpUx8SeZTvWamb7bp n83w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=W6HVRBePOzSU+y/dOxpGpTSMxwPiM/2NiAt5phUK814=; b=dtJlA6n9QhOESbYCqA4bH75Xu5NpoFujZgNh2xhu5UfdecTBo3nxzDAC0aYoGqJBx2 zHSjoanNL/Jpg00o01w1jQ0uXQ1AYyG54M+wQuLwVXGde8YTbAlGWGcwbBMyav1OMeX0 ijWiiTZkb1ckJ+TmqBZkIfSP61ZZ1LjVbEf1OTQGEfz4Nv3Sc1VbvnZILOz3jAhaFIUA kilJm4sLkHgud2ppN3H/hJeARtf2pOb6fuELUpX8NAhlMlqPsQFwPbdIqsDUqR28sVTx l8pv1aCB4cvc1SxrXam+eMWOVFhk5HBGXpKo7X6nih5INuyY7NYljzbRjDOOhyEo4g0O KFLA== X-Gm-Message-State: APjAAAVzrOHos5DbelUNMgmFLr1H7WjM/axfqqrEoMaUQj32BlklJDQ/ GlbmlHlH72x6/etucsk/mSn/vp6WO9reRA== X-Google-Smtp-Source: APXvYqwv2KJVXekvVFnacxA7PxQTwgmUAgQRyvWHyc0b09KqcfXzQGCIszjMaNXTGvV6Fh8sG3d3VQ== X-Received: by 2002:adf:ea0d:: with SMTP id q13mr4328093wrm.101.1561566647900; Wed, 26 Jun 2019 09:30:47 -0700 (PDT) Received: from localhost.localdomain (178.43.130.103.ipv4.supernova.orange.pl. [178.43.130.103]) by smtp.gmail.com with ESMTPSA id n10sm16989663wrw.83.2019.06.26.09.30.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 26 Jun 2019 09:30:47 -0700 (PDT) From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm@gmail.com, shallyv@marvell.com, arturx.trybula@intel.com Date: Wed, 26 Jun 2019 18:30:29 +0200 Message-Id: <1561566633-12329-4-git-send-email-tjozwiakgm@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> References: <1560032553-15528-1-git-send-email-tjozwiakgm@gmail.com> <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> Subject: [dpdk-dev] [PATCH v3 3/7] app/test-compress-perf: add verification test case X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tomasz Jozwiak This patch adds a verification part to compression-perf-tool as a separate test case, which can be executed multi-threaded. Signed-off-by: Tomasz Jozwiak --- app/test-compress-perf/Makefile | 1 + app/test-compress-perf/comp_perf_test_verify.c | 122 ++++++++++++++++++------- app/test-compress-perf/comp_perf_test_verify.h | 24 ++++- app/test-compress-perf/main.c | 1 + app/test-compress-perf/meson.build | 1 + 5 files changed, 112 insertions(+), 37 deletions(-) diff --git a/app/test-compress-perf/Makefile b/app/test-compress-perf/Makefile index de74129..f54d9a4 100644 --- a/app/test-compress-perf/Makefile +++ b/app/test-compress-perf/Makefile @@ -12,6 +12,7 @@ CFLAGS += -O3 # all source are stored in SRCS-y SRCS-y := main.c SRCS-y += comp_perf_options_parse.c +SRCS-y += comp_perf_test_verify.c SRCS-y += comp_perf_test_common.c include $(RTE_SDK)/mk/rte.app.mk diff --git a/app/test-compress-perf/comp_perf_test_verify.c b/app/test-compress-perf/comp_perf_test_verify.c index 28a0fe8..c2aab70 100644 --- a/app/test-compress-perf/comp_perf_test_verify.c +++ b/app/test-compress-perf/comp_perf_test_verify.c @@ -8,14 +8,48 @@ #include #include "comp_perf_test_verify.h" +#include "comp_perf_test_common.h" + +void +cperf_verify_test_destructor(void *arg) +{ + if (arg) { + comp_perf_free_memory(&((struct cperf_verify_ctx *)arg)->mem); + rte_free(arg); + } +} + +void * +cperf_verify_test_constructor(uint8_t dev_id, uint16_t qp_id, + struct comp_test_data *options) +{ + struct cperf_verify_ctx *ctx = NULL; + + ctx = rte_malloc(NULL, sizeof(struct cperf_verify_ctx), 0); + + if (ctx != NULL) { + ctx->mem.dev_id = dev_id; + ctx->mem.qp_id = qp_id; + ctx->options = options; + + if (!comp_perf_allocate_memory(ctx->options, &ctx->mem) && + !prepare_bufs(ctx->options, &ctx->mem)) + return ctx; + } + + cperf_verify_test_destructor(ctx); + return NULL; +} static int -main_loop(struct comp_test_data *test_data, uint8_t level, - enum rte_comp_xform_type type, - uint8_t *output_data_ptr, - size_t *output_data_sz) +main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) { - uint8_t dev_id = test_data->cdev_id; + struct comp_test_data *test_data = ctx->options; + uint8_t *output_data_ptr; + size_t *output_data_sz; + struct cperf_mem_resources *mem = &ctx->mem; + + uint8_t dev_id = mem->dev_id; uint32_t i, iter, num_iter; struct rte_comp_op **ops, **deq_ops; void *priv_xform = NULL; @@ -33,7 +67,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } ops = rte_zmalloc_socket(NULL, - 2 * test_data->total_bufs * sizeof(struct rte_comp_op *), + 2 * mem->total_bufs * sizeof(struct rte_comp_op *), 0, rte_socket_id()); if (ops == NULL) { @@ -42,7 +76,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, return -1; } - deq_ops = &ops[test_data->total_bufs]; + deq_ops = &ops[mem->total_bufs]; if (type == RTE_COMP_COMPRESS) { xform = (struct rte_comp_xform) { @@ -50,14 +84,16 @@ main_loop(struct comp_test_data *test_data, uint8_t level, .compress = { .algo = RTE_COMP_ALGO_DEFLATE, .deflate.huffman = test_data->huffman_enc, - .level = level, + .level = test_data->level, .window_size = test_data->window_sz, .chksum = RTE_COMP_CHECKSUM_NONE, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; - input_bufs = test_data->decomp_bufs; - output_bufs = test_data->comp_bufs; + output_data_ptr = ctx->mem.compressed_data; + output_data_sz = &ctx->comp_data_sz; + input_bufs = mem->decomp_bufs; + output_bufs = mem->comp_bufs; out_seg_sz = test_data->out_seg_sz; } else { xform = (struct rte_comp_xform) { @@ -69,8 +105,10 @@ main_loop(struct comp_test_data *test_data, uint8_t level, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; - input_bufs = test_data->comp_bufs; - output_bufs = test_data->decomp_bufs; + output_data_ptr = ctx->mem.decompressed_data; + output_data_sz = &ctx->decomp_data_sz; + input_bufs = mem->comp_bufs; + output_bufs = mem->decomp_bufs; out_seg_sz = test_data->seg_sz; } @@ -85,8 +123,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, num_iter = 1; for (iter = 0; iter < num_iter; iter++) { - uint32_t total_ops = test_data->total_bufs; - uint32_t remaining_ops = test_data->total_bufs; + uint32_t total_ops = mem->total_bufs; + uint32_t remaining_ops = mem->total_bufs; uint32_t total_deq_ops = 0; uint32_t total_enq_ops = 0; uint16_t ops_unused = 0; @@ -113,7 +151,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, /* Allocate compression operations */ if (ops_needed && !rte_comp_op_bulk_alloc( - test_data->op_pool, + mem->op_pool, &ops[ops_unused], ops_needed)) { RTE_LOG(ERR, USER1, @@ -149,7 +187,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, ops[op_id]->private_xform = priv_xform; } - num_enq = rte_compressdev_enqueue_burst(dev_id, 0, ops, + num_enq = rte_compressdev_enqueue_burst(dev_id, + mem->qp_id, ops, num_ops); if (num_enq == 0) { struct rte_compressdev_stats stats; @@ -165,7 +204,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, remaining_ops -= num_enq; total_enq_ops += num_enq; - num_deq = rte_compressdev_dequeue_burst(dev_id, 0, + num_deq = rte_compressdev_dequeue_burst(dev_id, + mem->qp_id, deq_ops, test_data->burst_sz); total_deq_ops += num_deq; @@ -220,15 +260,17 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } } } - rte_mempool_put_bulk(test_data->op_pool, + rte_mempool_put_bulk(mem->op_pool, (void **)deq_ops, num_deq); allocated -= num_deq; } /* Dequeue the last operations */ while (total_deq_ops < total_ops) { - num_deq = rte_compressdev_dequeue_burst(dev_id, 0, - deq_ops, test_data->burst_sz); + num_deq = rte_compressdev_dequeue_burst(dev_id, + mem->qp_id, + deq_ops, + test_data->burst_sz); if (num_deq == 0) { struct rte_compressdev_stats stats; @@ -291,7 +333,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } } } - rte_mempool_put_bulk(test_data->op_pool, + rte_mempool_put_bulk(mem->op_pool, (void **)deq_ops, num_deq); allocated -= num_deq; } @@ -300,45 +342,45 @@ main_loop(struct comp_test_data *test_data, uint8_t level, if (output_data_sz) *output_data_sz = output_size; end: - rte_mempool_put_bulk(test_data->op_pool, (void **)ops, allocated); + rte_mempool_put_bulk(mem->op_pool, (void **)ops, allocated); rte_compressdev_private_xform_free(dev_id, priv_xform); rte_free(ops); return res; } - - int -cperf_verification(struct comp_test_data *test_data, uint8_t level) +cperf_verify_test_runner(void *test_ctx) { + struct cperf_verify_ctx *ctx = test_ctx; + struct comp_test_data *test_data = ctx->options; int ret = EXIT_SUCCESS; + static rte_atomic16_t display_once = RTE_ATOMIC16_INIT(0); + uint32_t lcore = rte_lcore_id(); + + ctx->mem.lcore_id = lcore; test_data->ratio = 0; - if (main_loop(test_data, level, RTE_COMP_COMPRESS, - test_data->compressed_data, - &test_data->comp_data_sz) < 0) { + if (main_loop(ctx, RTE_COMP_COMPRESS) < 0) { ret = EXIT_FAILURE; goto end; } - if (main_loop(test_data, level, RTE_COMP_DECOMPRESS, - test_data->decompressed_data, - &test_data->decomp_data_sz) < 0) { + if (main_loop(ctx, RTE_COMP_DECOMPRESS) < 0) { ret = EXIT_FAILURE; goto end; } - if (test_data->decomp_data_sz != test_data->input_data_sz) { + if (ctx->decomp_data_sz != test_data->input_data_sz) { RTE_LOG(ERR, USER1, "Decompressed data length not equal to input data length\n"); RTE_LOG(ERR, USER1, "Decompressed size = %zu, expected = %zu\n", - test_data->decomp_data_sz, test_data->input_data_sz); + ctx->decomp_data_sz, test_data->input_data_sz); ret = EXIT_FAILURE; goto end; } else { - if (memcmp(test_data->decompressed_data, + if (memcmp(ctx->mem.decompressed_data, test_data->input_data, test_data->input_data_sz) != 0) { RTE_LOG(ERR, USER1, @@ -348,9 +390,19 @@ cperf_verification(struct comp_test_data *test_data, uint8_t level) } } - test_data->ratio = (double) test_data->comp_data_sz / + ctx->ratio = (double) ctx->comp_data_sz / test_data->input_data_sz * 100; + if (!ctx->silent) { + if (rte_atomic16_test_and_set(&display_once)) { + printf("%12s%6s%12s%17s\n", + "lcore id", "Level", "Comp size", "Comp ratio [%]"); + } + printf("%12u%6u%12zu%17.2f\n", + ctx->mem.lcore_id, + test_data->level, ctx->comp_data_sz, ctx->ratio); + } + end: return ret; } diff --git a/app/test-compress-perf/comp_perf_test_verify.h b/app/test-compress-perf/comp_perf_test_verify.h index 67c6b49..ae8b742 100644 --- a/app/test-compress-perf/comp_perf_test_verify.h +++ b/app/test-compress-perf/comp_perf_test_verify.h @@ -1,13 +1,33 @@ /* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation + * Copyright(c) 2018-2019 Intel Corporation */ #ifndef _COMP_PERF_TEST_VERIFY_ #define _COMP_PERF_TEST_VERIFY_ +#include + #include "comp_perf_options.h" +#include "comp_perf_test_common.h" + +struct cperf_verify_ctx { + struct cperf_mem_resources mem; + struct comp_test_data *options; + + int silent; + size_t comp_data_sz; + size_t decomp_data_sz; + double ratio; +}; + +void +cperf_verify_test_destructor(void *arg); int -cperf_verification(struct comp_test_data *test_data, uint8_t level); +cperf_verify_test_runner(void *test_ctx); + +void * +cperf_verify_test_constructor(uint8_t dev_id, uint16_t qp_id, + struct comp_test_data *options); #endif diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index d142b91..eba2795 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -8,6 +8,7 @@ #include #include "comp_perf_options.h" +#include "comp_perf_test_verify.h" #include "comp_perf.h" #include "comp_perf_test_common.h" diff --git a/app/test-compress-perf/meson.build b/app/test-compress-perf/meson.build index 00413c6..c6246e5 100644 --- a/app/test-compress-perf/meson.build +++ b/app/test-compress-perf/meson.build @@ -4,5 +4,6 @@ allow_experimental_apis = true sources = files('comp_perf_options_parse.c', 'main.c', + 'comp_perf_test_verify.c', 'comp_perf_test_common.c') deps = ['compressdev'] From patchwork Wed Jun 26 16:30:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Tomasz_J=C3=B3=C5=BAwiak?= X-Patchwork-Id: 55422 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8A4D12C30; Wed, 26 Jun 2019 18:30:58 +0200 (CEST) Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by dpdk.org (Postfix) with ESMTP id C0EAA2B99 for ; Wed, 26 Jun 2019 18:30:49 +0200 (CEST) Received: by mail-wr1-f65.google.com with SMTP id k11so3503128wrl.1 for ; Wed, 26 Jun 2019 09:30:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=8lDIjerjVwnaAuBwNiZuvlb1F2/RjY/09Bae1SYc5Kk=; b=F1rMnrUxU3JueL8cqAb0J0BKTMUwjVD2+60OMo81lLolrw1B/1iYjCmJzvtZ1H8C7E r2Nf/hw0yiSErZgBKmORpt2SIS4A2kC+9dVVvGnfuJ6Kk8MHlpIc28pYnY3/G0kO/4Ro wpnv0OQwixI2v2vIAPOVav1X7mwBxCMxg0/AGuCgdsys7OaenB//Ul6zqHkHZ+RoTADN qEKPYkNZ0PwA5oegn0rvgGtHjElLtEU0YdZUMcdNk8hMQKK7NP6SH4NGx/LeqrT1mB6k gyVt5mON2j2310GZU/h884AkO+/gey+bjD6c8TpToEn296JyivLZQ7vCUKFkvEdU2NtK 0wNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=8lDIjerjVwnaAuBwNiZuvlb1F2/RjY/09Bae1SYc5Kk=; b=pIONLi/YwylD2ClYELnYSNKX/+zXvPvWwmO2qhwzHQJrbCwmHpNfWv12yz9TOw+ki0 9onmophdu6bhiHSd46GIz7QRCZsDHwBd0MGaYR6o1ZWQaOgsoqBKBiOku0zNxmbYVdqI 7Mdw4rq/VxuhE0yQO+RVk1uigqGE40vhtGmjXbEqdoqt72gzHspnRazFQ8kYfKSTnC6x TqOOKFIyf6GeIs53F7vCp6PTvIkBnEyhMzXIvJ+SLEpvtWcZaGvC2sh2iKxh7JnDlO3N hpSR3Tjlq8enex5tRd300UtMiCmbORMaIUtZdpgBzP85lDX1L3Uigzd1n0jRBo+BXkJs BMqg== X-Gm-Message-State: APjAAAUQxu6k9qxnOYazPklDsJpmlslP3vPNka233W4kvp7DV2N/67SS KYKnfZXlX/4jFl6BSyJcqwggXF7/eoeRmQ== X-Google-Smtp-Source: APXvYqzQDABgGJLZ2ADvta9ITYmokHcXHLplnaaCvxxxv2b0KFc9gpxcVo3HURDeUHohkEXytIqFAQ== X-Received: by 2002:adf:8028:: with SMTP id 37mr4302139wrk.106.1561566649275; Wed, 26 Jun 2019 09:30:49 -0700 (PDT) Received: from localhost.localdomain (178.43.130.103.ipv4.supernova.orange.pl. [178.43.130.103]) by smtp.gmail.com with ESMTPSA id n10sm16989663wrw.83.2019.06.26.09.30.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 26 Jun 2019 09:30:48 -0700 (PDT) From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm@gmail.com, shallyv@marvell.com, arturx.trybula@intel.com Date: Wed, 26 Jun 2019 18:30:30 +0200 Message-Id: <1561566633-12329-5-git-send-email-tjozwiakgm@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> References: <1560032553-15528-1-git-send-email-tjozwiakgm@gmail.com> <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> Subject: [dpdk-dev] [PATCH v3 4/7] app/test-compress-perf: add benchmark test case X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tomasz Jozwiak This patch adds a benchmark part to compression-perf-tool as a separate test case, which can be executed multi-threaded. Signed-off-by: Tomasz Jozwiak Signed-off-by: Tomasz Jozwiak --- app/test-compress-perf/Makefile | 1 + app/test-compress-perf/comp_perf_test_benchmark.c | 139 ++++++++++++++++------ app/test-compress-perf/comp_perf_test_benchmark.h | 25 +++- app/test-compress-perf/main.c | 1 + app/test-compress-perf/meson.build | 1 + 5 files changed, 129 insertions(+), 38 deletions(-) diff --git a/app/test-compress-perf/Makefile b/app/test-compress-perf/Makefile index f54d9a4..d1a6820 100644 --- a/app/test-compress-perf/Makefile +++ b/app/test-compress-perf/Makefile @@ -13,6 +13,7 @@ CFLAGS += -O3 SRCS-y := main.c SRCS-y += comp_perf_options_parse.c SRCS-y += comp_perf_test_verify.c +SRCS-y += comp_perf_test_benchmark.c SRCS-y += comp_perf_test_common.c include $(RTE_SDK)/mk/rte.app.mk diff --git a/app/test-compress-perf/comp_perf_test_benchmark.c b/app/test-compress-perf/comp_perf_test_benchmark.c index 5752906..5f27bae 100644 --- a/app/test-compress-perf/comp_perf_test_benchmark.c +++ b/app/test-compress-perf/comp_perf_test_benchmark.c @@ -10,11 +10,45 @@ #include "comp_perf_test_benchmark.h" +void +cperf_benchmark_test_destructor(void *arg) +{ + if (arg) { + comp_perf_free_memory( + &((struct cperf_benchmark_ctx *)arg)->ver.mem); + rte_free(arg); + } +} + +void * +cperf_benchmark_test_constructor(uint8_t dev_id, uint16_t qp_id, + struct comp_test_data *options) +{ + struct cperf_benchmark_ctx *ctx = NULL; + + ctx = rte_malloc(NULL, sizeof(struct cperf_benchmark_ctx), 0); + + if (ctx != NULL) { + ctx->ver.mem.dev_id = dev_id; + ctx->ver.mem.qp_id = qp_id; + ctx->ver.options = options; + ctx->ver.silent = 1; /* ver. part will be silent */ + + if (!comp_perf_allocate_memory(ctx->ver.options, &ctx->ver.mem) + && !prepare_bufs(ctx->ver.options, &ctx->ver.mem)) + return ctx; + } + + cperf_benchmark_test_destructor(ctx); + return NULL; +} + static int -main_loop(struct comp_test_data *test_data, uint8_t level, - enum rte_comp_xform_type type) +main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) { - uint8_t dev_id = test_data->cdev_id; + struct comp_test_data *test_data = ctx->ver.options; + struct cperf_mem_resources *mem = &ctx->ver.mem; + uint8_t dev_id = mem->dev_id; uint32_t i, iter, num_iter; struct rte_comp_op **ops, **deq_ops; void *priv_xform = NULL; @@ -31,7 +65,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } ops = rte_zmalloc_socket(NULL, - 2 * test_data->total_bufs * sizeof(struct rte_comp_op *), + 2 * mem->total_bufs * sizeof(struct rte_comp_op *), 0, rte_socket_id()); if (ops == NULL) { @@ -40,7 +74,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, return -1; } - deq_ops = &ops[test_data->total_bufs]; + deq_ops = &ops[mem->total_bufs]; if (type == RTE_COMP_COMPRESS) { xform = (struct rte_comp_xform) { @@ -48,14 +82,14 @@ main_loop(struct comp_test_data *test_data, uint8_t level, .compress = { .algo = RTE_COMP_ALGO_DEFLATE, .deflate.huffman = test_data->huffman_enc, - .level = level, + .level = test_data->level, .window_size = test_data->window_sz, .chksum = RTE_COMP_CHECKSUM_NONE, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; - input_bufs = test_data->decomp_bufs; - output_bufs = test_data->comp_bufs; + input_bufs = mem->decomp_bufs; + output_bufs = mem->comp_bufs; out_seg_sz = test_data->out_seg_sz; } else { xform = (struct rte_comp_xform) { @@ -67,8 +101,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; - input_bufs = test_data->comp_bufs; - output_bufs = test_data->decomp_bufs; + input_bufs = mem->comp_bufs; + output_bufs = mem->decomp_bufs; out_seg_sz = test_data->seg_sz; } @@ -82,13 +116,13 @@ main_loop(struct comp_test_data *test_data, uint8_t level, uint64_t tsc_start, tsc_end, tsc_duration; - tsc_start = tsc_end = tsc_duration = 0; - tsc_start = rte_rdtsc(); num_iter = test_data->num_iter; + tsc_start = tsc_end = tsc_duration = 0; + tsc_start = rte_rdtsc_precise(); for (iter = 0; iter < num_iter; iter++) { - uint32_t total_ops = test_data->total_bufs; - uint32_t remaining_ops = test_data->total_bufs; + uint32_t total_ops = mem->total_bufs; + uint32_t remaining_ops = mem->total_bufs; uint32_t total_deq_ops = 0; uint32_t total_enq_ops = 0; uint16_t ops_unused = 0; @@ -113,7 +147,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, /* Allocate compression operations */ if (ops_needed && !rte_comp_op_bulk_alloc( - test_data->op_pool, + mem->op_pool, &ops[ops_unused], ops_needed)) { RTE_LOG(ERR, USER1, @@ -149,7 +183,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, ops[op_id]->private_xform = priv_xform; } - num_enq = rte_compressdev_enqueue_burst(dev_id, 0, ops, + num_enq = rte_compressdev_enqueue_burst(dev_id, + mem->qp_id, ops, num_ops); if (num_enq == 0) { struct rte_compressdev_stats stats; @@ -165,7 +200,8 @@ main_loop(struct comp_test_data *test_data, uint8_t level, remaining_ops -= num_enq; total_enq_ops += num_enq; - num_deq = rte_compressdev_dequeue_burst(dev_id, 0, + num_deq = rte_compressdev_dequeue_burst(dev_id, + mem->qp_id, deq_ops, test_data->burst_sz); total_deq_ops += num_deq; @@ -177,7 +213,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, if (op->status != RTE_COMP_OP_STATUS_SUCCESS) { RTE_LOG(ERR, USER1, - "Some operations were not successful\n"); + "Some operations were not successful\n"); goto end; } @@ -198,15 +234,17 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } } } - rte_mempool_put_bulk(test_data->op_pool, + rte_mempool_put_bulk(mem->op_pool, (void **)deq_ops, num_deq); allocated -= num_deq; } /* Dequeue the last operations */ while (total_deq_ops < total_ops) { - num_deq = rte_compressdev_dequeue_burst(dev_id, 0, - deq_ops, test_data->burst_sz); + num_deq = rte_compressdev_dequeue_burst(dev_id, + mem->qp_id, + deq_ops, + test_data->burst_sz); if (num_deq == 0) { struct rte_compressdev_stats stats; @@ -226,7 +264,7 @@ main_loop(struct comp_test_data *test_data, uint8_t level, if (op->status != RTE_COMP_OP_STATUS_SUCCESS) { RTE_LOG(ERR, USER1, - "Some operations were not successful\n"); + "Some operations were not successful\n"); goto end; } @@ -247,65 +285,92 @@ main_loop(struct comp_test_data *test_data, uint8_t level, } } } - rte_mempool_put_bulk(test_data->op_pool, + rte_mempool_put_bulk(mem->op_pool, (void **)deq_ops, num_deq); allocated -= num_deq; } } - tsc_end = rte_rdtsc(); + tsc_end = rte_rdtsc_precise(); tsc_duration = tsc_end - tsc_start; if (type == RTE_COMP_COMPRESS) - test_data->comp_tsc_duration[level] = + ctx->comp_tsc_duration[test_data->level] = tsc_duration / num_iter; else - test_data->decomp_tsc_duration[level] = + ctx->decomp_tsc_duration[test_data->level] = tsc_duration / num_iter; end: - rte_mempool_put_bulk(test_data->op_pool, (void **)ops, allocated); + rte_mempool_put_bulk(mem->op_pool, (void **)ops, allocated); rte_compressdev_private_xform_free(dev_id, priv_xform); rte_free(ops); return res; } int -cperf_benchmark(struct comp_test_data *test_data, uint8_t level) +cperf_benchmark_test_runner(void *test_ctx) { + struct cperf_benchmark_ctx *ctx = test_ctx; + struct comp_test_data *test_data = ctx->ver.options; + uint32_t lcore = rte_lcore_id(); + static rte_atomic16_t display_once = RTE_ATOMIC16_INIT(0); + + ctx->ver.mem.lcore_id = lcore; int i, ret = EXIT_SUCCESS; /* + * First the verification part is needed + */ + if (cperf_verify_test_runner(&ctx->ver)) { + ret = EXIT_FAILURE; + goto end; + } + + /* * Run the tests twice, discarding the first performance * results, before the cache is warmed up */ for (i = 0; i < 2; i++) { - if (main_loop(test_data, level, RTE_COMP_COMPRESS) < 0) { + if (main_loop(ctx, RTE_COMP_COMPRESS) < 0) { ret = EXIT_FAILURE; goto end; } } for (i = 0; i < 2; i++) { - if (main_loop(test_data, level, RTE_COMP_DECOMPRESS) < 0) { + if (main_loop(ctx, RTE_COMP_DECOMPRESS) < 0) { ret = EXIT_FAILURE; goto end; } } - test_data->comp_tsc_byte = - (double)(test_data->comp_tsc_duration[level]) / + ctx->comp_tsc_byte = + (double)(ctx->comp_tsc_duration[test_data->level]) / test_data->input_data_sz; - test_data->decomp_tsc_byte = - (double)(test_data->decomp_tsc_duration[level]) / + ctx->decomp_tsc_byte = + (double)(ctx->decomp_tsc_duration[test_data->level]) / test_data->input_data_sz; - test_data->comp_gbps = rte_get_tsc_hz() / test_data->comp_tsc_byte * 8 / + ctx->comp_gbps = rte_get_tsc_hz() / ctx->comp_tsc_byte * 8 / + 1000000000; + + ctx->decomp_gbps = rte_get_tsc_hz() / ctx->decomp_tsc_byte * 8 / 1000000000; - test_data->decomp_gbps = rte_get_tsc_hz() / test_data->decomp_tsc_byte - * 8 / 1000000000; + if (rte_atomic16_test_and_set(&display_once)) { + printf("%12s%6s%12s%17s%15s%16s\n", + "lcore id", "Level", "Comp size", "Comp ratio [%]", + "Comp [Gbps]", "Decomp [Gbps]"); + } + + printf("%12u%6u%12zu%17.2f%15.2f%16.2f\n", + ctx->ver.mem.lcore_id, + test_data->level, ctx->ver.comp_data_sz, ctx->ver.ratio, + ctx->comp_gbps, + ctx->decomp_gbps); + end: return ret; } diff --git a/app/test-compress-perf/comp_perf_test_benchmark.h b/app/test-compress-perf/comp_perf_test_benchmark.h index b193445..d9b2694 100644 --- a/app/test-compress-perf/comp_perf_test_benchmark.h +++ b/app/test-compress-perf/comp_perf_test_benchmark.h @@ -5,9 +5,32 @@ #ifndef _COMP_PERF_TEST_BENCHMARK_ #define _COMP_PERF_TEST_BENCHMARK_ +#include + #include "comp_perf_options.h" +#include "comp_perf_test_common.h" +#include "comp_perf_test_verify.h" + +struct cperf_benchmark_ctx { + struct cperf_verify_ctx ver; + + /* Store TSC duration for all levels (including level 0) */ + uint64_t comp_tsc_duration[RTE_COMP_LEVEL_MAX + 1]; + uint64_t decomp_tsc_duration[RTE_COMP_LEVEL_MAX + 1]; + double comp_gbps; + double decomp_gbps; + double comp_tsc_byte; + double decomp_tsc_byte; +}; + +void +cperf_benchmark_test_destructor(void *arg); int -cperf_benchmark(struct comp_test_data *test_data, uint8_t level); +cperf_benchmark_test_runner(void *test_ctx); + +void * +cperf_benchmark_test_constructor(uint8_t dev_id, uint16_t qp_id, + struct comp_test_data *options); #endif diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index eba2795..1044532 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -9,6 +9,7 @@ #include "comp_perf_options.h" #include "comp_perf_test_verify.h" +#include "comp_perf_test_benchmark.h" #include "comp_perf.h" #include "comp_perf_test_common.h" diff --git a/app/test-compress-perf/meson.build b/app/test-compress-perf/meson.build index c6246e5..1136f04 100644 --- a/app/test-compress-perf/meson.build +++ b/app/test-compress-perf/meson.build @@ -5,5 +5,6 @@ allow_experimental_apis = true sources = files('comp_perf_options_parse.c', 'main.c', 'comp_perf_test_verify.c', + 'comp_perf_test_benchmark.c', 'comp_perf_test_common.c') deps = ['compressdev'] From patchwork Wed Jun 26 16:30:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Tomasz_J=C3=B3=C5=BAwiak?= X-Patchwork-Id: 55423 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 837ED2C57; Wed, 26 Jun 2019 18:31:04 +0200 (CEST) Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by dpdk.org (Postfix) with ESMTP id 14B073DC for ; Wed, 26 Jun 2019 18:30:51 +0200 (CEST) Received: by mail-wr1-f67.google.com with SMTP id f9so3438013wre.12 for ; Wed, 26 Jun 2019 09:30:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=kqB/PIfvDh3mRjU+l5+jeGf3zHlztbv0YgDNwlZ7VK4=; b=OnDrvptVKwb9gTofQP/KhW6I9sC/TbPCcWTMn7oT/vyfxdi5b0CRM22rT9urHXOaRe jVCD2rrKu4vxc8TcPsn6xJkTbyGtJ10MmKBiu2bYTSA6lTMgJcEHFZhNadAbZrYF795O 757Z2PkVR75JcU7B7cnMExm4kGekBd1+vP+dMET1h6JG9xxNETxSkLXOlomnV7v2vL1H nFhel763fkFhi9aqB4mRJ9rlP/pmEoYk4Gyk2frDU/d3jfHB1zA6dC0jMyhSz5RvaLBb 4eI1vXGDlp9PzRFgwTqBS7Ja0nuVBqv7MZP7c98xstC7mN72/oeyfdqwvj4pgMyTSAe3 ADiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=kqB/PIfvDh3mRjU+l5+jeGf3zHlztbv0YgDNwlZ7VK4=; b=iHg1N8t/iQDZWve5+dAOUYMg7lreP/bztaFxoJ4049GFqr5WDB4wIfSKHnCSvySQsf J6lTEt6xzoZKgAcake9VcU8x+fuVafDMX4+6oiQt+WSi4xdcOLw2rzI5CFiHofFoHR27 VhfSWoWa5ocY+OAcXiYF5SuQt8nCmDRDQuw2ZyeH76YjFqvxofbK33R5bFFUdJBqyn+1 5FM47wCc0sJj3VisuV0VF324f00SRjXev6X6JN1zf8/ooeCtNRm7L6bzRDMx7e24ughW SLTQtjbItAXxX/w/97gwxn0pfVnCTK5vW+LjsMnUYDbUCliEUGS+3pNHhuQ2b6BeUpJt Uafg== X-Gm-Message-State: APjAAAVLsgWJ7XKH8Gv0yXesQ927A7hn6sppYcHt7TdaCWVWxe9eosxM RVf4aMeiS7v6Kx8gr839iQ+5w5oNMIX0aw== X-Google-Smtp-Source: APXvYqzah0Wrmag8NnCoSJX3cAfvXnmrAEWmdnW7NNGTR2cV0apRBatR58InlIbzAPiGt5lC1RjEDQ== X-Received: by 2002:a5d:4a8d:: with SMTP id o13mr4189911wrq.350.1561566650646; Wed, 26 Jun 2019 09:30:50 -0700 (PDT) Received: from localhost.localdomain (178.43.130.103.ipv4.supernova.orange.pl. [178.43.130.103]) by smtp.gmail.com with ESMTPSA id n10sm16989663wrw.83.2019.06.26.09.30.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 26 Jun 2019 09:30:50 -0700 (PDT) From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm@gmail.com, shallyv@marvell.com, arturx.trybula@intel.com Date: Wed, 26 Jun 2019 18:30:31 +0200 Message-Id: <1561566633-12329-6-git-send-email-tjozwiakgm@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> References: <1560032553-15528-1-git-send-email-tjozwiakgm@gmail.com> <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> Subject: [dpdk-dev] [PATCH v3 5/7] doc: update dpdk-test-compress-perf description X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tomasz Jozwiak This patch updates a dpdk-test-compress-perf documentation. Signed-off-by: Tomasz Jozwiak --- doc/guides/tools/comp_perf.rst | 34 +++++++++++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 3 deletions(-) diff --git a/doc/guides/tools/comp_perf.rst b/doc/guides/tools/comp_perf.rst index 52869c1..71eef18 100644 --- a/doc/guides/tools/comp_perf.rst +++ b/doc/guides/tools/comp_perf.rst @@ -6,7 +6,9 @@ dpdk-test-compress-perf Tool The ``dpdk-test-compress-perf`` tool is a Data Plane Development Kit (DPDK) utility that allows measuring performance parameters of PMDs available in the -compress tree. The tool reads the data from a file (--input-file), +compress tree. User can use multiple cores to run tests on but only +one type of compression PMD can be measured during single application +execution. The tool reads the data from a file (--input-file), dumps all the file into a buffer and fills out the data of input mbufs, which are passed to compress device with compression operations. Then, the output buffers are fed into the decompression stage, and the resulting @@ -26,9 +28,35 @@ Limitations * Stateful operation is not supported in this version. +EAL Options +~~~~~~~~~~~ + +The following are the EAL command-line options that can be used in conjunction +with the ``dpdk-test-compress-perf`` application. +See the DPDK Getting Started Guides for more information on these options. + +* ``-c `` or ``-l `` + + Set the hexadecimal bitmask of the cores to run on. The corelist is a + list cores to use. + +.. Note:: + + One lcore is needed for process admin, tests are run on all other cores. + To run tests on two lcores, three lcores must be passed to the tool. + +* ``-w `` + + Add a PCI device in white list. + +* ``--vdev `` + + Add a virtual device. + +Appication Options +~~~~~~~~~~~~~~~~~~ -Command line options --------------------- + ``--ptest [benchmark/verify]``: set test type (default: benchmark) ``--driver-name NAME``: compress driver to use From patchwork Wed Jun 26 16:30:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Tomasz_J=C3=B3=C5=BAwiak?= X-Patchwork-Id: 55424 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 22E802E8F; Wed, 26 Jun 2019 18:31:06 +0200 (CEST) Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by dpdk.org (Postfix) with ESMTP id A87EB2BB5 for ; Wed, 26 Jun 2019 18:30:52 +0200 (CEST) Received: by mail-wm1-f67.google.com with SMTP id v19so2744640wmj.5 for ; Wed, 26 Jun 2019 09:30:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=vJmq5DaP1Wt8GCjaDZEZhPlf5UC13Z3m+XChLBtGZXk=; b=qNCCq9CuKXMM3VNSmQ06HZRe4RsrAVnFw6SXGGhjm460T5uAgiWu3FtpLPTl8XVoGT Yy2wKz1ifyPPYPB5oGTpluVlnnKYbJfOzC5Sg3Y6Z2aXLnAmmcVxsJMRx0DjnqVNTMwV QTr3XoNuMvlW3MSOQUuo08EivHV1vUmnHaa0iFkW/IRkG/7XC049Pen59ReIQUBAhooM 0YmNwlutyEPTw3GSb16B/vV5ObebNcrJAel8ppGJpwMN2g9IXdvXLoRu99B/02qYTSlJ oGV7/Brr+8qiIX1fTVwI5zJ1q5+Et0dwUeMbIQn78BXT/054VzU7BT287YyWhi26s0a9 fCeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=vJmq5DaP1Wt8GCjaDZEZhPlf5UC13Z3m+XChLBtGZXk=; b=N4xabFyC4n6I9BXh5lMmt8IM888xWz1VnCAWakKvMtyQXml/4UFUXXMufe5LzSg2mZ 3DL8g7mZGKifCVy5ri6/IyGWUTENWkBXfsmjnGOZFnM3Dbto75rrIJVkvvp7hLI7NEx7 kY9PH0ErBqPK/NznQIgZF6Jq/12xXngRhwKwqEvybWpmzSA694IlWB3wVCBuWLQkUUtx 1Q0pM4C6zDK79A/LUhPTjIhCOHiASX0tYDmWkP9JuSKQyH4c7Ta2M5b2O5mD9q/PcN9d shKk9x7bbz2OA5Yp/KOfl88bx0ZtxisqxJk93OoICdbXFOZ4hStFotDL0DWIZq0BRfdL 7A9w== X-Gm-Message-State: APjAAAVQlIMRbTevWbOijKOUp7ONoHH1oiS3y7k0AyZCSeInYmR2cDSn defxWMSOlXSOfI573zZEf0PwDOc/SbR9VA== X-Google-Smtp-Source: APXvYqwWNKhnbmlTenROTBr8OsA2sjVPfLQWb1cj6Cp6K6EdkcMho3IPWvEP4lO0WSrFZs4Om4e5EA== X-Received: by 2002:a1c:ab06:: with SMTP id u6mr3254104wme.125.1561566652141; Wed, 26 Jun 2019 09:30:52 -0700 (PDT) Received: from localhost.localdomain (178.43.130.103.ipv4.supernova.orange.pl. [178.43.130.103]) by smtp.gmail.com with ESMTPSA id n10sm16989663wrw.83.2019.06.26.09.30.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 26 Jun 2019 09:30:51 -0700 (PDT) From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm@gmail.com, shallyv@marvell.com, arturx.trybula@intel.com Date: Wed, 26 Jun 2019 18:30:32 +0200 Message-Id: <1561566633-12329-7-git-send-email-tjozwiakgm@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> References: <1560032553-15528-1-git-send-email-tjozwiakgm@gmail.com> <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> Subject: [dpdk-dev] [PATCH v3 6/7] app/test-compress-perf: add force process termination X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tomasz Jozwiak This patch adds a possibility to force controlled process termination as a result of two signals: SIGTERM and SIGINT Signed-off-by: Tomasz Jozwiak --- app/test-compress-perf/comp_perf_options.h | 1 + app/test-compress-perf/comp_perf_test_benchmark.c | 13 ++++++++++++ app/test-compress-perf/comp_perf_test_verify.c | 14 ++++++++++++ app/test-compress-perf/main.c | 26 +++++++++++++++++++++-- 4 files changed, 52 insertions(+), 2 deletions(-) diff --git a/app/test-compress-perf/comp_perf_options.h b/app/test-compress-perf/comp_perf_options.h index 79e63d5..534212d 100644 --- a/app/test-compress-perf/comp_perf_options.h +++ b/app/test-compress-perf/comp_perf_options.h @@ -68,6 +68,7 @@ struct comp_test_data { double ratio; enum cleanup_st cleanup; + int perf_comp_force_stop; }; int diff --git a/app/test-compress-perf/comp_perf_test_benchmark.c b/app/test-compress-perf/comp_perf_test_benchmark.c index 5f27bae..9effccc 100644 --- a/app/test-compress-perf/comp_perf_test_benchmark.c +++ b/app/test-compress-perf/comp_perf_test_benchmark.c @@ -183,6 +183,9 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) ops[op_id]->private_xform = priv_xform; } + if (unlikely(test_data->perf_comp_force_stop)) + goto end; + num_enq = rte_compressdev_enqueue_burst(dev_id, mem->qp_id, ops, num_ops); @@ -241,6 +244,9 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) /* Dequeue the last operations */ while (total_deq_ops < total_ops) { + if (unlikely(test_data->perf_comp_force_stop)) + goto end; + num_deq = rte_compressdev_dequeue_burst(dev_id, mem->qp_id, deq_ops, @@ -305,6 +311,13 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) rte_mempool_put_bulk(mem->op_pool, (void **)ops, allocated); rte_compressdev_private_xform_free(dev_id, priv_xform); rte_free(ops); + + if (test_data->perf_comp_force_stop) { + RTE_LOG(ERR, USER1, + "lcore: %d Perf. test has been aborted by user\n", + mem->lcore_id); + res = -1; + } return res; } diff --git a/app/test-compress-perf/comp_perf_test_verify.c b/app/test-compress-perf/comp_perf_test_verify.c index c2aab70..b2cd7a0 100644 --- a/app/test-compress-perf/comp_perf_test_verify.c +++ b/app/test-compress-perf/comp_perf_test_verify.c @@ -187,6 +187,9 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) ops[op_id]->private_xform = priv_xform; } + if (unlikely(test_data->perf_comp_force_stop)) + goto end; + num_enq = rte_compressdev_enqueue_burst(dev_id, mem->qp_id, ops, num_ops); @@ -267,6 +270,9 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) /* Dequeue the last operations */ while (total_deq_ops < total_ops) { + if (unlikely(test_data->perf_comp_force_stop)) + goto end; + num_deq = rte_compressdev_dequeue_burst(dev_id, mem->qp_id, deq_ops, @@ -345,6 +351,14 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) rte_mempool_put_bulk(mem->op_pool, (void **)ops, allocated); rte_compressdev_private_xform_free(dev_id, priv_xform); rte_free(ops); + + if (test_data->perf_comp_force_stop) { + RTE_LOG(ERR, USER1, + "lcore: %d Perf. test has been aborted by user\n", + mem->lcore_id); + res = -1; + } + return res; } diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index 1044532..77b279b 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -2,6 +2,10 @@ * Copyright(c) 2018 Intel Corporation */ +#include +#include +#include + #include #include #include @@ -42,6 +46,8 @@ static const struct cperf_test cperf_testmap[] = { } }; +static struct comp_test_data *test_data; + static int comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id) { @@ -283,12 +289,24 @@ comp_perf_dump_input_data(struct comp_test_data *test_data) return ret; } +static void +comp_perf_cleanup_on_signal(int signalNumber __rte_unused) +{ + test_data->perf_comp_force_stop = 1; +} + +static void +comp_perf_register_cleanup_on_signal(void) +{ + signal(SIGTERM, comp_perf_cleanup_on_signal); + signal(SIGINT, comp_perf_cleanup_on_signal); +} + int main(int argc, char **argv) { uint8_t level_idx = 0; int ret, i; - struct comp_test_data *test_data; void *ctx[RTE_MAX_LCORE] = {}; uint8_t enabled_cdevs[RTE_COMPRESS_MAX_DEVS]; int nb_compressdevs = 0; @@ -310,6 +328,8 @@ main(int argc, char **argv) rte_exit(EXIT_FAILURE, "Cannot reserve memory in socket %d\n", rte_socket_id()); + comp_perf_register_cleanup_on_signal(); + ret = EXIT_SUCCESS; test_data->cleanup = ST_TEST_DATA; comp_perf_options_default(test_data); @@ -430,8 +450,10 @@ main(int argc, char **argv) /* fallthrough */ case ST_COMPDEV: for (i = 0; i < nb_compressdevs && - i < RTE_COMPRESS_MAX_DEVS; i++) + i < RTE_COMPRESS_MAX_DEVS; i++) { rte_compressdev_stop(enabled_cdevs[i]); + rte_compressdev_close(enabled_cdevs[i]); + } /* fallthrough */ case ST_TEST_DATA: rte_free(test_data); From patchwork Wed Jun 26 16:30:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Tomasz_J=C3=B3=C5=BAwiak?= X-Patchwork-Id: 55425 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 29ED5326D; Wed, 26 Jun 2019 18:31:08 +0200 (CEST) Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by dpdk.org (Postfix) with ESMTP id 88C992BE2 for ; Wed, 26 Jun 2019 18:30:53 +0200 (CEST) Received: by mail-wm1-f67.google.com with SMTP id f17so2749159wme.2 for ; Wed, 26 Jun 2019 09:30:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=RGi8oR/dCFC4fozBcIL38NsQEWD/kvseX+NUKc7uU2U=; b=W1iWZLcxO7Txni/tBkdnLxEXzOg+I8bW32Rw9kSEPJUtsBbWmwOAjgGsh1XK+t8Sex qZdTI3TTTTUiYGLTa91E783SOGRiZE4bi4Z9iZC6rdL7F4pOiZGrHshfHBU11VIun9cF JirtR+GqCxNAqXCowK+OQ7AvhReMxVtD4Fqli3IxMpElVljKEqx39FFLDQzzm12nNv7D novkiFvjHhjt82Ctrvy06vFS7F73V+SPAxXIlJCPFcJNBOgSIQ19t+0CSH+5IzHRoEeP q1ib9rLVnpy+MSprwGLoNhb6xfj1kgkO5Uyou/BeSE3rgqNplZXz2cfsldIKbKZbO0gJ gBKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=RGi8oR/dCFC4fozBcIL38NsQEWD/kvseX+NUKc7uU2U=; b=G7aO3hmZEtynOjzEa7r5GHXywplUwCH6sIr36m86Qo6F9sDICs23tUTPw6aqIpf22x +08Z0GaRvXsbYQmFw7QV86y2EbqEY6Js7rJ6WbfwBCXfvTIA+NhWCr0yEQTvscH7qg+k eOFMu+OdhgAoG8hz1I0DtDE0+MlvfdSrbSfuhPT/iP32bxCw/wZgEwmxrqAPgHqWyGcI NuBFalWMCCYjJV+0Ff3WTzhaV8HrrBaubFEpz3gafOc2bKnjrRbnclXNKCvxsVJRjCef Imv9uZKAAKb7HZL6q5RTAYgNb4vcKINzcA3qOQQeTkHB0OEK8PnicoXtSnRGnDRU9Uu5 D7ZA== X-Gm-Message-State: APjAAAXk0LHAKoXSdX79fzS8sfaB3Xs4VKL0C1XdQa+GNbFNUlsZPBbi 5OUju9y1ghyl0/iuSlgvQ3yS1+43du0XAQ== X-Google-Smtp-Source: APXvYqyMxdYBPawwkqraHoTKXc4xxJx9My7SbLPc4ZA5CfEKNHbP0F4nfhutlVEzwy1TuiimYjoLNg== X-Received: by 2002:a1c:f314:: with SMTP id q20mr3060885wmq.74.1561566653118; Wed, 26 Jun 2019 09:30:53 -0700 (PDT) Received: from localhost.localdomain (178.43.130.103.ipv4.supernova.orange.pl. [178.43.130.103]) by smtp.gmail.com with ESMTPSA id n10sm16989663wrw.83.2019.06.26.09.30.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 26 Jun 2019 09:30:52 -0700 (PDT) From: Tomasz Jozwiak To: dev@dpdk.org, fiona.trahe@intel.com, tjozwiakgm@gmail.com, shallyv@marvell.com, arturx.trybula@intel.com Date: Wed, 26 Jun 2019 18:30:33 +0200 Message-Id: <1561566633-12329-8-git-send-email-tjozwiakgm@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> References: <1560032553-15528-1-git-send-email-tjozwiakgm@gmail.com> <1561566633-12329-1-git-send-email-tjozwiakgm@gmail.com> Subject: [dpdk-dev] [PATCH v3 7/7] doc: update release notes for 19.08 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tomasz Jozwiak Added release note entry for test-compress-perf application Signed-off-by: Tomasz Jozwiak --- doc/guides/rel_notes/release_19_08.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/doc/guides/rel_notes/release_19_08.rst b/doc/guides/rel_notes/release_19_08.rst index 7c0435a..2b2e029 100644 --- a/doc/guides/rel_notes/release_19_08.rst +++ b/doc/guides/rel_notes/release_19_08.rst @@ -99,6 +99,9 @@ New Features Updated ``librte_telemetry`` to fetch the global metrics from the ``librte_metrics`` library. +* **Updated test-compress-perf tool application.** + + Added multiple cores feature to compression perf tool application. Removed Items -------------