Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/120251/?format=api
{ "id": 120251, "url": "https://patches.dpdk.org/api/patches/120251/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/20221129082109.6809-5-syalavarthi@marvell.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20221129082109.6809-5-syalavarthi@marvell.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20221129082109.6809-5-syalavarthi@marvell.com", "date": "2022-11-29T08:21:02", "name": "[v2,05/12] app/mldev: add ordered inference test case", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "9071709c61341d7a5a52cf2540228f45329b0b9e", "submitter": { "id": 2480, "url": "https://patches.dpdk.org/api/people/2480/?format=api", "name": "Srikanth Yalavarthi", "email": "syalavarthi@marvell.com" }, "delegate": { "id": 1, "url": "https://patches.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "https://patches.dpdk.org/project/dpdk/patch/20221129082109.6809-5-syalavarthi@marvell.com/mbox/", "series": [ { "id": 25915, "url": "https://patches.dpdk.org/api/series/25915/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/list/?series=25915", "date": "2022-11-29T08:20:58", "name": "[v2,01/12] app/mldev: implement test framework for mldev", "version": 2, "mbox": "https://patches.dpdk.org/series/25915/mbox/" } ], "comments": "https://patches.dpdk.org/api/patches/120251/comments/", "check": "success", "checks": "https://patches.dpdk.org/api/patches/120251/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 3C2F1A0093;\n\tTue, 29 Nov 2022 09:21:50 +0100 (CET)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 7CD3542D38;\n\tTue, 29 Nov 2022 09:21:23 +0100 (CET)", "from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com\n [67.231.148.174])\n by mails.dpdk.org (Postfix) with ESMTP id 0AC6C42686\n for <dev@dpdk.org>; Tue, 29 Nov 2022 09:21:17 +0100 (CET)", "from pps.filterd (m0045849.ppops.net [127.0.0.1])\n by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id\n 2AT3NpLT005657 for <dev@dpdk.org>; Tue, 29 Nov 2022 00:21:17 -0800", "from dc5-exch02.marvell.com ([199.233.59.182])\n by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3m5a508yyp-4\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT)\n for <dev@dpdk.org>; Tue, 29 Nov 2022 00:21:17 -0800", "from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com\n (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18;\n Tue, 29 Nov 2022 00:21:15 -0800", "from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com\n (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend\n Transport; Tue, 29 Nov 2022 00:21:14 -0800", "from ml-host-33.caveonetworks.com (unknown [10.110.143.233])\n by maili.marvell.com (Postfix) with ESMTP id C60383F707C;\n Tue, 29 Nov 2022 00:21:14 -0800 (PST)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com;\n h=from : to : cc :\n subject : date : message-id : in-reply-to : references : mime-version :\n content-type; s=pfpt0220; bh=Q4P1rhVVd6ciEHyvGmukmqyfQ9ftG2wWL/8AjAnhEaM=;\n b=NKCM15AZhVNbVn3z9wZWzB1wWu+706QPLlKg87Lwr11ZPGemkUhlTlLg7tQ41j/ci1Pc\n 4lhkRzN9Sf58MOz4H3gPzV89mByq503onlAX6GYkSbHexF9si8PUmeOdV4Nv/3Hxt3+u\n +DPQoebbjhtehgZ7wjwq5SJympXWoBOWdVou6baVocMZ7xSLEkLrLaHMPhCA/+zSG90k\n 8iUh166c7sKaL4PVzjuwXSfoOgRhuoitmgxUZGsoRAjNZodZ53ymvxnbS1a5t9pcNNZM\n nkJXQp1B/HgL7APViEDyZhQleh38kGKSH1IZRADCgU/0zmqmsr7WjBwrj4VYgnFt8FHP EA==", "From": "Srikanth Yalavarthi <syalavarthi@marvell.com>", "To": "Srikanth Yalavarthi <syalavarthi@marvell.com>", "CC": "<dev@dpdk.org>, <sshankarnara@marvell.com>, <jerinj@marvell.com>", "Subject": "[PATCH v2 05/12] app/mldev: add ordered inference test case", "Date": "Tue, 29 Nov 2022 00:21:02 -0800", "Message-ID": "<20221129082109.6809-5-syalavarthi@marvell.com>", "X-Mailer": "git-send-email 2.17.1", "In-Reply-To": "<20221129082109.6809-1-syalavarthi@marvell.com>", "References": "<20221129070746.20396-2-syalavarthi@marvell.com>\n <20221129082109.6809-1-syalavarthi@marvell.com>", "MIME-Version": "1.0", "Content-Type": "text/plain", "X-Proofpoint-ORIG-GUID": "TJ_xMA0gGFp2t0d-mjAPIFrPD293wCV1", "X-Proofpoint-GUID": "TJ_xMA0gGFp2t0d-mjAPIFrPD293wCV1", "X-Proofpoint-Virus-Version": "vendor=baseguard\n engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1\n definitions=2022-11-29_06,2022-11-28_02,2022-06-22_01", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Added an ordered test case to execute inferences with single\nor multiple models. In this test case inference requests for\na model are enqueued after completion of all requests for\nthe previous model. Test supports inference repetitions.\n\nOperations sequence when testing with N models and R reps,\n\n(load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N\n\nTest case can be executed by selecting \"inference_ordered\" test\nand repetitions can be specified through \"--repetitions\" argument.\n\nSigned-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>\n---\n app/test-mldev/meson.build | 2 +\n app/test-mldev/ml_options.c | 73 ++-\n app/test-mldev/ml_options.h | 17 +-\n app/test-mldev/test_inference_common.c | 565 ++++++++++++++++++++++++\n app/test-mldev/test_inference_common.h | 65 +++\n app/test-mldev/test_inference_ordered.c | 119 +++++\n app/test-mldev/test_model_common.h | 10 +\n 7 files changed, 839 insertions(+), 12 deletions(-)\n create mode 100644 app/test-mldev/test_inference_common.c\n create mode 100644 app/test-mldev/test_inference_common.h\n create mode 100644 app/test-mldev/test_inference_ordered.c", "diff": "diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build\nindex b09e1ccc8a..475d76d126 100644\n--- a/app/test-mldev/meson.build\n+++ b/app/test-mldev/meson.build\n@@ -16,6 +16,8 @@ sources = files(\n 'test_device_ops.c',\n 'test_model_common.c',\n 'test_model_ops.c',\n+ 'test_inference_common.c',\n+ 'test_inference_ordered.c',\n )\n \n deps += ['mldev']\ndiff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c\nindex 15043c0992..10dad18fff 100644\n--- a/app/test-mldev/ml_options.c\n+++ b/app/test-mldev/ml_options.c\n@@ -29,6 +29,7 @@ ml_options_default(struct ml_options *opt)\n \topt->dev_id = 0;\n \topt->socket_id = SOCKET_ID_ANY;\n \topt->nb_filelist = 0;\n+\topt->repetitions = 1;\n \topt->debug = false;\n }\n \n@@ -96,6 +97,60 @@ ml_parse_models(struct ml_options *opt, const char *arg)\n \treturn ret;\n }\n \n+static int\n+ml_parse_filelist(struct ml_options *opt, const char *arg)\n+{\n+\tconst char *delim = \",\";\n+\tchar filelist[PATH_MAX];\n+\tchar *token;\n+\n+\tif (opt->nb_filelist >= ML_TEST_MAX_MODELS) {\n+\t\tml_err(\"Exceeded filelist count, max = %d\\n\", ML_TEST_MAX_MODELS);\n+\t\treturn -1;\n+\t}\n+\n+\tstrlcpy(filelist, arg, PATH_MAX);\n+\n+\t/* model */\n+\ttoken = strtok(filelist, delim);\n+\tif (token == NULL) {\n+\t\tml_err(\"Invalid filelist, model not specified = %s\\n\", arg);\n+\t\treturn -EINVAL;\n+\t}\n+\tstrlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);\n+\n+\t/* input */\n+\ttoken = strtok(NULL, delim);\n+\tif (token == NULL) {\n+\t\tml_err(\"Invalid filelist, input not specified = %s\\n\", arg);\n+\t\treturn -EINVAL;\n+\t}\n+\tstrlcpy(opt->filelist[opt->nb_filelist].input, token, PATH_MAX);\n+\n+\t/* output */\n+\ttoken = strtok(NULL, delim);\n+\tif (token == NULL) {\n+\t\tml_err(\"Invalid filelist, output not specified = %s\\n\", arg);\n+\t\treturn -EINVAL;\n+\t}\n+\tstrlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);\n+\n+\topt->nb_filelist++;\n+\n+\tif (opt->nb_filelist == 0) {\n+\t\tml_err(\"Empty filelist. Need at least one filelist entry for the test.\");\n+\t\treturn -EINVAL;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int\n+ml_parse_repetitions(struct ml_options *opt, const char *arg)\n+{\n+\treturn parser_read_uint64(&opt->repetitions, arg);\n+}\n+\n static void\n ml_dump_test_options(const char *testname)\n {\n@@ -106,6 +161,12 @@ ml_dump_test_options(const char *testname)\n \t\tprintf(\"\\t\\t--models : comma separated list of models\\n\");\n \t\tprintf(\"\\n\");\n \t}\n+\n+\tif (strcmp(testname, \"inference_ordered\") == 0) {\n+\t\tprintf(\"\\t\\t--filelist : comma separated list of model, input and output\\n\"\n+\t\t \"\\t\\t--repetitions : number of inference repetitions\\n\");\n+\t\tprintf(\"\\n\");\n+\t}\n }\n \n static void\n@@ -124,8 +185,9 @@ print_usage(char *program)\n }\n \n static struct option lgopts[] = {\n-\t{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},\n-\t{ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};\n+\t{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},\n+\t{ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},\n+\t{ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};\n \n static int\n ml_opts_parse_long(int opt_idx, struct ml_options *opt)\n@@ -133,10 +195,9 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)\n \tunsigned int i;\n \n \tstruct long_opt_parser parsermap[] = {\n-\t\t{ML_TEST, ml_parse_test_name},\n-\t\t{ML_DEVICE_ID, ml_parse_dev_id},\n-\t\t{ML_SOCKET_ID, ml_parse_socket_id},\n-\t\t{ML_MODELS, ml_parse_models},\n+\t\t{ML_TEST, ml_parse_test_name},\t {ML_DEVICE_ID, ml_parse_dev_id},\n+\t\t{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},\n+\t\t{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},\n \t};\n \n \tfor (i = 0; i < RTE_DIM(parsermap); i++) {\ndiff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h\nindex 8faf3b5deb..ad8aee5964 100644\n--- a/app/test-mldev/ml_options.h\n+++ b/app/test-mldev/ml_options.h\n@@ -13,15 +13,19 @@\n #define ML_TEST_MAX_MODELS 8\n \n /* Options names */\n-#define ML_TEST\t (\"test\")\n-#define ML_DEVICE_ID (\"dev_id\")\n-#define ML_SOCKET_ID (\"socket_id\")\n-#define ML_MODELS (\"models\")\n-#define ML_DEBUG (\"debug\")\n-#define ML_HELP\t (\"help\")\n+#define ML_TEST\t (\"test\")\n+#define ML_DEVICE_ID (\"dev_id\")\n+#define ML_SOCKET_ID (\"socket_id\")\n+#define ML_MODELS (\"models\")\n+#define ML_FILELIST (\"filelist\")\n+#define ML_REPETITIONS (\"repetitions\")\n+#define ML_DEBUG (\"debug\")\n+#define ML_HELP\t (\"help\")\n \n struct ml_filelist {\n \tchar model[PATH_MAX];\n+\tchar input[PATH_MAX];\n+\tchar output[PATH_MAX];\n };\n \n struct ml_options {\n@@ -30,6 +34,7 @@ struct ml_options {\n \tint socket_id;\n \tstruct ml_filelist filelist[ML_TEST_MAX_MODELS];\n \tuint8_t nb_filelist;\n+\tuint64_t repetitions;\n \tbool debug;\n };\n \ndiff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c\nnew file mode 100644\nindex 0000000000..8b5dc89346\n--- /dev/null\n+++ b/app/test-mldev/test_inference_common.c\n@@ -0,0 +1,565 @@\n+/* SPDX-License-Identifier: BSD-3-Clause\n+ * Copyright (c) 2022 Marvell.\n+ */\n+\n+#include <errno.h>\n+#include <stdbool.h>\n+#include <stdint.h>\n+#include <stdio.h>\n+#include <unistd.h>\n+\n+#include <rte_common.h>\n+#include <rte_launch.h>\n+#include <rte_lcore.h>\n+#include <rte_malloc.h>\n+#include <rte_memzone.h>\n+#include <rte_mldev.h>\n+\n+#include \"ml_common.h\"\n+#include \"ml_options.h\"\n+#include \"ml_test.h\"\n+#include \"test_common.h\"\n+#include \"test_inference_common.h\"\n+\n+/* Enqueue inference requests with burst size equal to 1 */\n+static int\n+ml_enqueue_single(void *arg)\n+{\n+\tstruct test_inference *t = ml_test_priv((struct ml_test *)arg);\n+\tstruct ml_request *req = NULL;\n+\tstruct rte_ml_op *op = NULL;\n+\tstruct ml_core_args *args;\n+\tuint64_t model_enq = 0;\n+\tuint32_t burst_enq;\n+\tuint32_t lcore_id;\n+\tint16_t fid;\n+\tint ret;\n+\n+\tlcore_id = rte_lcore_id();\n+\targs = &t->args[lcore_id];\n+\tmodel_enq = 0;\n+\n+\tif (args->nb_reqs == 0)\n+\t\treturn 0;\n+\n+next_rep:\n+\tfid = args->start_fid;\n+\n+next_model:\n+\tret = rte_mempool_get(t->op_pool, (void **)&op);\n+\tif (ret != 0)\n+\t\tgoto next_model;\n+\n+retry:\n+\tret = rte_mempool_get(t->model[fid].io_pool, (void **)&req);\n+\tif (ret != 0)\n+\t\tgoto retry;\n+\n+\top->model_id = t->model[fid].id;\n+\top->nb_batches = t->model[fid].info.batch_size;\n+\top->mempool = t->op_pool;\n+\n+\top->input.addr = req->input;\n+\top->input.length = t->model[fid].inp_qsize;\n+\top->input.next = NULL;\n+\n+\top->output.addr = req->output;\n+\top->output.length = t->model[fid].out_qsize;\n+\top->output.next = NULL;\n+\n+\top->user_ptr = req;\n+\treq->niters++;\n+\treq->fid = fid;\n+\n+enqueue_req:\n+\tburst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);\n+\tif (burst_enq == 0)\n+\t\tgoto enqueue_req;\n+\n+\tfid++;\n+\tif (likely(fid <= args->end_fid))\n+\t\tgoto next_model;\n+\n+\tmodel_enq++;\n+\tif (likely(model_enq < args->nb_reqs))\n+\t\tgoto next_rep;\n+\n+\treturn 0;\n+}\n+\n+/* Dequeue inference requests with burst size equal to 1 */\n+static int\n+ml_dequeue_single(void *arg)\n+{\n+\tstruct test_inference *t = ml_test_priv((struct ml_test *)arg);\n+\tstruct rte_ml_op_error error;\n+\tstruct rte_ml_op *op = NULL;\n+\tstruct ml_core_args *args;\n+\tstruct ml_request *req;\n+\tuint64_t total_deq = 0;\n+\tuint8_t nb_filelist;\n+\tuint32_t burst_deq;\n+\tuint32_t lcore_id;\n+\n+\tlcore_id = rte_lcore_id();\n+\targs = &t->args[lcore_id];\n+\tnb_filelist = args->end_fid - args->start_fid + 1;\n+\n+\tif (args->nb_reqs == 0)\n+\t\treturn 0;\n+\n+dequeue_req:\n+\tburst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);\n+\n+\tif (likely(burst_deq == 1)) {\n+\t\ttotal_deq += burst_deq;\n+\t\tif (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {\n+\t\t\trte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);\n+\t\t\tml_err(\"error_code = 0x%016lx, error_message = %s\\n\", error.errcode,\n+\t\t\t error.message);\n+\t\t}\n+\t\treq = (struct ml_request *)op->user_ptr;\n+\t\trte_mempool_put(t->model[req->fid].io_pool, req);\n+\t\trte_mempool_put(t->op_pool, op);\n+\t}\n+\n+\tif (likely(total_deq < args->nb_reqs * nb_filelist))\n+\t\tgoto dequeue_req;\n+\n+\treturn 0;\n+}\n+\n+bool\n+test_inference_cap_check(struct ml_options *opt)\n+{\n+\tstruct rte_ml_dev_info dev_info;\n+\n+\tif (!ml_test_cap_check(opt))\n+\t\treturn false;\n+\n+\trte_ml_dev_info_get(opt->dev_id, &dev_info);\n+\tif (opt->nb_filelist > dev_info.max_models) {\n+\t\tml_err(\"Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)\",\n+\t\t opt->nb_filelist, dev_info.max_models);\n+\t\treturn false;\n+\t}\n+\n+\treturn true;\n+}\n+\n+int\n+test_inference_opt_check(struct ml_options *opt)\n+{\n+\tuint32_t i;\n+\tint ret;\n+\n+\t/* check common opts */\n+\tret = ml_test_opt_check(opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+\t/* check file availability */\n+\tfor (i = 0; i < opt->nb_filelist; i++) {\n+\t\tif (access(opt->filelist[i].model, F_OK) == -1) {\n+\t\t\tml_err(\"Model file not accessible: id = %u, file = %s\", i,\n+\t\t\t opt->filelist[i].model);\n+\t\t\treturn -ENOENT;\n+\t\t}\n+\n+\t\tif (access(opt->filelist[i].input, F_OK) == -1) {\n+\t\t\tml_err(\"Input file not accessible: id = %u, file = %s\", i,\n+\t\t\t opt->filelist[i].input);\n+\t\t\treturn -ENOENT;\n+\t\t}\n+\t}\n+\n+\tif (opt->repetitions == 0) {\n+\t\tml_err(\"Invalid option, repetitions = %\" PRIu64 \"\\n\", opt->repetitions);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* check number of available lcores. */\n+\tif (rte_lcore_count() < 3) {\n+\t\tml_err(\"Insufficient lcores = %u\\n\", rte_lcore_count());\n+\t\tml_err(\"Minimum lcores required to create %u queue-pairs = %u\\n\", 1, 3);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+void\n+test_inference_opt_dump(struct ml_options *opt)\n+{\n+\tuint32_t i;\n+\n+\t/* dump common opts */\n+\tml_test_opt_dump(opt);\n+\n+\t/* dump test opts */\n+\tml_dump(\"repetitions\", \"%\" PRIu64, opt->repetitions);\n+\n+\tml_dump_begin(\"filelist\");\n+\tfor (i = 0; i < opt->nb_filelist; i++) {\n+\t\tml_dump_list(\"model\", i, opt->filelist[i].model);\n+\t\tml_dump_list(\"input\", i, opt->filelist[i].input);\n+\t\tml_dump_list(\"output\", i, opt->filelist[i].output);\n+\t}\n+\tml_dump_end;\n+}\n+\n+int\n+test_inference_setup(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t;\n+\tvoid *test_inference;\n+\tint ret = 0;\n+\tuint32_t i;\n+\n+\ttest_inference = rte_zmalloc_socket(test->name, sizeof(struct test_inference),\n+\t\t\t\t\t RTE_CACHE_LINE_SIZE, opt->socket_id);\n+\tif (test_inference == NULL) {\n+\t\tml_err(\"failed to allocate memory for test_model\");\n+\t\tret = -ENOMEM;\n+\t\tgoto error;\n+\t}\n+\ttest->test_priv = test_inference;\n+\tt = ml_test_priv(test);\n+\n+\tt->nb_used = 0;\n+\tt->cmn.result = ML_TEST_FAILED;\n+\tt->cmn.opt = opt;\n+\n+\t/* get device info */\n+\tret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);\n+\tif (ret < 0) {\n+\t\tml_err(\"failed to get device info\");\n+\t\tgoto error;\n+\t}\n+\n+\tt->enqueue = ml_enqueue_single;\n+\tt->dequeue = ml_dequeue_single;\n+\n+\t/* set model initial state */\n+\tfor (i = 0; i < opt->nb_filelist; i++)\n+\t\tt->model[i].state = MODEL_INITIAL;\n+\n+\treturn 0;\n+\n+error:\n+\tif (test_inference != NULL)\n+\t\trte_free(test_inference);\n+\n+\treturn ret;\n+}\n+\n+void\n+test_inference_destroy(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t;\n+\n+\tRTE_SET_USED(opt);\n+\n+\tt = ml_test_priv(test);\n+\tif (t != NULL)\n+\t\trte_free(t);\n+}\n+\n+int\n+ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct rte_ml_dev_qp_conf qp_conf;\n+\tstruct test_inference *t;\n+\tint ret;\n+\n+\tt = ml_test_priv(test);\n+\n+\tret = ml_test_device_configure(test, opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+\t/* setup queue pairs */\n+\tqp_conf.nb_desc = t->cmn.dev_info.max_desc;\n+\tqp_conf.cb = NULL;\n+\n+\tret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);\n+\tif (ret != 0) {\n+\t\tml_err(\"Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\\n\",\n+\t\t opt->dev_id, 0);\n+\t\tgoto error;\n+\t}\n+\n+\tret = ml_test_device_start(test, opt);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\treturn 0;\n+\n+error:\n+\tml_test_device_close(test, opt);\n+\n+\treturn ret;\n+}\n+\n+int\n+ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt)\n+{\n+\tint ret;\n+\n+\tret = ml_test_device_stop(test, opt);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\tret = ml_test_device_close(test, opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+\treturn 0;\n+\n+error:\n+\tml_test_device_close(test, opt);\n+\n+\treturn ret;\n+}\n+\n+/* Callback for IO pool create. This function would compute the fields of ml_request\n+ * structure and prepare the quantized input data.\n+ */\n+static void\n+ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)\n+{\n+\tstruct test_inference *t = ml_test_priv((struct ml_test *)opaque);\n+\tstruct ml_request *req = (struct ml_request *)obj;\n+\n+\tRTE_SET_USED(mp);\n+\tRTE_SET_USED(obj_idx);\n+\n+\treq->input = RTE_PTR_ADD(\n+\t\tobj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));\n+\treq->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,\n+\t\t\t\t\t\t\t t->cmn.dev_info.min_align_size));\n+\treq->niters = 0;\n+\n+\t/* quantize data */\n+\trte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,\n+\t\t\t t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);\n+}\n+\n+int\n+ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t fid)\n+{\n+\tstruct test_inference *t = ml_test_priv(test);\n+\tchar mz_name[RTE_MEMZONE_NAMESIZE];\n+\tchar mp_name[RTE_MEMPOOL_NAMESIZE];\n+\tconst struct rte_memzone *mz;\n+\tuint64_t nb_buffers;\n+\tuint32_t buff_size;\n+\tuint32_t mz_size;\n+\tuint32_t fsize;\n+\tFILE *fp;\n+\tint ret;\n+\n+\t/* get input buffer size */\n+\tret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,\n+\t\t\t\t &t->model[fid].inp_qsize, &t->model[fid].inp_dsize);\n+\tif (ret != 0) {\n+\t\tml_err(\"Failed to get input size, model : %s\\n\", opt->filelist[fid].model);\n+\t\treturn ret;\n+\t}\n+\n+\t/* get output buffer size */\n+\tret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,\n+\t\t\t\t\tt->model[fid].info.batch_size, &t->model[fid].out_qsize,\n+\t\t\t\t\t&t->model[fid].out_dsize);\n+\tif (ret != 0) {\n+\t\tml_err(\"Failed to get input size, model : %s\\n\", opt->filelist[fid].model);\n+\t\treturn ret;\n+\t}\n+\n+\t/* allocate buffer for user data */\n+\tmz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;\n+\tsprintf(mz_name, \"ml_user_data_%d\", fid);\n+\tmz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);\n+\tif (mz == NULL) {\n+\t\tml_err(\"Memzone allocation failed for ml_user_data\\n\");\n+\t\tret = -ENOMEM;\n+\t\tgoto error;\n+\t}\n+\n+\tt->model[fid].input = mz->addr;\n+\tt->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);\n+\n+\t/* load input file */\n+\tfp = fopen(opt->filelist[fid].input, \"r\");\n+\tif (fp == NULL) {\n+\t\tml_err(\"Failed to open input file : %s\\n\", opt->filelist[fid].input);\n+\t\tret = -errno;\n+\t\tgoto error;\n+\t}\n+\n+\tfseek(fp, 0, SEEK_END);\n+\tfsize = ftell(fp);\n+\tfseek(fp, 0, SEEK_SET);\n+\tif (fsize != t->model[fid].inp_dsize) {\n+\t\tml_err(\"Invalid input file, size = %u (expected size = %\" PRIu64 \")\\n\", fsize,\n+\t\t t->model[fid].inp_dsize);\n+\t\tret = -EINVAL;\n+\t\tfclose(fp);\n+\t\tgoto error;\n+\t}\n+\n+\tif (fread(t->model[fid].input, 1, t->model[fid].inp_dsize, fp) != t->model[fid].inp_dsize) {\n+\t\tml_err(\"Failed to read input file : %s\\n\", opt->filelist[fid].input);\n+\t\tret = -errno;\n+\t\tfclose(fp);\n+\t\tgoto error;\n+\t}\n+\tfclose(fp);\n+\n+\t/* create mempool for quantized input and output buffers. ml_request_initialize is\n+\t * used as a callback for object creation.\n+\t */\n+\tbuff_size = RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size) +\n+\t\t RTE_ALIGN_CEIL(t->model[fid].inp_qsize, t->cmn.dev_info.min_align_size) +\n+\t\t RTE_ALIGN_CEIL(t->model[fid].out_qsize, t->cmn.dev_info.min_align_size);\n+\tnb_buffers = RTE_MIN((uint64_t)ML_TEST_MAX_POOL_SIZE, opt->repetitions);\n+\n+\tt->fid = fid;\n+\tsprintf(mp_name, \"ml_io_pool_%d\", fid);\n+\tt->model[fid].io_pool = rte_mempool_create(mp_name, nb_buffers, buff_size, 0, 0, NULL, NULL,\n+\t\t\t\t\t\t ml_request_initialize, test, opt->socket_id, 0);\n+\tif (t->model[fid].io_pool == NULL) {\n+\t\tml_err(\"Failed to create io pool : %s\\n\", \"ml_io_pool\");\n+\t\tret = -ENOMEM;\n+\t\tgoto error;\n+\t}\n+\n+\treturn 0;\n+\n+error:\n+\tif (mz != NULL)\n+\t\trte_memzone_free(mz);\n+\n+\tif (t->model[fid].io_pool != NULL) {\n+\t\trte_mempool_free(t->model[fid].io_pool);\n+\t\tt->model[fid].io_pool = NULL;\n+\t}\n+\n+\treturn ret;\n+}\n+\n+void\n+ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, int16_t fid)\n+{\n+\tchar mz_name[RTE_MEMZONE_NAMESIZE];\n+\tchar mp_name[RTE_MEMPOOL_NAMESIZE];\n+\tconst struct rte_memzone *mz;\n+\tstruct rte_mempool *mp;\n+\n+\tRTE_SET_USED(test);\n+\tRTE_SET_USED(opt);\n+\n+\t/* release user data memzone */\n+\tsprintf(mz_name, \"ml_user_data_%d\", fid);\n+\tmz = rte_memzone_lookup(mz_name);\n+\tif (mz != NULL)\n+\t\trte_memzone_free(mz);\n+\n+\t/* destroy io pool */\n+\tsprintf(mp_name, \"ml_io_pool_%d\", fid);\n+\tmp = rte_mempool_lookup(mp_name);\n+\tif (mp != NULL)\n+\t\trte_mempool_free(mp);\n+}\n+\n+int\n+ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t = ml_test_priv(test);\n+\n+\t/* create op pool */\n+\tt->op_pool = rte_ml_op_pool_create(\"ml_test_op_pool\", ML_TEST_MAX_POOL_SIZE, 0, 0,\n+\t\t\t\t\t opt->socket_id);\n+\tif (t->op_pool == NULL) {\n+\t\tml_err(\"Failed to create op pool : %s\\n\", \"ml_op_pool\");\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+void\n+ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t = ml_test_priv(test);\n+\n+\tRTE_SET_USED(opt);\n+\n+\t/* release op pool */\n+\tif (t->op_pool != NULL)\n+\t\trte_mempool_free(t->op_pool);\n+}\n+\n+/* Callback for mempool object iteration. This call would dequantize output data. */\n+static void\n+ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)\n+{\n+\tstruct test_inference *t = ml_test_priv((struct ml_test *)opaque);\n+\tstruct ml_request *req = (struct ml_request *)obj;\n+\tstruct ml_model *model = &t->model[req->fid];\n+\n+\tRTE_SET_USED(mp);\n+\tRTE_SET_USED(obj_idx);\n+\n+\tif (req->niters == 0)\n+\t\treturn;\n+\n+\tt->nb_used++;\n+\trte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,\n+\t\t\t req->output, model->output);\n+}\n+\n+int\n+ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid)\n+{\n+\tstruct test_inference *t = ml_test_priv(test);\n+\n+\tRTE_SET_USED(opt);\n+\n+\trte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);\n+\n+\tif (t->nb_used > 0)\n+\t\tt->cmn.result = ML_TEST_SUCCESS;\n+\telse\n+\t\tt->cmn.result = ML_TEST_FAILED;\n+\n+\treturn t->cmn.result;\n+}\n+\n+int\n+ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,\n+\t\t\t int16_t end_fid)\n+{\n+\tstruct test_inference *t = ml_test_priv(test);\n+\tuint32_t lcore_id;\n+\tuint32_t id = 0;\n+\n+\tRTE_LCORE_FOREACH_WORKER(lcore_id)\n+\t{\n+\t\tif (id == 2)\n+\t\t\tbreak;\n+\n+\t\tt->args[lcore_id].nb_reqs = opt->repetitions;\n+\t\tt->args[lcore_id].start_fid = start_fid;\n+\t\tt->args[lcore_id].end_fid = end_fid;\n+\n+\t\tif (id % 2 == 0)\n+\t\t\trte_eal_remote_launch(t->enqueue, test, lcore_id);\n+\t\telse\n+\t\t\trte_eal_remote_launch(t->dequeue, test, lcore_id);\n+\n+\t\tid++;\n+\t}\n+\n+\treturn 0;\n+}\ndiff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h\nnew file mode 100644\nindex 0000000000..91007954b4\n--- /dev/null\n+++ b/app/test-mldev/test_inference_common.h\n@@ -0,0 +1,65 @@\n+/* SPDX-License-Identifier: BSD-3-Clause\n+ * Copyright (c) 2022 Marvell.\n+ */\n+\n+#ifndef _ML_TEST_INFERENCE_COMMON_\n+#define _ML_TEST_INFERENCE_COMMON_\n+\n+#include <stdbool.h>\n+#include <stdint.h>\n+\n+#include <rte_common.h>\n+#include <rte_mempool.h>\n+\n+#include \"ml_options.h\"\n+#include \"ml_test.h\"\n+#include \"test_common.h\"\n+#include \"test_model_common.h\"\n+\n+struct ml_request {\n+\tvoid *input;\n+\tvoid *output;\n+\tint16_t fid;\n+\tuint64_t niters;\n+};\n+\n+struct ml_core_args {\n+\tuint64_t nb_reqs;\n+\tint16_t start_fid;\n+\tint16_t end_fid;\n+};\n+\n+struct test_inference {\n+\t/* common data */\n+\tstruct test_common cmn;\n+\n+\t/* test specific data */\n+\tstruct ml_model model[ML_TEST_MAX_MODELS];\n+\tstruct rte_mempool *op_pool;\n+\n+\tuint64_t nb_used;\n+\tint16_t fid;\n+\n+\tint (*enqueue)(void *arg);\n+\tint (*dequeue)(void *arg);\n+\n+\tstruct ml_core_args args[RTE_MAX_LCORE];\n+} __rte_cache_aligned;\n+\n+bool test_inference_cap_check(struct ml_options *opt);\n+int test_inference_opt_check(struct ml_options *opt);\n+void test_inference_opt_dump(struct ml_options *opt);\n+int test_inference_setup(struct ml_test *test, struct ml_options *opt);\n+void test_inference_destroy(struct ml_test *test, struct ml_options *opt);\n+\n+int ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt);\n+int ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt);\n+int ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t fid);\n+void ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, int16_t fid);\n+int ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt);\n+void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);\n+int ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid);\n+int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,\n+\t\t\t int16_t end_fid);\n+\n+#endif /* _ML_TEST_INFERENCE_COMMON_ */\ndiff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c\nnew file mode 100644\nindex 0000000000..84e6bf9109\n--- /dev/null\n+++ b/app/test-mldev/test_inference_ordered.c\n@@ -0,0 +1,119 @@\n+/* SPDX-License-Identifier: BSD-3-Clause\n+ * Copyright (c) 2022 Marvell.\n+ */\n+\n+#include <stdint.h>\n+\n+#include <rte_common.h>\n+#include <rte_launch.h>\n+\n+#include \"ml_common.h\"\n+#include \"ml_test.h\"\n+#include \"test_inference_common.h\"\n+#include \"test_model_common.h\"\n+\n+static int\n+test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t;\n+\tint16_t fid = 0;\n+\tint ret = 0;\n+\n+\tt = ml_test_priv(test);\n+\n+\tret = ml_inference_mldev_setup(test, opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+\tret = ml_inference_mem_setup(test, opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+next_model:\n+\t/* load model */\n+\tret = ml_model_load(test, opt, &t->model[fid], fid);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\t/* start model */\n+\tret = ml_model_start(test, opt, &t->model[fid], fid);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\tret = ml_inference_iomem_setup(test, opt, fid);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\t/* launch inferences for one model using available queue pairs */\n+\tret = ml_inference_launch_cores(test, opt, fid, fid);\n+\tif (ret != 0) {\n+\t\tml_err(\"failed to launch cores\");\n+\t\tgoto error;\n+\t}\n+\n+\trte_eal_mp_wait_lcore();\n+\n+\tret = ml_inference_result(test, opt, fid);\n+\tif (ret != ML_TEST_SUCCESS)\n+\t\tgoto error;\n+\n+\tml_inference_iomem_destroy(test, opt, fid);\n+\n+\t/* stop model */\n+\tret = ml_model_stop(test, opt, &t->model[fid], fid);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\t/* unload model */\n+\tret = ml_model_unload(test, opt, &t->model[fid], fid);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\tfid++;\n+\tif (fid < opt->nb_filelist)\n+\t\tgoto next_model;\n+\n+\tml_inference_mem_destroy(test, opt);\n+\n+\tret = ml_inference_mldev_destroy(test, opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+\tt->cmn.result = ML_TEST_SUCCESS;\n+\n+\treturn 0;\n+\n+error:\n+\tml_inference_iomem_destroy(test, opt, fid);\n+\tml_inference_mem_destroy(test, opt);\n+\tml_model_stop(test, opt, &t->model[fid], fid);\n+\tml_model_unload(test, opt, &t->model[fid], fid);\n+\n+\tt->cmn.result = ML_TEST_FAILED;\n+\n+\treturn ret;\n+}\n+\n+static int\n+test_inference_ordered_result(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t;\n+\n+\tRTE_SET_USED(opt);\n+\n+\tt = ml_test_priv(test);\n+\n+\treturn t->cmn.result;\n+}\n+\n+static const struct ml_test_ops inference_ordered = {\n+\t.cap_check = test_inference_cap_check,\n+\t.opt_check = test_inference_opt_check,\n+\t.opt_dump = test_inference_opt_dump,\n+\t.test_setup = test_inference_setup,\n+\t.test_destroy = test_inference_destroy,\n+\t.test_driver = test_inference_ordered_driver,\n+\t.test_result = test_inference_ordered_result,\n+};\n+\n+ML_TEST_REGISTER(inference_ordered);\ndiff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h\nindex 302e4eb45f..c45ae80853 100644\n--- a/app/test-mldev/test_model_common.h\n+++ b/app/test-mldev/test_model_common.h\n@@ -23,6 +23,16 @@ struct ml_model {\n \tint16_t id;\n \tstruct rte_ml_model_info info;\n \tenum model_state state;\n+\n+\tuint64_t inp_dsize;\n+\tuint64_t inp_qsize;\n+\tuint64_t out_dsize;\n+\tuint64_t out_qsize;\n+\n+\tuint8_t *input;\n+\tuint8_t *output;\n+\n+\tstruct rte_mempool *io_pool;\n };\n \n int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,\n", "prefixes": [ "v2", "05/12" ] }