Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/123305/?format=api
{ "id": 123305, "url": "https://patches.dpdk.org/api/patches/123305/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/20230207154943.18779-6-syalavarthi@marvell.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20230207154943.18779-6-syalavarthi@marvell.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20230207154943.18779-6-syalavarthi@marvell.com", "date": "2023-02-07T15:49:36", "name": "[v4,05/12] app/mldev: add ordered inference test case", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "b0f92ed60e576f6a813a6c7e97fdf57c583d234c", "submitter": { "id": 2480, "url": "https://patches.dpdk.org/api/people/2480/?format=api", "name": "Srikanth Yalavarthi", "email": "syalavarthi@marvell.com" }, "delegate": { "id": 1, "url": "https://patches.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "https://patches.dpdk.org/project/dpdk/patch/20230207154943.18779-6-syalavarthi@marvell.com/mbox/", "series": [ { "id": 26858, "url": "https://patches.dpdk.org/api/series/26858/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/list/?series=26858", "date": "2023-02-07T15:49:31", "name": "Implementation of mldev test application", "version": 4, "mbox": "https://patches.dpdk.org/series/26858/mbox/" } ], "comments": "https://patches.dpdk.org/api/patches/123305/comments/", "check": "warning", "checks": "https://patches.dpdk.org/api/patches/123305/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 08E4E41C30;\n\tTue, 7 Feb 2023 16:50:31 +0100 (CET)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 5F36E42D32;\n\tTue, 7 Feb 2023 16:49:57 +0100 (CET)", "from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com\n [67.231.148.174])\n by mails.dpdk.org (Postfix) with ESMTP id D171542B71\n for <dev@dpdk.org>; Tue, 7 Feb 2023 16:49:50 +0100 (CET)", "from pps.filterd (m0045849.ppops.net [127.0.0.1])\n by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id\n 317EYvZx001487 for <dev@dpdk.org>; Tue, 7 Feb 2023 07:49:50 -0800", "from dc5-exch01.marvell.com ([199.233.59.181])\n by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3nkdyrspn2-3\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT)\n for <dev@dpdk.org>; Tue, 07 Feb 2023 07:49:49 -0800", "from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com\n (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.42;\n Tue, 7 Feb 2023 07:49:48 -0800", "from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com\n (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend\n Transport; Tue, 7 Feb 2023 07:49:48 -0800", "from ml-host-33.caveonetworks.com (unknown [10.110.143.233])\n by maili.marvell.com (Postfix) with ESMTP id 26F9C3F7083;\n Tue, 7 Feb 2023 07:49:48 -0800 (PST)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com;\n h=from : to : cc :\n subject : date : message-id : in-reply-to : references : mime-version :\n content-type; s=pfpt0220; bh=ZodCdpVtfYhJ9Jr88ZYkV1o1tfNYl8STORgoQVfwIug=;\n b=dwD2H8inU7NsxKiSuPwKfddVJrrzkbqmkUIkKvi4Vcc8UF+U65uDKQZ4uSPJCjlrJmLw\n 7nu/GHXZM7cuwMwadUvY8MH1dGUU+nk967CiaoipJwKmvo+F5QXLg6ebuxyDTjYX8nR0\n J88izKGXCNltYipfZb7b/ay8TrudKypsI1ng20rSxqyZjcS2aEevVByd0nIZOgacms//\n 3vDLtMuNeNyh3bCSFSKKYAD3U/AW/Vh2UL0g9bL6fJAufcf9CWfDUVNe1Pt1hisTc7PL\n 1fQ+MNdY15yJN5ACEUtCC8g5MFqVxc6srNuoxttgtO+k+DZqyoZGcSVzBgwkNKQ18aK1 lQ==", "From": "Srikanth Yalavarthi <syalavarthi@marvell.com>", "To": "Srikanth Yalavarthi <syalavarthi@marvell.com>", "CC": "<dev@dpdk.org>, <sshankarnara@marvell.com>, <jerinj@marvell.com>,\n <aprabhu@marvell.com>, <ptakkar@marvell.com>, <pshukla@marvell.com>", "Subject": "[PATCH v4 05/12] app/mldev: add ordered inference test case", "Date": "Tue, 7 Feb 2023 07:49:36 -0800", "Message-ID": "<20230207154943.18779-6-syalavarthi@marvell.com>", "X-Mailer": "git-send-email 2.17.1", "In-Reply-To": "<20230207154943.18779-1-syalavarthi@marvell.com>", "References": "<20221129070746.20396-1-syalavarthi@marvell.com>\n <20230207154943.18779-1-syalavarthi@marvell.com>", "MIME-Version": "1.0", "Content-Type": "text/plain", "X-Proofpoint-ORIG-GUID": "P8PG8Pn38QJJA8wYjaVj4xnpBCheZGMr", "X-Proofpoint-GUID": "P8PG8Pn38QJJA8wYjaVj4xnpBCheZGMr", "X-Proofpoint-Virus-Version": "vendor=baseguard\n engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1\n definitions=2023-02-07_07,2023-02-06_03,2022-06-22_01", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Added an ordered test case to execute inferences with single\nor multiple models. In this test case inference requests for\na model are enqueued after completion of all requests for\nthe previous model. Test supports inference repetitions.\n\nOperations sequence when testing with N models and R reps,\n\n(load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N\n\nTest case can be executed by selecting \"inference_ordered\" test\nand repetitions can be specified through \"--repetitions\" argument.\n\nSigned-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>\n---\n app/test-mldev/meson.build | 2 +\n app/test-mldev/ml_options.c | 73 ++-\n app/test-mldev/ml_options.h | 17 +-\n app/test-mldev/test_inference_common.c | 565 ++++++++++++++++++++++++\n app/test-mldev/test_inference_common.h | 65 +++\n app/test-mldev/test_inference_ordered.c | 119 +++++\n app/test-mldev/test_model_common.h | 10 +\n 7 files changed, 839 insertions(+), 12 deletions(-)\n create mode 100644 app/test-mldev/test_inference_common.c\n create mode 100644 app/test-mldev/test_inference_common.h\n create mode 100644 app/test-mldev/test_inference_ordered.c", "diff": "diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build\nindex b09e1ccc8a..475d76d126 100644\n--- a/app/test-mldev/meson.build\n+++ b/app/test-mldev/meson.build\n@@ -16,6 +16,8 @@ sources = files(\n 'test_device_ops.c',\n 'test_model_common.c',\n 'test_model_ops.c',\n+ 'test_inference_common.c',\n+ 'test_inference_ordered.c',\n )\n \n deps += ['mldev']\ndiff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c\nindex 15043c0992..10dad18fff 100644\n--- a/app/test-mldev/ml_options.c\n+++ b/app/test-mldev/ml_options.c\n@@ -29,6 +29,7 @@ ml_options_default(struct ml_options *opt)\n \topt->dev_id = 0;\n \topt->socket_id = SOCKET_ID_ANY;\n \topt->nb_filelist = 0;\n+\topt->repetitions = 1;\n \topt->debug = false;\n }\n \n@@ -96,6 +97,60 @@ ml_parse_models(struct ml_options *opt, const char *arg)\n \treturn ret;\n }\n \n+static int\n+ml_parse_filelist(struct ml_options *opt, const char *arg)\n+{\n+\tconst char *delim = \",\";\n+\tchar filelist[PATH_MAX];\n+\tchar *token;\n+\n+\tif (opt->nb_filelist >= ML_TEST_MAX_MODELS) {\n+\t\tml_err(\"Exceeded filelist count, max = %d\\n\", ML_TEST_MAX_MODELS);\n+\t\treturn -1;\n+\t}\n+\n+\tstrlcpy(filelist, arg, PATH_MAX);\n+\n+\t/* model */\n+\ttoken = strtok(filelist, delim);\n+\tif (token == NULL) {\n+\t\tml_err(\"Invalid filelist, model not specified = %s\\n\", arg);\n+\t\treturn -EINVAL;\n+\t}\n+\tstrlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);\n+\n+\t/* input */\n+\ttoken = strtok(NULL, delim);\n+\tif (token == NULL) {\n+\t\tml_err(\"Invalid filelist, input not specified = %s\\n\", arg);\n+\t\treturn -EINVAL;\n+\t}\n+\tstrlcpy(opt->filelist[opt->nb_filelist].input, token, PATH_MAX);\n+\n+\t/* output */\n+\ttoken = strtok(NULL, delim);\n+\tif (token == NULL) {\n+\t\tml_err(\"Invalid filelist, output not specified = %s\\n\", arg);\n+\t\treturn -EINVAL;\n+\t}\n+\tstrlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);\n+\n+\topt->nb_filelist++;\n+\n+\tif (opt->nb_filelist == 0) {\n+\t\tml_err(\"Empty filelist. Need at least one filelist entry for the test.\");\n+\t\treturn -EINVAL;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int\n+ml_parse_repetitions(struct ml_options *opt, const char *arg)\n+{\n+\treturn parser_read_uint64(&opt->repetitions, arg);\n+}\n+\n static void\n ml_dump_test_options(const char *testname)\n {\n@@ -106,6 +161,12 @@ ml_dump_test_options(const char *testname)\n \t\tprintf(\"\\t\\t--models : comma separated list of models\\n\");\n \t\tprintf(\"\\n\");\n \t}\n+\n+\tif (strcmp(testname, \"inference_ordered\") == 0) {\n+\t\tprintf(\"\\t\\t--filelist : comma separated list of model, input and output\\n\"\n+\t\t \"\\t\\t--repetitions : number of inference repetitions\\n\");\n+\t\tprintf(\"\\n\");\n+\t}\n }\n \n static void\n@@ -124,8 +185,9 @@ print_usage(char *program)\n }\n \n static struct option lgopts[] = {\n-\t{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},\n-\t{ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};\n+\t{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},\n+\t{ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},\n+\t{ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};\n \n static int\n ml_opts_parse_long(int opt_idx, struct ml_options *opt)\n@@ -133,10 +195,9 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)\n \tunsigned int i;\n \n \tstruct long_opt_parser parsermap[] = {\n-\t\t{ML_TEST, ml_parse_test_name},\n-\t\t{ML_DEVICE_ID, ml_parse_dev_id},\n-\t\t{ML_SOCKET_ID, ml_parse_socket_id},\n-\t\t{ML_MODELS, ml_parse_models},\n+\t\t{ML_TEST, ml_parse_test_name},\t {ML_DEVICE_ID, ml_parse_dev_id},\n+\t\t{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},\n+\t\t{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},\n \t};\n \n \tfor (i = 0; i < RTE_DIM(parsermap); i++) {\ndiff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h\nindex 8faf3b5deb..ad8aee5964 100644\n--- a/app/test-mldev/ml_options.h\n+++ b/app/test-mldev/ml_options.h\n@@ -13,15 +13,19 @@\n #define ML_TEST_MAX_MODELS 8\n \n /* Options names */\n-#define ML_TEST\t (\"test\")\n-#define ML_DEVICE_ID (\"dev_id\")\n-#define ML_SOCKET_ID (\"socket_id\")\n-#define ML_MODELS (\"models\")\n-#define ML_DEBUG (\"debug\")\n-#define ML_HELP\t (\"help\")\n+#define ML_TEST\t (\"test\")\n+#define ML_DEVICE_ID (\"dev_id\")\n+#define ML_SOCKET_ID (\"socket_id\")\n+#define ML_MODELS (\"models\")\n+#define ML_FILELIST (\"filelist\")\n+#define ML_REPETITIONS (\"repetitions\")\n+#define ML_DEBUG (\"debug\")\n+#define ML_HELP\t (\"help\")\n \n struct ml_filelist {\n \tchar model[PATH_MAX];\n+\tchar input[PATH_MAX];\n+\tchar output[PATH_MAX];\n };\n \n struct ml_options {\n@@ -30,6 +34,7 @@ struct ml_options {\n \tint socket_id;\n \tstruct ml_filelist filelist[ML_TEST_MAX_MODELS];\n \tuint8_t nb_filelist;\n+\tuint64_t repetitions;\n \tbool debug;\n };\n \ndiff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c\nnew file mode 100644\nindex 0000000000..ff25c056a0\n--- /dev/null\n+++ b/app/test-mldev/test_inference_common.c\n@@ -0,0 +1,565 @@\n+/* SPDX-License-Identifier: BSD-3-Clause\n+ * Copyright (c) 2022 Marvell.\n+ */\n+\n+#include <errno.h>\n+#include <stdbool.h>\n+#include <stdint.h>\n+#include <stdio.h>\n+#include <unistd.h>\n+\n+#include <rte_common.h>\n+#include <rte_launch.h>\n+#include <rte_lcore.h>\n+#include <rte_malloc.h>\n+#include <rte_memzone.h>\n+#include <rte_mldev.h>\n+\n+#include \"ml_common.h\"\n+#include \"ml_options.h\"\n+#include \"ml_test.h\"\n+#include \"test_common.h\"\n+#include \"test_inference_common.h\"\n+\n+/* Enqueue inference requests with burst size equal to 1 */\n+static int\n+ml_enqueue_single(void *arg)\n+{\n+\tstruct test_inference *t = ml_test_priv((struct ml_test *)arg);\n+\tstruct ml_request *req = NULL;\n+\tstruct rte_ml_op *op = NULL;\n+\tstruct ml_core_args *args;\n+\tuint64_t model_enq = 0;\n+\tuint32_t burst_enq;\n+\tuint32_t lcore_id;\n+\tuint16_t fid;\n+\tint ret;\n+\n+\tlcore_id = rte_lcore_id();\n+\targs = &t->args[lcore_id];\n+\tmodel_enq = 0;\n+\n+\tif (args->nb_reqs == 0)\n+\t\treturn 0;\n+\n+next_rep:\n+\tfid = args->start_fid;\n+\n+next_model:\n+\tret = rte_mempool_get(t->op_pool, (void **)&op);\n+\tif (ret != 0)\n+\t\tgoto next_model;\n+\n+retry:\n+\tret = rte_mempool_get(t->model[fid].io_pool, (void **)&req);\n+\tif (ret != 0)\n+\t\tgoto retry;\n+\n+\top->model_id = t->model[fid].id;\n+\top->nb_batches = t->model[fid].info.batch_size;\n+\top->mempool = t->op_pool;\n+\n+\top->input.addr = req->input;\n+\top->input.length = t->model[fid].inp_qsize;\n+\top->input.next = NULL;\n+\n+\top->output.addr = req->output;\n+\top->output.length = t->model[fid].out_qsize;\n+\top->output.next = NULL;\n+\n+\top->user_ptr = req;\n+\treq->niters++;\n+\treq->fid = fid;\n+\n+enqueue_req:\n+\tburst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);\n+\tif (burst_enq == 0)\n+\t\tgoto enqueue_req;\n+\n+\tfid++;\n+\tif (likely(fid <= args->end_fid))\n+\t\tgoto next_model;\n+\n+\tmodel_enq++;\n+\tif (likely(model_enq < args->nb_reqs))\n+\t\tgoto next_rep;\n+\n+\treturn 0;\n+}\n+\n+/* Dequeue inference requests with burst size equal to 1 */\n+static int\n+ml_dequeue_single(void *arg)\n+{\n+\tstruct test_inference *t = ml_test_priv((struct ml_test *)arg);\n+\tstruct rte_ml_op_error error;\n+\tstruct rte_ml_op *op = NULL;\n+\tstruct ml_core_args *args;\n+\tstruct ml_request *req;\n+\tuint64_t total_deq = 0;\n+\tuint8_t nb_filelist;\n+\tuint32_t burst_deq;\n+\tuint32_t lcore_id;\n+\n+\tlcore_id = rte_lcore_id();\n+\targs = &t->args[lcore_id];\n+\tnb_filelist = args->end_fid - args->start_fid + 1;\n+\n+\tif (args->nb_reqs == 0)\n+\t\treturn 0;\n+\n+dequeue_req:\n+\tburst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);\n+\n+\tif (likely(burst_deq == 1)) {\n+\t\ttotal_deq += burst_deq;\n+\t\tif (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {\n+\t\t\trte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);\n+\t\t\tml_err(\"error_code = 0x%016lx, error_message = %s\\n\", error.errcode,\n+\t\t\t error.message);\n+\t\t}\n+\t\treq = (struct ml_request *)op->user_ptr;\n+\t\trte_mempool_put(t->model[req->fid].io_pool, req);\n+\t\trte_mempool_put(t->op_pool, op);\n+\t}\n+\n+\tif (likely(total_deq < args->nb_reqs * nb_filelist))\n+\t\tgoto dequeue_req;\n+\n+\treturn 0;\n+}\n+\n+bool\n+test_inference_cap_check(struct ml_options *opt)\n+{\n+\tstruct rte_ml_dev_info dev_info;\n+\n+\tif (!ml_test_cap_check(opt))\n+\t\treturn false;\n+\n+\trte_ml_dev_info_get(opt->dev_id, &dev_info);\n+\tif (opt->nb_filelist > dev_info.max_models) {\n+\t\tml_err(\"Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)\",\n+\t\t opt->nb_filelist, dev_info.max_models);\n+\t\treturn false;\n+\t}\n+\n+\treturn true;\n+}\n+\n+int\n+test_inference_opt_check(struct ml_options *opt)\n+{\n+\tuint32_t i;\n+\tint ret;\n+\n+\t/* check common opts */\n+\tret = ml_test_opt_check(opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+\t/* check file availability */\n+\tfor (i = 0; i < opt->nb_filelist; i++) {\n+\t\tif (access(opt->filelist[i].model, F_OK) == -1) {\n+\t\t\tml_err(\"Model file not accessible: id = %u, file = %s\", i,\n+\t\t\t opt->filelist[i].model);\n+\t\t\treturn -ENOENT;\n+\t\t}\n+\n+\t\tif (access(opt->filelist[i].input, F_OK) == -1) {\n+\t\t\tml_err(\"Input file not accessible: id = %u, file = %s\", i,\n+\t\t\t opt->filelist[i].input);\n+\t\t\treturn -ENOENT;\n+\t\t}\n+\t}\n+\n+\tif (opt->repetitions == 0) {\n+\t\tml_err(\"Invalid option, repetitions = %\" PRIu64 \"\\n\", opt->repetitions);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* check number of available lcores. */\n+\tif (rte_lcore_count() < 3) {\n+\t\tml_err(\"Insufficient lcores = %u\\n\", rte_lcore_count());\n+\t\tml_err(\"Minimum lcores required to create %u queue-pairs = %u\\n\", 1, 3);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+void\n+test_inference_opt_dump(struct ml_options *opt)\n+{\n+\tuint32_t i;\n+\n+\t/* dump common opts */\n+\tml_test_opt_dump(opt);\n+\n+\t/* dump test opts */\n+\tml_dump(\"repetitions\", \"%\" PRIu64, opt->repetitions);\n+\n+\tml_dump_begin(\"filelist\");\n+\tfor (i = 0; i < opt->nb_filelist; i++) {\n+\t\tml_dump_list(\"model\", i, opt->filelist[i].model);\n+\t\tml_dump_list(\"input\", i, opt->filelist[i].input);\n+\t\tml_dump_list(\"output\", i, opt->filelist[i].output);\n+\t}\n+\tml_dump_end;\n+}\n+\n+int\n+test_inference_setup(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t;\n+\tvoid *test_inference;\n+\tint ret = 0;\n+\tuint32_t i;\n+\n+\ttest_inference = rte_zmalloc_socket(test->name, sizeof(struct test_inference),\n+\t\t\t\t\t RTE_CACHE_LINE_SIZE, opt->socket_id);\n+\tif (test_inference == NULL) {\n+\t\tml_err(\"failed to allocate memory for test_model\");\n+\t\tret = -ENOMEM;\n+\t\tgoto error;\n+\t}\n+\ttest->test_priv = test_inference;\n+\tt = ml_test_priv(test);\n+\n+\tt->nb_used = 0;\n+\tt->cmn.result = ML_TEST_FAILED;\n+\tt->cmn.opt = opt;\n+\n+\t/* get device info */\n+\tret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);\n+\tif (ret < 0) {\n+\t\tml_err(\"failed to get device info\");\n+\t\tgoto error;\n+\t}\n+\n+\tt->enqueue = ml_enqueue_single;\n+\tt->dequeue = ml_dequeue_single;\n+\n+\t/* set model initial state */\n+\tfor (i = 0; i < opt->nb_filelist; i++)\n+\t\tt->model[i].state = MODEL_INITIAL;\n+\n+\treturn 0;\n+\n+error:\n+\tif (test_inference != NULL)\n+\t\trte_free(test_inference);\n+\n+\treturn ret;\n+}\n+\n+void\n+test_inference_destroy(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t;\n+\n+\tRTE_SET_USED(opt);\n+\n+\tt = ml_test_priv(test);\n+\tif (t != NULL)\n+\t\trte_free(t);\n+}\n+\n+int\n+ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct rte_ml_dev_qp_conf qp_conf;\n+\tstruct test_inference *t;\n+\tint ret;\n+\n+\tt = ml_test_priv(test);\n+\n+\tret = ml_test_device_configure(test, opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+\t/* setup queue pairs */\n+\tqp_conf.nb_desc = t->cmn.dev_info.max_desc;\n+\tqp_conf.cb = NULL;\n+\n+\tret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);\n+\tif (ret != 0) {\n+\t\tml_err(\"Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\\n\",\n+\t\t opt->dev_id, 0);\n+\t\tgoto error;\n+\t}\n+\n+\tret = ml_test_device_start(test, opt);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\treturn 0;\n+\n+error:\n+\tml_test_device_close(test, opt);\n+\n+\treturn ret;\n+}\n+\n+int\n+ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt)\n+{\n+\tint ret;\n+\n+\tret = ml_test_device_stop(test, opt);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\tret = ml_test_device_close(test, opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+\treturn 0;\n+\n+error:\n+\tml_test_device_close(test, opt);\n+\n+\treturn ret;\n+}\n+\n+/* Callback for IO pool create. This function would compute the fields of ml_request\n+ * structure and prepare the quantized input data.\n+ */\n+static void\n+ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)\n+{\n+\tstruct test_inference *t = ml_test_priv((struct ml_test *)opaque);\n+\tstruct ml_request *req = (struct ml_request *)obj;\n+\n+\tRTE_SET_USED(mp);\n+\tRTE_SET_USED(obj_idx);\n+\n+\treq->input = RTE_PTR_ADD(\n+\t\tobj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));\n+\treq->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,\n+\t\t\t\t\t\t\t t->cmn.dev_info.min_align_size));\n+\treq->niters = 0;\n+\n+\t/* quantize data */\n+\trte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,\n+\t\t\t t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);\n+}\n+\n+int\n+ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t fid)\n+{\n+\tstruct test_inference *t = ml_test_priv(test);\n+\tchar mz_name[RTE_MEMZONE_NAMESIZE];\n+\tchar mp_name[RTE_MEMPOOL_NAMESIZE];\n+\tconst struct rte_memzone *mz;\n+\tuint64_t nb_buffers;\n+\tuint32_t buff_size;\n+\tuint32_t mz_size;\n+\tuint32_t fsize;\n+\tFILE *fp;\n+\tint ret;\n+\n+\t/* get input buffer size */\n+\tret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,\n+\t\t\t\t &t->model[fid].inp_qsize, &t->model[fid].inp_dsize);\n+\tif (ret != 0) {\n+\t\tml_err(\"Failed to get input size, model : %s\\n\", opt->filelist[fid].model);\n+\t\treturn ret;\n+\t}\n+\n+\t/* get output buffer size */\n+\tret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,\n+\t\t\t\t\tt->model[fid].info.batch_size, &t->model[fid].out_qsize,\n+\t\t\t\t\t&t->model[fid].out_dsize);\n+\tif (ret != 0) {\n+\t\tml_err(\"Failed to get input size, model : %s\\n\", opt->filelist[fid].model);\n+\t\treturn ret;\n+\t}\n+\n+\t/* allocate buffer for user data */\n+\tmz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;\n+\tsprintf(mz_name, \"ml_user_data_%d\", fid);\n+\tmz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);\n+\tif (mz == NULL) {\n+\t\tml_err(\"Memzone allocation failed for ml_user_data\\n\");\n+\t\tret = -ENOMEM;\n+\t\tgoto error;\n+\t}\n+\n+\tt->model[fid].input = mz->addr;\n+\tt->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);\n+\n+\t/* load input file */\n+\tfp = fopen(opt->filelist[fid].input, \"r\");\n+\tif (fp == NULL) {\n+\t\tml_err(\"Failed to open input file : %s\\n\", opt->filelist[fid].input);\n+\t\tret = -errno;\n+\t\tgoto error;\n+\t}\n+\n+\tfseek(fp, 0, SEEK_END);\n+\tfsize = ftell(fp);\n+\tfseek(fp, 0, SEEK_SET);\n+\tif (fsize != t->model[fid].inp_dsize) {\n+\t\tml_err(\"Invalid input file, size = %u (expected size = %\" PRIu64 \")\\n\", fsize,\n+\t\t t->model[fid].inp_dsize);\n+\t\tret = -EINVAL;\n+\t\tfclose(fp);\n+\t\tgoto error;\n+\t}\n+\n+\tif (fread(t->model[fid].input, 1, t->model[fid].inp_dsize, fp) != t->model[fid].inp_dsize) {\n+\t\tml_err(\"Failed to read input file : %s\\n\", opt->filelist[fid].input);\n+\t\tret = -errno;\n+\t\tfclose(fp);\n+\t\tgoto error;\n+\t}\n+\tfclose(fp);\n+\n+\t/* create mempool for quantized input and output buffers. ml_request_initialize is\n+\t * used as a callback for object creation.\n+\t */\n+\tbuff_size = RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size) +\n+\t\t RTE_ALIGN_CEIL(t->model[fid].inp_qsize, t->cmn.dev_info.min_align_size) +\n+\t\t RTE_ALIGN_CEIL(t->model[fid].out_qsize, t->cmn.dev_info.min_align_size);\n+\tnb_buffers = RTE_MIN((uint64_t)ML_TEST_MAX_POOL_SIZE, opt->repetitions);\n+\n+\tt->fid = fid;\n+\tsprintf(mp_name, \"ml_io_pool_%d\", fid);\n+\tt->model[fid].io_pool = rte_mempool_create(mp_name, nb_buffers, buff_size, 0, 0, NULL, NULL,\n+\t\t\t\t\t\t ml_request_initialize, test, opt->socket_id, 0);\n+\tif (t->model[fid].io_pool == NULL) {\n+\t\tml_err(\"Failed to create io pool : %s\\n\", \"ml_io_pool\");\n+\t\tret = -ENOMEM;\n+\t\tgoto error;\n+\t}\n+\n+\treturn 0;\n+\n+error:\n+\tif (mz != NULL)\n+\t\trte_memzone_free(mz);\n+\n+\tif (t->model[fid].io_pool != NULL) {\n+\t\trte_mempool_free(t->model[fid].io_pool);\n+\t\tt->model[fid].io_pool = NULL;\n+\t}\n+\n+\treturn ret;\n+}\n+\n+void\n+ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, uint16_t fid)\n+{\n+\tchar mz_name[RTE_MEMZONE_NAMESIZE];\n+\tchar mp_name[RTE_MEMPOOL_NAMESIZE];\n+\tconst struct rte_memzone *mz;\n+\tstruct rte_mempool *mp;\n+\n+\tRTE_SET_USED(test);\n+\tRTE_SET_USED(opt);\n+\n+\t/* release user data memzone */\n+\tsprintf(mz_name, \"ml_user_data_%d\", fid);\n+\tmz = rte_memzone_lookup(mz_name);\n+\tif (mz != NULL)\n+\t\trte_memzone_free(mz);\n+\n+\t/* destroy io pool */\n+\tsprintf(mp_name, \"ml_io_pool_%d\", fid);\n+\tmp = rte_mempool_lookup(mp_name);\n+\tif (mp != NULL)\n+\t\trte_mempool_free(mp);\n+}\n+\n+int\n+ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t = ml_test_priv(test);\n+\n+\t/* create op pool */\n+\tt->op_pool = rte_ml_op_pool_create(\"ml_test_op_pool\", ML_TEST_MAX_POOL_SIZE, 0, 0,\n+\t\t\t\t\t opt->socket_id);\n+\tif (t->op_pool == NULL) {\n+\t\tml_err(\"Failed to create op pool : %s\\n\", \"ml_op_pool\");\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+void\n+ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t = ml_test_priv(test);\n+\n+\tRTE_SET_USED(opt);\n+\n+\t/* release op pool */\n+\tif (t->op_pool != NULL)\n+\t\trte_mempool_free(t->op_pool);\n+}\n+\n+/* Callback for mempool object iteration. This call would dequantize output data. */\n+static void\n+ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)\n+{\n+\tstruct test_inference *t = ml_test_priv((struct ml_test *)opaque);\n+\tstruct ml_request *req = (struct ml_request *)obj;\n+\tstruct ml_model *model = &t->model[req->fid];\n+\n+\tRTE_SET_USED(mp);\n+\tRTE_SET_USED(obj_idx);\n+\n+\tif (req->niters == 0)\n+\t\treturn;\n+\n+\tt->nb_used++;\n+\trte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,\n+\t\t\t req->output, model->output);\n+}\n+\n+int\n+ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid)\n+{\n+\tstruct test_inference *t = ml_test_priv(test);\n+\n+\tRTE_SET_USED(opt);\n+\n+\trte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);\n+\n+\tif (t->nb_used > 0)\n+\t\tt->cmn.result = ML_TEST_SUCCESS;\n+\telse\n+\t\tt->cmn.result = ML_TEST_FAILED;\n+\n+\treturn t->cmn.result;\n+}\n+\n+int\n+ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,\n+\t\t\t uint16_t end_fid)\n+{\n+\tstruct test_inference *t = ml_test_priv(test);\n+\tuint32_t lcore_id;\n+\tuint32_t id = 0;\n+\n+\tRTE_LCORE_FOREACH_WORKER(lcore_id)\n+\t{\n+\t\tif (id == 2)\n+\t\t\tbreak;\n+\n+\t\tt->args[lcore_id].nb_reqs = opt->repetitions;\n+\t\tt->args[lcore_id].start_fid = start_fid;\n+\t\tt->args[lcore_id].end_fid = end_fid;\n+\n+\t\tif (id % 2 == 0)\n+\t\t\trte_eal_remote_launch(t->enqueue, test, lcore_id);\n+\t\telse\n+\t\t\trte_eal_remote_launch(t->dequeue, test, lcore_id);\n+\n+\t\tid++;\n+\t}\n+\n+\treturn 0;\n+}\ndiff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h\nnew file mode 100644\nindex 0000000000..99baad5bfd\n--- /dev/null\n+++ b/app/test-mldev/test_inference_common.h\n@@ -0,0 +1,65 @@\n+/* SPDX-License-Identifier: BSD-3-Clause\n+ * Copyright (c) 2022 Marvell.\n+ */\n+\n+#ifndef _ML_TEST_INFERENCE_COMMON_\n+#define _ML_TEST_INFERENCE_COMMON_\n+\n+#include <stdbool.h>\n+#include <stdint.h>\n+\n+#include <rte_common.h>\n+#include <rte_mempool.h>\n+\n+#include \"ml_options.h\"\n+#include \"ml_test.h\"\n+#include \"test_common.h\"\n+#include \"test_model_common.h\"\n+\n+struct ml_request {\n+\tuint8_t *input;\n+\tuint8_t *output;\n+\tuint16_t fid;\n+\tuint64_t niters;\n+};\n+\n+struct ml_core_args {\n+\tuint64_t nb_reqs;\n+\tuint16_t start_fid;\n+\tuint16_t end_fid;\n+};\n+\n+struct test_inference {\n+\t/* common data */\n+\tstruct test_common cmn;\n+\n+\t/* test specific data */\n+\tstruct ml_model model[ML_TEST_MAX_MODELS];\n+\tstruct rte_mempool *op_pool;\n+\n+\tuint64_t nb_used;\n+\tuint16_t fid;\n+\n+\tint (*enqueue)(void *arg);\n+\tint (*dequeue)(void *arg);\n+\n+\tstruct ml_core_args args[RTE_MAX_LCORE];\n+} __rte_cache_aligned;\n+\n+bool test_inference_cap_check(struct ml_options *opt);\n+int test_inference_opt_check(struct ml_options *opt);\n+void test_inference_opt_dump(struct ml_options *opt);\n+int test_inference_setup(struct ml_test *test, struct ml_options *opt);\n+void test_inference_destroy(struct ml_test *test, struct ml_options *opt);\n+\n+int ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt);\n+int ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt);\n+int ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t fid);\n+void ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, uint16_t fid);\n+int ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt);\n+void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);\n+int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid);\n+int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,\n+\t\t\t uint16_t end_fid);\n+\n+#endif /* _ML_TEST_INFERENCE_COMMON_ */\ndiff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c\nnew file mode 100644\nindex 0000000000..ee9c7bf14e\n--- /dev/null\n+++ b/app/test-mldev/test_inference_ordered.c\n@@ -0,0 +1,119 @@\n+/* SPDX-License-Identifier: BSD-3-Clause\n+ * Copyright (c) 2022 Marvell.\n+ */\n+\n+#include <stdint.h>\n+\n+#include <rte_common.h>\n+#include <rte_launch.h>\n+\n+#include \"ml_common.h\"\n+#include \"ml_test.h\"\n+#include \"test_inference_common.h\"\n+#include \"test_model_common.h\"\n+\n+static int\n+test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t;\n+\tuint16_t fid = 0;\n+\tint ret = 0;\n+\n+\tt = ml_test_priv(test);\n+\n+\tret = ml_inference_mldev_setup(test, opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+\tret = ml_inference_mem_setup(test, opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+next_model:\n+\t/* load model */\n+\tret = ml_model_load(test, opt, &t->model[fid], fid);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\t/* start model */\n+\tret = ml_model_start(test, opt, &t->model[fid], fid);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\tret = ml_inference_iomem_setup(test, opt, fid);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\t/* launch inferences for one model using available queue pairs */\n+\tret = ml_inference_launch_cores(test, opt, fid, fid);\n+\tif (ret != 0) {\n+\t\tml_err(\"failed to launch cores\");\n+\t\tgoto error;\n+\t}\n+\n+\trte_eal_mp_wait_lcore();\n+\n+\tret = ml_inference_result(test, opt, fid);\n+\tif (ret != ML_TEST_SUCCESS)\n+\t\tgoto error;\n+\n+\tml_inference_iomem_destroy(test, opt, fid);\n+\n+\t/* stop model */\n+\tret = ml_model_stop(test, opt, &t->model[fid], fid);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\t/* unload model */\n+\tret = ml_model_unload(test, opt, &t->model[fid], fid);\n+\tif (ret != 0)\n+\t\tgoto error;\n+\n+\tfid++;\n+\tif (fid < opt->nb_filelist)\n+\t\tgoto next_model;\n+\n+\tml_inference_mem_destroy(test, opt);\n+\n+\tret = ml_inference_mldev_destroy(test, opt);\n+\tif (ret != 0)\n+\t\treturn ret;\n+\n+\tt->cmn.result = ML_TEST_SUCCESS;\n+\n+\treturn 0;\n+\n+error:\n+\tml_inference_iomem_destroy(test, opt, fid);\n+\tml_inference_mem_destroy(test, opt);\n+\tml_model_stop(test, opt, &t->model[fid], fid);\n+\tml_model_unload(test, opt, &t->model[fid], fid);\n+\n+\tt->cmn.result = ML_TEST_FAILED;\n+\n+\treturn ret;\n+}\n+\n+static int\n+test_inference_ordered_result(struct ml_test *test, struct ml_options *opt)\n+{\n+\tstruct test_inference *t;\n+\n+\tRTE_SET_USED(opt);\n+\n+\tt = ml_test_priv(test);\n+\n+\treturn t->cmn.result;\n+}\n+\n+static const struct ml_test_ops inference_ordered = {\n+\t.cap_check = test_inference_cap_check,\n+\t.opt_check = test_inference_opt_check,\n+\t.opt_dump = test_inference_opt_dump,\n+\t.test_setup = test_inference_setup,\n+\t.test_destroy = test_inference_destroy,\n+\t.test_driver = test_inference_ordered_driver,\n+\t.test_result = test_inference_ordered_result,\n+};\n+\n+ML_TEST_REGISTER(inference_ordered);\ndiff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h\nindex cdd1215cbc..6eb70a352a 100644\n--- a/app/test-mldev/test_model_common.h\n+++ b/app/test-mldev/test_model_common.h\n@@ -23,6 +23,16 @@ struct ml_model {\n \tuint16_t id;\n \tstruct rte_ml_model_info info;\n \tenum model_state state;\n+\n+\tuint64_t inp_dsize;\n+\tuint64_t inp_qsize;\n+\tuint64_t out_dsize;\n+\tuint64_t out_qsize;\n+\n+\tuint8_t *input;\n+\tuint8_t *output;\n+\n+\tstruct rte_mempool *io_pool;\n };\n \n int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,\n", "prefixes": [ "v4", "05/12" ] }