Message ID | 20231026124347.22477-1-syalavarthi@marvell.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5EC4443208; Thu, 26 Oct 2023 14:43:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 070F740E40; Thu, 26 Oct 2023 14:43:56 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id B0D6E4025F for <dev@dpdk.org>; Thu, 26 Oct 2023 14:43:54 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 39QALkxx007523 for <dev@dpdk.org>; Thu, 26 Oct 2023 05:43:54 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=lgRwm0Gda9VLiIALlimcCZh61/Yzok7XYgjvbOi5mPY=; b=FGZ3/i2gqzIJ4OFCU0nkAtqftCUn6wB0iVoUhgdUdnLG2UfEsDXHobd2/JViy1/00gQ/ 4AGQcV/gnty71YU5ZZYp8hZzIHRVD01zfUjQotnU6GSKe7UjX3ADuPdPHVjvZQkdq/LN AUU/DCrtbTEHA+1HIleUcPzY7YGkjxL9EjhmE1td9rip9dYr3U+TA6zXgL4fUyHpfsBq XxoXhqKV8zynezq4S/4XgDhbI+jtExI/EY+vipbvL86q/GyAB2+nC2P7gGR5wo7KYBUk lIoosHJdIPjkjxQXYjh1DZxoxjVNxuwa6NG3KpnNLuaVsxGu3Qy5aYGCjDgdmmhk9iHd dQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3txcsr25pc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for <dev@dpdk.org>; Thu, 26 Oct 2023 05:43:53 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 26 Oct 2023 05:43:51 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 26 Oct 2023 05:43:52 -0700 Received: from ml-host-33.caveonetworks.com (unknown [10.110.143.233]) by maili.marvell.com (Postfix) with ESMTP id 88D5E3F7097; Thu, 26 Oct 2023 05:43:51 -0700 (PDT) From: Srikanth Yalavarthi <syalavarthi@marvell.com> To: CC: <dev@dpdk.org>, <syalavarthi@marvell.com>, <sshankarnara@marvell.com>, <aprabhu@marvell.com>, <ptakkar@marvell.com> Subject: [PATCH v9 00/34] Implementation of revised ml/cnxk driver Date: Thu, 26 Oct 2023 05:43:09 -0700 Message-ID: <20231026124347.22477-1-syalavarthi@marvell.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230830155927.3566-1-syalavarthi@marvell.com> References: <20230830155927.3566-1-syalavarthi@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-ORIG-GUID: KVxRGmcJZT23FSloltmORY5ans5rLRkR X-Proofpoint-GUID: KVxRGmcJZT23FSloltmORY5ans5rLRkR X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.987,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-10-26_10,2023-10-26_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org |
Series |
Implementation of revised ml/cnxk driver
|
|
Message
Srikanth Yalavarthi
Oct. 26, 2023, 12:43 p.m. UTC
This patch series is an implementation of revised ml/cnxk driver to support models compiled with TVM compiler framework. TVM models use a hybrid mode for execution, with regions of the model executing on the ML accelerator and the rest executing on CPU cores. This series of commits reorganizes the ml/cnxk driver and adds support to execute multiple regions with-in a TVM model. v9: - Fixed incorrect IO layout for TVM Marvell models - Set byte offset to zero for I/O tensors - Updated max layers macro definition. Set to TVMDP max layers. - Fixed TVM model IO type to RTE IO type map v8: - Updated CMake dependency resolution of external dependencies - Updated mldevs/cnxk documentation - Updated meson config files for cn9k and cn10k to include cmake v7: - Updated steps to build dependencies in cnxk mldev documentation - Replace str functions with rte_str functions - Drop use of rte_exit in ml/cnxk driver v6: - Added depends info for series. This series depends on patch-132887 - Fix merge conflicts with dpdk-23.11-rc1 - Fix issues with ml/cnxk driver release notes - Added build dependency information for dlpack headers v5: - Fix build failures for individual patches in the series - Finished build testing with devtools/test-meson-builds.sh script v4: - Squashed release notes - Updated external build dependency info in documentation v3: - Reduced use of RTE_MLDEV_CNXK_ENABLE_MVTVM macro - Added stubs file with dummy functions to use when TVM is disabled - Dropped patch with internal function to read firmware - Updated ML CNXK PMD documentation - Added external library dependency info in documentation - Added release notes for 23.11 v2: - Fix xstats reporting - Fix issues reported by klocwork static analysis tool - Update external header inclusions v1: - Initial changes Anup Prabhu (2): ml/cnxk: enable OCM check for multilayer TVM model ml/cnxk: enable fast-path ops for TVM models Prince Takkar (2): ml/cnxk: update internal TVM model info structure ml/cnxk: support quantize and dequantize callback Srikanth Yalavarthi (30): ml/cnxk: drop support for register polling ml/cnxk: add generic cnxk device structure ml/cnxk: add generic model and layer structures ml/cnxk: add generic cnxk request structure ml/cnxk: add generic cnxk xstats structures ml/cnxk: rename cnxk ops function pointers struct ml/cnxk: update device handling functions ml/cnxk: update queue-pair handling functions ml/cnxk: update model load and unload functions ml/cnxk: update model start and stop functions ml/cnxk: update model utility functions ml/cnxk: update data quantization functions ml/cnxk: update device debug functions ml/cnxk: update device stats functions ml/cnxk: update device and model xstats functions ml/cnxk: update fast path functions ml/cnxk: move error handling to cnxk layer ml/cnxk: support config and close of tvmdp library ml/cnxk: add structures to support TVM model type ml/cnxk: add support for identify model type ml/cnxk: add support to parse TVM model objects ml/cnxk: fetch layer info and load TVM model ml/cnxk: update internal info for TVM model ml/cnxk: enable model unload in tvmdp library ml/cnxk: support start and stop for TVM models ml/cnxk: support device dump for TVM models ml/cnxk: enable reporting model runtime as xstats ml/cnxk: implement I/O alloc and free callbacks ml/cnxk: add generic ML malloc and free callback ml/cnxk: enable creation of mvtvm virtual device config/arm/arm64_cn10k_linux_gcc | 1 + config/arm/arm64_cn9k_linux_gcc | 1 + doc/guides/mldevs/cnxk.rst | 223 +- doc/guides/rel_notes/release_23_11.rst | 3 + drivers/ml/cnxk/cn10k_ml_dev.c | 416 ++-- drivers/ml/cnxk/cn10k_ml_dev.h | 457 +--- drivers/ml/cnxk/cn10k_ml_model.c | 403 ++-- drivers/ml/cnxk/cn10k_ml_model.h | 151 +- drivers/ml/cnxk/cn10k_ml_ocm.c | 111 +- drivers/ml/cnxk/cn10k_ml_ocm.h | 15 +- drivers/ml/cnxk/cn10k_ml_ops.c | 2828 ++++++++---------------- drivers/ml/cnxk/cn10k_ml_ops.h | 358 ++- drivers/ml/cnxk/cnxk_ml_dev.c | 22 + drivers/ml/cnxk/cnxk_ml_dev.h | 120 + drivers/ml/cnxk/cnxk_ml_io.c | 95 + drivers/ml/cnxk/cnxk_ml_io.h | 90 + drivers/ml/cnxk/cnxk_ml_model.c | 94 + drivers/ml/cnxk/cnxk_ml_model.h | 192 ++ drivers/ml/cnxk/cnxk_ml_ops.c | 1690 ++++++++++++++ drivers/ml/cnxk/cnxk_ml_ops.h | 87 + drivers/ml/cnxk/cnxk_ml_utils.c | 15 + drivers/ml/cnxk/cnxk_ml_utils.h | 17 + drivers/ml/cnxk/cnxk_ml_xstats.h | 152 ++ drivers/ml/cnxk/meson.build | 70 + drivers/ml/cnxk/mvtvm_ml_dev.c | 196 ++ drivers/ml/cnxk/mvtvm_ml_dev.h | 40 + drivers/ml/cnxk/mvtvm_ml_model.c | 409 ++++ drivers/ml/cnxk/mvtvm_ml_model.h | 90 + drivers/ml/cnxk/mvtvm_ml_ops.c | 652 ++++++ drivers/ml/cnxk/mvtvm_ml_ops.h | 82 + drivers/ml/cnxk/mvtvm_ml_stubs.c | 141 ++ drivers/ml/cnxk/mvtvm_ml_stubs.h | 36 + 32 files changed, 6298 insertions(+), 2959 deletions(-) create mode 100644 drivers/ml/cnxk/cnxk_ml_dev.c create mode 100644 drivers/ml/cnxk/cnxk_ml_dev.h create mode 100644 drivers/ml/cnxk/cnxk_ml_io.c create mode 100644 drivers/ml/cnxk/cnxk_ml_io.h create mode 100644 drivers/ml/cnxk/cnxk_ml_model.c create mode 100644 drivers/ml/cnxk/cnxk_ml_model.h create mode 100644 drivers/ml/cnxk/cnxk_ml_ops.c create mode 100644 drivers/ml/cnxk/cnxk_ml_ops.h create mode 100644 drivers/ml/cnxk/cnxk_ml_utils.c create mode 100644 drivers/ml/cnxk/cnxk_ml_utils.h create mode 100644 drivers/ml/cnxk/cnxk_ml_xstats.h create mode 100644 drivers/ml/cnxk/mvtvm_ml_dev.c create mode 100644 drivers/ml/cnxk/mvtvm_ml_dev.h create mode 100644 drivers/ml/cnxk/mvtvm_ml_model.c create mode 100644 drivers/ml/cnxk/mvtvm_ml_model.h create mode 100644 drivers/ml/cnxk/mvtvm_ml_ops.c create mode 100644 drivers/ml/cnxk/mvtvm_ml_ops.h create mode 100644 drivers/ml/cnxk/mvtvm_ml_stubs.c create mode 100644 drivers/ml/cnxk/mvtvm_ml_stubs.h
Comments
On Fri, Oct 27, 2023 at 12:12 AM Srikanth Yalavarthi <syalavarthi@marvell.com> wrote: > > This patch series is an implementation of revised ml/cnxk driver > to support models compiled with TVM compiler framework. TVM models > use a hybrid mode for execution, with regions of the model executing > on the ML accelerator and the rest executing on CPU cores. > > This series of commits reorganizes the ml/cnxk driver and adds support > to execute multiple regions with-in a TVM model. > > v9: > - Fixed incorrect IO layout for TVM Marvell models > - Set byte offset to zero for I/O tensors > - Updated max layers macro definition. Set to TVMDP max layers. > - Fixed TVM model IO type to RTE IO type map Series applied to dpdk-next-net-mrvl/for-next-net with following chages. Thanks. 1) cnxl ml driver update in doc/guides/rel_notes/release_23_11.rst moved to close to mldev subsystem changes 2) [for-next-net]dell[dpdk-next-net-mrvl] $ git diff diff --git a/doc/guides/mldevs/cnxk.rst b/doc/guides/mldevs/cnxk.rst index 28e5b5b87f..25e8ff783a 100644 --- a/doc/guides/mldevs/cnxk.rst +++ b/doc/guides/mldevs/cnxk.rst @@ -212,9 +212,9 @@ not part of DPDK and must be installed separately: .. note:: In order for meson to find the dependencies during the configure stage, - it is required to add the cmake paths <install_prefix>/lib/cmake/dlpack, - <install_prefix>/lib/cmake/dmlc and <install_prefix>/lib/cmake/tvm to - CMAKE_PREFIX_PATH and <install_prefix>/lib/pkgconfig to PKG_CONFIG_PATH. + it is required to update CMAKE_PREFIX_PATH and PKG_CONFIG_PATH as below. + CMAKE_PREFIX_PATH='<install_prefix>/lib/cmake/tvm:<install_prefix>/lib/cmake/dlpack:<install_prefix>/lib/cmake/dmlc' + PKG_CONFIG_PATH='<install_prefix>/lib/pkgconfig' For the record, I have used following build command to test code with external build dep. Assumung all depended libraries installed at /export/cross_ml/install/ CMAKE_PREFIX_PATH='/export/cross_ml/install/lib/cmake/tvm:/export/cross_ml/install/lib/cmake/dlpack:/export/cross_ml/install/lib/cmake/dmlc' PKG_CONFIG_PATH='/export/cross_ml/install/lib/pkgconfig/' meson setup --cross config/arm/arm64_cn10k_linux_gcc -Denable_docs=true -Dexamples=all -Dc_args='-I/export/cross_ml/install/include' -Dc_link_args='-L/export/cross_ml/install/lib' build Also, -Dc_args='-I/export/cross_ml/install/include' -Dc_link_args='-L/export/cross_ml/install/lib' could be removed when following patch merged through main tree. https://patches.dpdk.org/project/dpdk/patch/20231029082004.5576-1-syalavarthi@marvell.com/