From patchwork Wed Oct 18 06:47:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srikanth Yalavarthi X-Patchwork-Id: 132850 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D1E2B43196; Wed, 18 Oct 2023 08:49:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9AE2242E0B; Wed, 18 Oct 2023 08:48:28 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9E55140E6E for ; Wed, 18 Oct 2023 08:48:17 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 39I3vKJu020024 for ; Tue, 17 Oct 2023 23:48:17 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=oy6txxdPTmoJxOBcijBOsnU3N0aKGOKQamaFMjj285w=; b=N6uBjdYQJXZKNmbDSxr7q5DfkINDE1Q2Gxadzf3b053wYFIYYUm4B0LhFIMS+2lnEDH2 wyaOvBhJrExzBEPZanIYB2W5Pf7uqCR0dTIQMDuwndgRMVB7rfJLur/rqaa6OkPaD1dP /iINniaFMt7EdYOhm/h5uoQkq48TQi1iGSzUQFZo+h4H92yxxmB0wQuj5N6NVGcKApKM u3qyXyozU6L22RgFnvj6yC0gUj+fxjMpstsw0cACMcI/RwY2cPArWOBZnvRqAhABzgfP GpqfwneTQyhvtYZtRHDVM9B6B4pr0jrkRu2oUr+5uiZo+RtAljvjoZnwA/Au/BW5RZs6 zA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3tstb3ursq-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 17 Oct 2023 23:48:16 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Tue, 17 Oct 2023 23:48:12 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Tue, 17 Oct 2023 23:48:12 -0700 Received: from ml-host-33.caveonetworks.com (unknown [10.110.143.233]) by maili.marvell.com (Postfix) with ESMTP id 1D1EA3F704A; Tue, 17 Oct 2023 23:48:12 -0700 (PDT) From: Srikanth Yalavarthi To: Srikanth Yalavarthi CC: , , , Subject: [PATCH v5 07/34] ml/cnxk: update device handling functions Date: Tue, 17 Oct 2023 23:47:35 -0700 Message-ID: <20231018064806.24145-8-syalavarthi@marvell.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231018064806.24145-1-syalavarthi@marvell.com> References: <20230830155927.3566-1-syalavarthi@marvell.com> <20231018064806.24145-1-syalavarthi@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: N_9dtkjOXsDMK9AR6o-3AEjHNhffKngL X-Proofpoint-GUID: N_9dtkjOXsDMK9AR6o-3AEjHNhffKngL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-10-18_04,2023-10-17_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Implement CNXK wrapper functions for dev_info_get, dev_configure, dev_close, dev_start and dev_stop. The wrapper functions allocate / release common resources for the ML driver and invoke device specific functions. Signed-off-by: Srikanth Yalavarthi --- drivers/ml/cnxk/cn10k_ml_ops.c | 230 ++------------------------ drivers/ml/cnxk/cn10k_ml_ops.h | 16 +- drivers/ml/cnxk/cnxk_ml_dev.h | 3 + drivers/ml/cnxk/cnxk_ml_ops.c | 286 ++++++++++++++++++++++++++++++++- drivers/ml/cnxk/cnxk_ml_ops.h | 3 + 5 files changed, 314 insertions(+), 224 deletions(-) diff --git a/drivers/ml/cnxk/cn10k_ml_ops.c b/drivers/ml/cnxk/cn10k_ml_ops.c index 66b38fc1eb..6d8f2c8777 100644 --- a/drivers/ml/cnxk/cn10k_ml_ops.c +++ b/drivers/ml/cnxk/cn10k_ml_ops.c @@ -101,7 +101,7 @@ qp_memzone_name_get(char *name, int size, int dev_id, int qp_id) snprintf(name, size, "cnxk_ml_qp_mem_%u:%u", dev_id, qp_id); } -static int +int cnxk_ml_qp_destroy(const struct rte_ml_dev *dev, struct cnxk_ml_qp *qp) { const struct rte_memzone *qp_mem; @@ -861,20 +861,12 @@ cn10k_ml_cache_model_data(struct rte_ml_dev *dev, uint16_t model_id) } int -cn10k_ml_dev_info_get(struct rte_ml_dev *dev, struct rte_ml_dev_info *dev_info) +cn10k_ml_dev_info_get(struct cnxk_ml_dev *cnxk_mldev, struct rte_ml_dev_info *dev_info) { struct cn10k_ml_dev *cn10k_mldev; - struct cnxk_ml_dev *cnxk_mldev; - if (dev_info == NULL) - return -EINVAL; - - cnxk_mldev = dev->data->dev_private; cn10k_mldev = &cnxk_mldev->cn10k_mldev; - memset(dev_info, 0, sizeof(struct rte_ml_dev_info)); - dev_info->driver_name = dev->device->driver->name; - dev_info->max_models = ML_CNXK_MAX_MODELS; if (cn10k_mldev->hw_queue_lock) dev_info->max_queue_pairs = ML_CN10K_MAX_QP_PER_DEVICE_SL; else @@ -889,143 +881,17 @@ cn10k_ml_dev_info_get(struct rte_ml_dev *dev, struct rte_ml_dev_info *dev_info) } int -cn10k_ml_dev_configure(struct rte_ml_dev *dev, const struct rte_ml_dev_config *conf) +cn10k_ml_dev_configure(struct cnxk_ml_dev *cnxk_mldev, const struct rte_ml_dev_config *conf) { - struct rte_ml_dev_info dev_info; struct cn10k_ml_dev *cn10k_mldev; - struct cnxk_ml_dev *cnxk_mldev; - struct cnxk_ml_model *model; struct cn10k_ml_ocm *ocm; - struct cnxk_ml_qp *qp; - uint16_t model_id; - uint32_t mz_size; uint16_t tile_id; - uint16_t qp_id; int ret; - if (dev == NULL || conf == NULL) - return -EINVAL; + RTE_SET_USED(conf); - /* Get CN10K device handle */ - cnxk_mldev = dev->data->dev_private; cn10k_mldev = &cnxk_mldev->cn10k_mldev; - cn10k_ml_dev_info_get(dev, &dev_info); - if (conf->nb_models > dev_info.max_models) { - plt_err("Invalid device config, nb_models > %u\n", dev_info.max_models); - return -EINVAL; - } - - if (conf->nb_queue_pairs > dev_info.max_queue_pairs) { - plt_err("Invalid device config, nb_queue_pairs > %u\n", dev_info.max_queue_pairs); - return -EINVAL; - } - - if (cnxk_mldev->state == ML_CNXK_DEV_STATE_PROBED) { - plt_ml_dbg("Configuring ML device, nb_queue_pairs = %u, nb_models = %u", - conf->nb_queue_pairs, conf->nb_models); - - /* Load firmware */ - ret = cn10k_ml_fw_load(cnxk_mldev); - if (ret != 0) - return ret; - } else if (cnxk_mldev->state == ML_CNXK_DEV_STATE_CONFIGURED) { - plt_ml_dbg("Re-configuring ML device, nb_queue_pairs = %u, nb_models = %u", - conf->nb_queue_pairs, conf->nb_models); - } else if (cnxk_mldev->state == ML_CNXK_DEV_STATE_STARTED) { - plt_err("Device can't be reconfigured in started state\n"); - return -ENOTSUP; - } else if (cnxk_mldev->state == ML_CNXK_DEV_STATE_CLOSED) { - plt_err("Device can't be reconfigured after close\n"); - return -ENOTSUP; - } - - /* Configure queue-pairs */ - if (dev->data->queue_pairs == NULL) { - mz_size = sizeof(dev->data->queue_pairs[0]) * conf->nb_queue_pairs; - dev->data->queue_pairs = - rte_zmalloc("cn10k_mldev_queue_pairs", mz_size, RTE_CACHE_LINE_SIZE); - if (dev->data->queue_pairs == NULL) { - dev->data->nb_queue_pairs = 0; - plt_err("Failed to get memory for queue_pairs, nb_queue_pairs %u", - conf->nb_queue_pairs); - return -ENOMEM; - } - } else { /* Re-configure */ - void **queue_pairs; - - /* Release all queue pairs as ML spec doesn't support queue_pair_destroy. */ - for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) { - qp = dev->data->queue_pairs[qp_id]; - if (qp != NULL) { - ret = cn10k_ml_dev_queue_pair_release(dev, qp_id); - if (ret < 0) - return ret; - } - } - - queue_pairs = dev->data->queue_pairs; - queue_pairs = - rte_realloc(queue_pairs, sizeof(queue_pairs[0]) * conf->nb_queue_pairs, - RTE_CACHE_LINE_SIZE); - if (queue_pairs == NULL) { - dev->data->nb_queue_pairs = 0; - plt_err("Failed to realloc queue_pairs, nb_queue_pairs = %u", - conf->nb_queue_pairs); - ret = -ENOMEM; - goto error; - } - - memset(queue_pairs, 0, sizeof(queue_pairs[0]) * conf->nb_queue_pairs); - dev->data->queue_pairs = queue_pairs; - } - dev->data->nb_queue_pairs = conf->nb_queue_pairs; - - /* Allocate ML models */ - if (dev->data->models == NULL) { - mz_size = sizeof(dev->data->models[0]) * conf->nb_models; - dev->data->models = rte_zmalloc("cn10k_mldev_models", mz_size, RTE_CACHE_LINE_SIZE); - if (dev->data->models == NULL) { - dev->data->nb_models = 0; - plt_err("Failed to get memory for ml_models, nb_models %u", - conf->nb_models); - ret = -ENOMEM; - goto error; - } - } else { - /* Re-configure */ - void **models; - - /* Stop and unload all models */ - for (model_id = 0; model_id < dev->data->nb_models; model_id++) { - model = dev->data->models[model_id]; - if (model != NULL) { - if (model->state == ML_CNXK_MODEL_STATE_STARTED) { - if (cn10k_ml_model_stop(dev, model_id) != 0) - plt_err("Could not stop model %u", model_id); - } - if (model->state == ML_CNXK_MODEL_STATE_LOADED) { - if (cn10k_ml_model_unload(dev, model_id) != 0) - plt_err("Could not unload model %u", model_id); - } - dev->data->models[model_id] = NULL; - } - } - - models = dev->data->models; - models = rte_realloc(models, sizeof(models[0]) * conf->nb_models, - RTE_CACHE_LINE_SIZE); - if (models == NULL) { - dev->data->nb_models = 0; - plt_err("Failed to realloc ml_models, nb_models = %u", conf->nb_models); - ret = -ENOMEM; - goto error; - } - memset(models, 0, sizeof(models[0]) * conf->nb_models); - dev->data->models = models; - } - dev->data->nb_models = conf->nb_models; - ocm = &cn10k_mldev->ocm; ocm->num_tiles = ML_CN10K_OCM_NUMTILES; ocm->size_per_tile = ML_CN10K_OCM_TILESIZE; @@ -1038,8 +904,7 @@ cn10k_ml_dev_configure(struct rte_ml_dev *dev, const struct rte_ml_dev_config *c rte_zmalloc("ocm_mask", ocm->mask_words * ocm->num_tiles, RTE_CACHE_LINE_SIZE); if (ocm->ocm_mask == NULL) { plt_err("Unable to allocate memory for OCM mask"); - ret = -ENOMEM; - goto error; + return -ENOMEM; } for (tile_id = 0; tile_id < ocm->num_tiles; tile_id++) { @@ -1050,10 +915,10 @@ cn10k_ml_dev_configure(struct rte_ml_dev *dev, const struct rte_ml_dev_config *c rte_spinlock_init(&ocm->lock); /* Initialize xstats */ - ret = cn10k_ml_xstats_init(dev); + ret = cn10k_ml_xstats_init(cnxk_mldev->mldev); if (ret != 0) { plt_err("Failed to initialize xstats"); - goto error; + return ret; } /* Set JCMDQ enqueue function */ @@ -1067,77 +932,25 @@ cn10k_ml_dev_configure(struct rte_ml_dev *dev, const struct rte_ml_dev_config *c cn10k_mldev->set_poll_ptr = cn10k_ml_set_poll_ptr; cn10k_mldev->get_poll_ptr = cn10k_ml_get_poll_ptr; - dev->enqueue_burst = cn10k_ml_enqueue_burst; - dev->dequeue_burst = cn10k_ml_dequeue_burst; - dev->op_error_get = cn10k_ml_op_error_get; - - cnxk_mldev->nb_models_loaded = 0; - cnxk_mldev->nb_models_started = 0; - cnxk_mldev->nb_models_stopped = 0; - cnxk_mldev->nb_models_unloaded = 0; - cnxk_mldev->state = ML_CNXK_DEV_STATE_CONFIGURED; + cnxk_mldev->mldev->enqueue_burst = cn10k_ml_enqueue_burst; + cnxk_mldev->mldev->dequeue_burst = cn10k_ml_dequeue_burst; + cnxk_mldev->mldev->op_error_get = cn10k_ml_op_error_get; return 0; - -error: - rte_free(dev->data->queue_pairs); - - rte_free(dev->data->models); - - return ret; } int -cn10k_ml_dev_close(struct rte_ml_dev *dev) +cn10k_ml_dev_close(struct cnxk_ml_dev *cnxk_mldev) { struct cn10k_ml_dev *cn10k_mldev; - struct cnxk_ml_dev *cnxk_mldev; - struct cnxk_ml_model *model; - struct cnxk_ml_qp *qp; - uint16_t model_id; - uint16_t qp_id; - if (dev == NULL) - return -EINVAL; - - cnxk_mldev = dev->data->dev_private; cn10k_mldev = &cnxk_mldev->cn10k_mldev; /* Release ocm_mask memory */ rte_free(cn10k_mldev->ocm.ocm_mask); - /* Stop and unload all models */ - for (model_id = 0; model_id < dev->data->nb_models; model_id++) { - model = dev->data->models[model_id]; - if (model != NULL) { - if (model->state == ML_CNXK_MODEL_STATE_STARTED) { - if (cn10k_ml_model_stop(dev, model_id) != 0) - plt_err("Could not stop model %u", model_id); - } - if (model->state == ML_CNXK_MODEL_STATE_LOADED) { - if (cn10k_ml_model_unload(dev, model_id) != 0) - plt_err("Could not unload model %u", model_id); - } - dev->data->models[model_id] = NULL; - } - } - - rte_free(dev->data->models); - - /* Destroy all queue pairs */ - for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) { - qp = dev->data->queue_pairs[qp_id]; - if (qp != NULL) { - if (cnxk_ml_qp_destroy(dev, qp) != 0) - plt_err("Could not destroy queue pair %u", qp_id); - dev->data->queue_pairs[qp_id] = NULL; - } - } - - rte_free(dev->data->queue_pairs); - /* Un-initialize xstats */ - cn10k_ml_xstats_uninit(dev); + cn10k_ml_xstats_uninit(cnxk_mldev->mldev); /* Unload firmware */ cn10k_ml_fw_unload(cnxk_mldev); @@ -1154,20 +967,15 @@ cn10k_ml_dev_close(struct rte_ml_dev *dev) roc_ml_reg_write64(&cn10k_mldev->roc, 0, ML_MLR_BASE); plt_ml_dbg("ML_MLR_BASE = 0x%016lx", roc_ml_reg_read64(&cn10k_mldev->roc, ML_MLR_BASE)); - cnxk_mldev->state = ML_CNXK_DEV_STATE_CLOSED; - - /* Remove PCI device */ - return rte_dev_remove(dev->device); + return 0; } int -cn10k_ml_dev_start(struct rte_ml_dev *dev) +cn10k_ml_dev_start(struct cnxk_ml_dev *cnxk_mldev) { struct cn10k_ml_dev *cn10k_mldev; - struct cnxk_ml_dev *cnxk_mldev; uint64_t reg_val64; - cnxk_mldev = dev->data->dev_private; cn10k_mldev = &cnxk_mldev->cn10k_mldev; reg_val64 = roc_ml_reg_read64(&cn10k_mldev->roc, ML_CFG); @@ -1175,19 +983,15 @@ cn10k_ml_dev_start(struct rte_ml_dev *dev) roc_ml_reg_write64(&cn10k_mldev->roc, reg_val64, ML_CFG); plt_ml_dbg("ML_CFG => 0x%016lx", roc_ml_reg_read64(&cn10k_mldev->roc, ML_CFG)); - cnxk_mldev->state = ML_CNXK_DEV_STATE_STARTED; - return 0; } int -cn10k_ml_dev_stop(struct rte_ml_dev *dev) +cn10k_ml_dev_stop(struct cnxk_ml_dev *cnxk_mldev) { struct cn10k_ml_dev *cn10k_mldev; - struct cnxk_ml_dev *cnxk_mldev; uint64_t reg_val64; - cnxk_mldev = dev->data->dev_private; cn10k_mldev = &cnxk_mldev->cn10k_mldev; reg_val64 = roc_ml_reg_read64(&cn10k_mldev->roc, ML_CFG); @@ -1195,8 +999,6 @@ cn10k_ml_dev_stop(struct rte_ml_dev *dev) roc_ml_reg_write64(&cn10k_mldev->roc, reg_val64, ML_CFG); plt_ml_dbg("ML_CFG => 0x%016lx", roc_ml_reg_read64(&cn10k_mldev->roc, ML_CFG)); - cnxk_mldev->state = ML_CNXK_DEV_STATE_CONFIGURED; - return 0; } @@ -1217,7 +1019,7 @@ cn10k_ml_dev_queue_pair_setup(struct rte_ml_dev *dev, uint16_t queue_pair_id, if (dev->data->queue_pairs[queue_pair_id] != NULL) cn10k_ml_dev_queue_pair_release(dev, queue_pair_id); - cn10k_ml_dev_info_get(dev, &dev_info); + cnxk_ml_dev_info_get(dev, &dev_info); if ((qp_conf->nb_desc > dev_info.max_desc) || (qp_conf->nb_desc == 0)) { plt_err("Could not setup queue pair for %u descriptors", qp_conf->nb_desc); return -EINVAL; diff --git a/drivers/ml/cnxk/cn10k_ml_ops.h b/drivers/ml/cnxk/cn10k_ml_ops.h index 16480b9ad8..d50b5bede7 100644 --- a/drivers/ml/cnxk/cn10k_ml_ops.h +++ b/drivers/ml/cnxk/cn10k_ml_ops.h @@ -10,6 +10,9 @@ #include +struct cnxk_ml_dev; +struct cnxk_ml_qp; + /* Firmware version string length */ #define MLDEV_FIRMWARE_VERSION_LENGTH 32 @@ -286,11 +289,11 @@ struct cn10k_ml_req { }; /* Device ops */ -int cn10k_ml_dev_info_get(struct rte_ml_dev *dev, struct rte_ml_dev_info *dev_info); -int cn10k_ml_dev_configure(struct rte_ml_dev *dev, const struct rte_ml_dev_config *conf); -int cn10k_ml_dev_close(struct rte_ml_dev *dev); -int cn10k_ml_dev_start(struct rte_ml_dev *dev); -int cn10k_ml_dev_stop(struct rte_ml_dev *dev); +int cn10k_ml_dev_info_get(struct cnxk_ml_dev *cnxk_mldev, struct rte_ml_dev_info *dev_info); +int cn10k_ml_dev_configure(struct cnxk_ml_dev *cnxk_mldev, const struct rte_ml_dev_config *conf); +int cn10k_ml_dev_close(struct cnxk_ml_dev *cnxk_mldev); +int cn10k_ml_dev_start(struct cnxk_ml_dev *cnxk_mldev); +int cn10k_ml_dev_stop(struct cnxk_ml_dev *cnxk_mldev); int cn10k_ml_dev_dump(struct rte_ml_dev *dev, FILE *fp); int cn10k_ml_dev_selftest(struct rte_ml_dev *dev); int cn10k_ml_dev_queue_pair_setup(struct rte_ml_dev *dev, uint16_t queue_pair_id, @@ -336,4 +339,7 @@ __rte_hot int cn10k_ml_op_error_get(struct rte_ml_dev *dev, struct rte_ml_op *op struct rte_ml_op_error *error); __rte_hot int cn10k_ml_inference_sync(struct rte_ml_dev *dev, struct rte_ml_op *op); +/* Temporarily set below functions as non-static */ +int cnxk_ml_qp_destroy(const struct rte_ml_dev *dev, struct cnxk_ml_qp *qp); + #endif /* _CN10K_ML_OPS_H_ */ diff --git a/drivers/ml/cnxk/cnxk_ml_dev.h b/drivers/ml/cnxk/cnxk_ml_dev.h index 51315de622..02605fa28f 100644 --- a/drivers/ml/cnxk/cnxk_ml_dev.h +++ b/drivers/ml/cnxk/cnxk_ml_dev.h @@ -53,6 +53,9 @@ struct cnxk_ml_dev { /* CN10K device structure */ struct cn10k_ml_dev cn10k_mldev; + + /* Maximum number of layers */ + uint64_t max_nb_layers; }; #endif /* _CNXK_ML_DEV_H_ */ diff --git a/drivers/ml/cnxk/cnxk_ml_ops.c b/drivers/ml/cnxk/cnxk_ml_ops.c index 03402681c5..07a4daabc5 100644 --- a/drivers/ml/cnxk/cnxk_ml_ops.c +++ b/drivers/ml/cnxk/cnxk_ml_ops.c @@ -5,15 +5,291 @@ #include #include +#include "cnxk_ml_dev.h" +#include "cnxk_ml_io.h" +#include "cnxk_ml_model.h" #include "cnxk_ml_ops.h" +int +cnxk_ml_dev_info_get(struct rte_ml_dev *dev, struct rte_ml_dev_info *dev_info) +{ + struct cnxk_ml_dev *cnxk_mldev; + + if (dev == NULL || dev_info == NULL) + return -EINVAL; + + cnxk_mldev = dev->data->dev_private; + + memset(dev_info, 0, sizeof(struct rte_ml_dev_info)); + dev_info->driver_name = dev->device->driver->name; + dev_info->max_models = ML_CNXK_MAX_MODELS; + + return cn10k_ml_dev_info_get(cnxk_mldev, dev_info); +} + +static int +cnxk_ml_dev_configure(struct rte_ml_dev *dev, const struct rte_ml_dev_config *conf) +{ + struct rte_ml_dev_info dev_info; + struct cnxk_ml_dev *cnxk_mldev; + struct cnxk_ml_model *model; + struct cnxk_ml_qp *qp; + uint16_t model_id; + uint32_t mz_size; + uint16_t qp_id; + int ret; + + if (dev == NULL) + return -EINVAL; + + /* Get CNXK device handle */ + cnxk_mldev = dev->data->dev_private; + + cnxk_ml_dev_info_get(dev, &dev_info); + if (conf->nb_models > dev_info.max_models) { + plt_err("Invalid device config, nb_models > %u\n", dev_info.max_models); + return -EINVAL; + } + + if (conf->nb_queue_pairs > dev_info.max_queue_pairs) { + plt_err("Invalid device config, nb_queue_pairs > %u\n", dev_info.max_queue_pairs); + return -EINVAL; + } + + if (cnxk_mldev->state == ML_CNXK_DEV_STATE_PROBED) { + plt_ml_dbg("Configuring ML device, nb_queue_pairs = %u, nb_models = %u", + conf->nb_queue_pairs, conf->nb_models); + + /* Load firmware */ + ret = cn10k_ml_fw_load(cnxk_mldev); + if (ret != 0) + return ret; + } else if (cnxk_mldev->state == ML_CNXK_DEV_STATE_CONFIGURED) { + plt_ml_dbg("Re-configuring ML device, nb_queue_pairs = %u, nb_models = %u", + conf->nb_queue_pairs, conf->nb_models); + } else if (cnxk_mldev->state == ML_CNXK_DEV_STATE_STARTED) { + plt_err("Device can't be reconfigured in started state\n"); + return -ENOTSUP; + } else if (cnxk_mldev->state == ML_CNXK_DEV_STATE_CLOSED) { + plt_err("Device can't be reconfigured after close\n"); + return -ENOTSUP; + } + + /* Configure queue-pairs */ + if (dev->data->queue_pairs == NULL) { + mz_size = sizeof(dev->data->queue_pairs[0]) * conf->nb_queue_pairs; + dev->data->queue_pairs = + rte_zmalloc("cnxk_mldev_queue_pairs", mz_size, RTE_CACHE_LINE_SIZE); + if (dev->data->queue_pairs == NULL) { + dev->data->nb_queue_pairs = 0; + plt_err("Failed to get memory for queue_pairs, nb_queue_pairs %u", + conf->nb_queue_pairs); + return -ENOMEM; + } + } else { /* Re-configure */ + void **queue_pairs; + + /* Release all queue pairs as ML spec doesn't support queue_pair_destroy. */ + for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) { + qp = dev->data->queue_pairs[qp_id]; + if (qp != NULL) { + ret = cn10k_ml_dev_queue_pair_release(dev, qp_id); + if (ret < 0) + return ret; + } + } + + queue_pairs = dev->data->queue_pairs; + queue_pairs = + rte_realloc(queue_pairs, sizeof(queue_pairs[0]) * conf->nb_queue_pairs, + RTE_CACHE_LINE_SIZE); + if (queue_pairs == NULL) { + dev->data->nb_queue_pairs = 0; + plt_err("Failed to realloc queue_pairs, nb_queue_pairs = %u", + conf->nb_queue_pairs); + ret = -ENOMEM; + goto error; + } + + memset(queue_pairs, 0, sizeof(queue_pairs[0]) * conf->nb_queue_pairs); + dev->data->queue_pairs = queue_pairs; + } + dev->data->nb_queue_pairs = conf->nb_queue_pairs; + + /* Allocate ML models */ + if (dev->data->models == NULL) { + mz_size = sizeof(dev->data->models[0]) * conf->nb_models; + dev->data->models = rte_zmalloc("cnxk_mldev_models", mz_size, RTE_CACHE_LINE_SIZE); + if (dev->data->models == NULL) { + dev->data->nb_models = 0; + plt_err("Failed to get memory for ml_models, nb_models %u", + conf->nb_models); + ret = -ENOMEM; + goto error; + } + } else { + /* Re-configure */ + void **models; + + /* Stop and unload all models */ + for (model_id = 0; model_id < dev->data->nb_models; model_id++) { + model = dev->data->models[model_id]; + if (model != NULL) { + if (model->state == ML_CNXK_MODEL_STATE_STARTED) { + if (cn10k_ml_model_stop(dev, model_id) != 0) + plt_err("Could not stop model %u", model_id); + } + if (model->state == ML_CNXK_MODEL_STATE_LOADED) { + if (cn10k_ml_model_unload(dev, model_id) != 0) + plt_err("Could not unload model %u", model_id); + } + dev->data->models[model_id] = NULL; + } + } + + models = dev->data->models; + models = rte_realloc(models, sizeof(models[0]) * conf->nb_models, + RTE_CACHE_LINE_SIZE); + if (models == NULL) { + dev->data->nb_models = 0; + plt_err("Failed to realloc ml_models, nb_models = %u", conf->nb_models); + ret = -ENOMEM; + goto error; + } + memset(models, 0, sizeof(models[0]) * conf->nb_models); + dev->data->models = models; + } + dev->data->nb_models = conf->nb_models; + + ret = cn10k_ml_dev_configure(cnxk_mldev, conf); + if (ret != 0) { + plt_err("Failed to configure CN10K ML Device"); + goto error; + } + + /* Set device capabilities */ + cnxk_mldev->max_nb_layers = + cnxk_mldev->cn10k_mldev.fw.req->cn10k_req.jd.fw_load.cap.s.max_models; + + cnxk_mldev->nb_models_loaded = 0; + cnxk_mldev->nb_models_started = 0; + cnxk_mldev->nb_models_stopped = 0; + cnxk_mldev->nb_models_unloaded = 0; + cnxk_mldev->state = ML_CNXK_DEV_STATE_CONFIGURED; + + return 0; + +error: + rte_free(dev->data->queue_pairs); + rte_free(dev->data->models); + + return ret; +} + +static int +cnxk_ml_dev_close(struct rte_ml_dev *dev) +{ + struct cnxk_ml_dev *cnxk_mldev; + struct cnxk_ml_model *model; + struct cnxk_ml_qp *qp; + uint16_t model_id; + uint16_t qp_id; + + if (dev == NULL) + return -EINVAL; + + cnxk_mldev = dev->data->dev_private; + + if (cn10k_ml_dev_close(cnxk_mldev) != 0) + plt_err("Failed to close CN10K ML Device"); + + /* Stop and unload all models */ + for (model_id = 0; model_id < dev->data->nb_models; model_id++) { + model = dev->data->models[model_id]; + if (model != NULL) { + if (model->state == ML_CNXK_MODEL_STATE_STARTED) { + if (cn10k_ml_model_stop(dev, model_id) != 0) + plt_err("Could not stop model %u", model_id); + } + if (model->state == ML_CNXK_MODEL_STATE_LOADED) { + if (cn10k_ml_model_unload(dev, model_id) != 0) + plt_err("Could not unload model %u", model_id); + } + dev->data->models[model_id] = NULL; + } + } + + rte_free(dev->data->models); + + /* Destroy all queue pairs */ + for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) { + qp = dev->data->queue_pairs[qp_id]; + if (qp != NULL) { + if (cnxk_ml_qp_destroy(dev, qp) != 0) + plt_err("Could not destroy queue pair %u", qp_id); + dev->data->queue_pairs[qp_id] = NULL; + } + } + + rte_free(dev->data->queue_pairs); + + cnxk_mldev->state = ML_CNXK_DEV_STATE_CLOSED; + + /* Remove PCI device */ + return rte_dev_remove(dev->device); +} + +static int +cnxk_ml_dev_start(struct rte_ml_dev *dev) +{ + struct cnxk_ml_dev *cnxk_mldev; + int ret; + + if (dev == NULL) + return -EINVAL; + + cnxk_mldev = dev->data->dev_private; + + ret = cn10k_ml_dev_start(cnxk_mldev); + if (ret != 0) { + plt_err("Failed to start CN10K ML Device"); + return ret; + } + + cnxk_mldev->state = ML_CNXK_DEV_STATE_STARTED; + + return 0; +} + +static int +cnxk_ml_dev_stop(struct rte_ml_dev *dev) +{ + struct cnxk_ml_dev *cnxk_mldev; + int ret; + + if (dev == NULL) + return -EINVAL; + + cnxk_mldev = dev->data->dev_private; + + ret = cn10k_ml_dev_stop(cnxk_mldev); + if (ret != 0) { + plt_err("Failed to stop CN10K ML Device"); + return ret; + } + + cnxk_mldev->state = ML_CNXK_DEV_STATE_CONFIGURED; + + return 0; +} + struct rte_ml_dev_ops cnxk_ml_ops = { /* Device control ops */ - .dev_info_get = cn10k_ml_dev_info_get, - .dev_configure = cn10k_ml_dev_configure, - .dev_close = cn10k_ml_dev_close, - .dev_start = cn10k_ml_dev_start, - .dev_stop = cn10k_ml_dev_stop, + .dev_info_get = cnxk_ml_dev_info_get, + .dev_configure = cnxk_ml_dev_configure, + .dev_close = cnxk_ml_dev_close, + .dev_start = cnxk_ml_dev_start, + .dev_stop = cnxk_ml_dev_stop, .dev_dump = cn10k_ml_dev_dump, .dev_selftest = cn10k_ml_dev_selftest, diff --git a/drivers/ml/cnxk/cnxk_ml_ops.h b/drivers/ml/cnxk/cnxk_ml_ops.h index a925c07580..2996928d7d 100644 --- a/drivers/ml/cnxk/cnxk_ml_ops.h +++ b/drivers/ml/cnxk/cnxk_ml_ops.h @@ -62,4 +62,7 @@ struct cnxk_ml_qp { extern struct rte_ml_dev_ops cnxk_ml_ops; +/* Temporarily set cnxk driver functions as non-static */ +int cnxk_ml_dev_info_get(struct rte_ml_dev *dev, struct rte_ml_dev_info *dev_info); + #endif /* _CNXK_ML_OPS_H_ */