From patchwork Wed Jun 13 12:14:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Jozwiak X-Patchwork-Id: 41057 X-Patchwork-Delegate: pablo.de.lara.guarch@intel.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BBF5B1EFDF; Wed, 13 Jun 2018 14:15:32 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id DBEE11EF4A for ; Wed, 13 Jun 2018 14:15:02 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 13 Jun 2018 05:14:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,218,1526367600"; d="scan'208";a="63727764" Received: from tjozwiax-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.103.104.42]) by fmsmga001.fm.intel.com with ESMTP; 13 Jun 2018 05:14:55 -0700 From: Tomasz Jozwiak To: fiona.trahe@intel.com, tomaszx.jozwiak@intel.com, dev@dpdk.org Date: Wed, 13 Jun 2018 14:14:07 +0200 Message-Id: <1528892062-4997-24-git-send-email-tomaszx.jozwiak@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1528892062-4997-1-git-send-email-tomaszx.jozwiak@intel.com> References: <1523040732-3290-1-git-send-email-fiona.trahe@intel.com> <1528892062-4997-1-git-send-email-tomaszx.jozwiak@intel.com> Subject: [dpdk-dev] [PATCH v3 23/38] crypto/qat: move code into appropriate files X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Fiona Trahe Move all code into appropriate files, no actual code changes. Specifically: - Rename rte_qat_cryptodev.c to qat_sym_pmd.c - Create qat_sym_pmd.h and populate with fn prototypes for qat_sym_pmd.c - Create qat_comp_pmd.c/.h and populate with placeholder functions - Create qat_asym_pmd.c/.h and populate with placeholder functions - Rename qat_crypto_capabilities.h to qat_sym_capabilities.h - Move CRYPTODEV_NAME_QAT_SYM_PMD from qat_common.h to qat_sym_pmd.h - Move qat_sym_dev_private from qat_device.h to qat_sym_pmd.h - Move prototype for qat_sym_dev_info_get frm qat_device.h 2 qat_sym_pmd.h - Move all qat_device.c sym dev_ops fns to qat_sym_pmd.c file - Move all qat_sym.c dev_ops fns to qat_sym_pmd.c file - Remove unused header file #includes from all files. - Move pci_id_qat_map, probe/release/register from rte_qat_cryptodev.c to qat_device.c - Moved stray comment for bpi_cipher_ctx_init() from qat_sym.c to qat_sym_session.c - Changed all files to use SPDX license header Signed-off-by: Fiona Trahe --- drivers/crypto/qat/Makefile | 4 +- drivers/crypto/qat/meson.build | 11 +- drivers/crypto/qat/qat_asym_pmd.c | 17 + drivers/crypto/qat/qat_asym_pmd.h | 15 + drivers/crypto/qat/qat_common.c | 54 +++ drivers/crypto/qat/qat_common.h | 15 +- drivers/crypto/qat/qat_comp_pmd.c | 18 + drivers/crypto/qat/qat_comp_pmd.h | 29 ++ drivers/crypto/qat/qat_device.c | 141 +++++--- drivers/crypto/qat/qat_device.h | 31 +- drivers/crypto/qat/qat_qp.c | 2 +- drivers/crypto/qat/qat_qp.h | 9 +- drivers/crypto/qat/qat_sym.c | 246 ++----------- drivers/crypto/qat/qat_sym.h | 28 +- ..._capabilities.h => qat_sym_capabilities.h} | 6 +- drivers/crypto/qat/qat_sym_pmd.c | 326 ++++++++++++++++++ drivers/crypto/qat/qat_sym_pmd.h | 39 +++ drivers/crypto/qat/qat_sym_session.c | 17 +- drivers/crypto/qat/qat_sym_session.h | 2 +- drivers/crypto/qat/rte_qat_cryptodev.c | 258 -------------- 20 files changed, 665 insertions(+), 603 deletions(-) create mode 100644 drivers/crypto/qat/qat_asym_pmd.c create mode 100644 drivers/crypto/qat/qat_asym_pmd.h create mode 100644 drivers/crypto/qat/qat_comp_pmd.c create mode 100644 drivers/crypto/qat/qat_comp_pmd.h rename drivers/crypto/qat/{qat_crypto_capabilities.h => qat_sym_capabilities.h} (99%) create mode 100644 drivers/crypto/qat/qat_sym_pmd.c create mode 100644 drivers/crypto/qat/qat_sym_pmd.h delete mode 100644 drivers/crypto/qat/rte_qat_cryptodev.c diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile index 902c47ff4..d467683fd 100644 --- a/drivers/crypto/qat/Makefile +++ b/drivers/crypto/qat/Makefile @@ -26,7 +26,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_common.c -SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_pmd.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_asym_pmd.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_comp_pmd.c # export include files SYMLINK-y-include += diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build index 12910c377..e22e08fba 100644 --- a/drivers/crypto/qat/meson.build +++ b/drivers/crypto/qat/meson.build @@ -5,11 +5,12 @@ dep = dependency('libcrypto', required: false) if not dep.found() build = false endif -sources = files('qat_sym.c', 'qat_qp.c', - 'qat_sym_session.c', - 'qat_common.c', - 'rte_qat_cryptodev.c', - 'qat_device.c') +sources = files('qat_common.c', + 'qat_qp.c', + 'qat_device.c', + 'qat_sym_pmd.c', 'qat_sym.c', 'qat_sym_session.c', + 'qat_asym_pmd.c', + 'qat_comp_pmd.c') includes += include_directories('qat_adf') deps += ['bus_pci'] ext_deps += dep diff --git a/drivers/crypto/qat/qat_asym_pmd.c b/drivers/crypto/qat/qat_asym_pmd.c new file mode 100644 index 000000000..8d36300ac --- /dev/null +++ b/drivers/crypto/qat/qat_asym_pmd.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include "qat_asym_pmd.h" + +int +qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused) +{ + return 0; +} + +int +qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused) +{ + return 0; +} diff --git a/drivers/crypto/qat/qat_asym_pmd.h b/drivers/crypto/qat/qat_asym_pmd.h new file mode 100644 index 000000000..0465e0300 --- /dev/null +++ b/drivers/crypto/qat/qat_asym_pmd.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _QAT_ASYM_PMD_H_ +#define _QAT_ASYM_PMD_H_ + +#include "qat_device.h" + +int +qat_asym_dev_create(struct qat_pci_device *qat_pci_dev); + +int +qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev); +#endif /* _QAT_ASYM_PMD_H_ */ diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c index a8865904f..f1759ea76 100644 --- a/drivers/crypto/qat/qat_common.c +++ b/drivers/crypto/qat/qat_common.c @@ -3,6 +3,7 @@ */ #include "qat_common.h" +#include "qat_device.h" #include "qat_logs.h" int @@ -51,3 +52,56 @@ qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start, return 0; } + +void qat_stats_get(struct qat_pci_device *dev, + struct qat_common_stats *stats, + enum qat_service_type service) +{ + int i; + struct qat_qp **qp; + + if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) { + PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d", + stats, dev, service); + return; + } + + qp = dev->qps_in_use[service]; + for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) { + if (qp[i] == NULL) { + PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d", + service, i); + continue; + } + + stats->enqueued_count += qp[i]->stats.enqueued_count; + stats->dequeued_count += qp[i]->stats.dequeued_count; + stats->enqueue_err_count += qp[i]->stats.enqueue_err_count; + stats->dequeue_err_count += qp[i]->stats.dequeue_err_count; + } +} + +void qat_stats_reset(struct qat_pci_device *dev, + enum qat_service_type service) +{ + int i; + struct qat_qp **qp; + + if (dev == NULL || service >= QAT_SERVICE_INVALID) { + PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d", + dev, service); + return; + } + + qp = dev->qps_in_use[service]; + for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) { + if (qp[i] == NULL) { + PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d", + service, i); + continue; + } + memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats)); + } + + PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service); +} diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h index fcf5c4c09..8ecebe954 100644 --- a/drivers/crypto/qat/qat_common.h +++ b/drivers/crypto/qat/qat_common.h @@ -8,9 +8,6 @@ #include -/**< Intel(R) QAT Symmetric Crypto PMD device name */ -#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat - /**< Intel(R) QAT device name for PCI registration */ #define QAT_PCI_NAME qat /* @@ -21,10 +18,9 @@ /* Intel(R) QuickAssist Technology device generation is enumerated * from one according to the generation of the device */ - enum qat_device_gen { QAT_GEN1 = 1, - QAT_GEN2, + QAT_GEN2 }; enum qat_service_type { @@ -62,8 +58,17 @@ struct qat_common_stats { /**< Total error count on operations dequeued */ }; +struct qat_pci_device; + int qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start, struct qat_sgl *list, uint32_t data_len); +void +qat_stats_get(struct qat_pci_device *dev, + struct qat_common_stats *stats, + enum qat_service_type service); +void +qat_stats_reset(struct qat_pci_device *dev, + enum qat_service_type service); #endif /* _QAT_COMMON_H_ */ diff --git a/drivers/crypto/qat/qat_comp_pmd.c b/drivers/crypto/qat/qat_comp_pmd.c new file mode 100644 index 000000000..547b3db49 --- /dev/null +++ b/drivers/crypto/qat/qat_comp_pmd.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include "qat_comp_pmd.h" + + +int +qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused) +{ + return 0; +} + +int +qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused) +{ + return 0; +} diff --git a/drivers/crypto/qat/qat_comp_pmd.h b/drivers/crypto/qat/qat_comp_pmd.h new file mode 100644 index 000000000..cc31246c3 --- /dev/null +++ b/drivers/crypto/qat/qat_comp_pmd.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _QAT_COMP_PMD_H_ +#define _QAT_COMP_PMD_H_ + +#include "qat_device.h" + + +/**< Intel(R) QAT Compression PMD device name */ +#define COMPRESSDEV_NAME_QAT_PMD comp_qat + + +/** private data structure for a QAT compression device. + * This QAT device is a device offering only a compression service, + * there can be one of these on each qat_pci_device (VF). + */ +struct qat_comp_dev_private { + struct qat_pci_device *qat_dev; + /**< The qat pci device hosting the service */ +}; + +int +qat_comp_dev_create(struct qat_pci_device *qat_pci_dev); + +int +qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev); +#endif /* _QAT_COMP_PMD_H_ */ diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c index 8b2ac5a5f..c9d4b32ed 100644 --- a/drivers/crypto/qat/qat_device.c +++ b/drivers/crypto/qat/qat_device.c @@ -4,7 +4,9 @@ #include "qat_device.h" #include "adf_transport_access_macros.h" -#include "qat_qp.h" +#include "qat_sym_pmd.h" +#include "qat_asym_pmd.h" +#include "qat_comp_pmd.h" /* Hardware device information per generation */ __extension__ @@ -24,58 +26,25 @@ struct qat_gen_hw_data qp_gen_config[] = { static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES]; static int qat_nb_pci_devices; -int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev, - __rte_unused struct rte_cryptodev_config *config) -{ - PMD_INIT_FUNC_TRACE(); - return 0; -} - -int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev) -{ - PMD_INIT_FUNC_TRACE(); - return 0; -} - -void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev) -{ - PMD_INIT_FUNC_TRACE(); -} - -int qat_sym_dev_close(struct rte_cryptodev *dev) -{ - int i, ret; - - PMD_INIT_FUNC_TRACE(); - - for (i = 0; i < dev->data->nb_queue_pairs; i++) { - ret = qat_sym_qp_release(dev, i); - if (ret < 0) - return ret; - } - - return 0; -} +/* + * The set of PCI devices this driver supports + */ -void qat_sym_dev_info_get(struct rte_cryptodev *dev, - struct rte_cryptodev_info *info) -{ - struct qat_sym_dev_private *internals = dev->data->dev_private; - const struct qat_qp_hw_data *sym_hw_qps = - qp_gen_config[internals->qat_dev->qat_dev_gen] - .qp_hw_data[QAT_SERVICE_SYMMETRIC]; - - PMD_INIT_FUNC_TRACE(); - if (info != NULL) { - info->max_nb_queue_pairs = - qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC); - info->feature_flags = dev->feature_flags; - info->capabilities = internals->qat_dev_capabilities; - info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS; - info->driver_id = cryptodev_qat_driver_id; - info->pci_dev = RTE_DEV_TO_PCI(dev->device); - } -} +static const struct rte_pci_id pci_id_qat_map[] = { + { + RTE_PCI_DEVICE(0x8086, 0x0443), + }, + { + RTE_PCI_DEVICE(0x8086, 0x37c9), + }, + { + RTE_PCI_DEVICE(0x8086, 0x19e3), + }, + { + RTE_PCI_DEVICE(0x8086, 0x6f55), + }, + {.device_id = 0}, +}; static struct qat_pci_device * @@ -203,3 +172,71 @@ qat_pci_device_release(struct rte_pci_device *pci_dev) name, qat_nb_pci_devices); return 0; } + +static int +qat_pci_dev_destroy(struct qat_pci_device *qat_pci_dev, + struct rte_pci_device *pci_dev) +{ + qat_sym_dev_destroy(qat_pci_dev); + qat_comp_dev_destroy(qat_pci_dev); + qat_asym_dev_destroy(qat_pci_dev); + return qat_pci_device_release(pci_dev); +} + +static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) +{ + int ret = 0; + struct qat_pci_device *qat_pci_dev; + + PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x", + pci_dev->addr.bus, + pci_dev->addr.devid, + pci_dev->addr.function); + + qat_pci_dev = qat_pci_device_allocate(pci_dev); + if (qat_pci_dev == NULL) + return -ENODEV; + + ret = qat_sym_dev_create(qat_pci_dev); + if (ret != 0) + goto error_out; + + ret = qat_comp_dev_create(qat_pci_dev); + if (ret != 0) + goto error_out; + + ret = qat_asym_dev_create(qat_pci_dev); + if (ret != 0) + goto error_out; + + return 0; + +error_out: + qat_pci_dev_destroy(qat_pci_dev, pci_dev); + return ret; + +} + +static int qat_pci_remove(struct rte_pci_device *pci_dev) +{ + struct qat_pci_device *qat_pci_dev; + + if (pci_dev == NULL) + return -EINVAL; + + qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev); + if (qat_pci_dev == NULL) + return 0; + + return qat_pci_dev_destroy(qat_pci_dev, pci_dev); +} + +static struct rte_pci_driver rte_qat_pmd = { + .id_table = pci_id_qat_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING, + .probe = qat_pci_probe, + .remove = qat_pci_remove +}; +RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd); +RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map); diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h index 5424a9a94..fd1819354 100644 --- a/drivers/crypto/qat/qat_device.h +++ b/drivers/crypto/qat/qat_device.h @@ -4,8 +4,8 @@ #ifndef _QAT_DEVICE_H_ #define _QAT_DEVICE_H_ -#include #include + #include "qat_common.h" #include "qat_logs.h" #include "adf_transport_access_macros.h" @@ -18,12 +18,6 @@ #define QAT_MAX_PCI_DEVICES 48 #define QAT_DEV_NAME_MAX_LEN 64 - -extern uint8_t cryptodev_qat_driver_id; - -extern int qat_sym_qp_release(struct rte_cryptodev *dev, - uint16_t queue_pair_id); - /* * This struct holds all the data about a QAT pci device * including data about all services it supports. @@ -63,20 +57,6 @@ struct qat_pci_device { }; -/** private data structure for a QAT device. - * This QAT device is a device offering only symmetric crypto service, - * there can be one of these on each qat_pci_device (VF), - * in future there may also be private data structures for other services. - */ -struct qat_sym_dev_private { - struct qat_pci_device *qat_dev; - /**< The qat pci device hosting the service */ - uint8_t sym_dev_id; - /**< Device instance for this rte_cryptodev */ - const struct rte_cryptodev_capabilities *qat_dev_capabilities; - /* QAT device symmetric crypto capabilities */ -}; - struct qat_gen_hw_data { enum qat_device_gen dev_gen; const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_PER_BUNDLE]; @@ -84,14 +64,6 @@ struct qat_gen_hw_data { extern struct qat_gen_hw_data qp_gen_config[]; -int qat_sym_dev_config(struct rte_cryptodev *dev, - struct rte_cryptodev_config *config); -int qat_sym_dev_start(struct rte_cryptodev *dev); -void qat_sym_dev_stop(struct rte_cryptodev *dev); -int qat_sym_dev_close(struct rte_cryptodev *dev); -void qat_sym_dev_info_get(struct rte_cryptodev *dev, - struct rte_cryptodev_info *info); - struct qat_pci_device * qat_pci_device_allocate(struct rte_pci_device *pci_dev); int @@ -99,4 +71,5 @@ qat_pci_device_release(struct rte_pci_device *pci_dev); struct qat_pci_device * qat_get_qat_dev_from_pci_dev(struct rte_pci_device *pci_dev); + #endif /* _QAT_DEVICE_H_ */ diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c index 869140fc0..7b2dc3f90 100644 --- a/drivers/crypto/qat/qat_qp.c +++ b/drivers/crypto/qat/qat_qp.c @@ -12,8 +12,8 @@ #include #include "qat_logs.h" -#include "qat_qp.h" #include "qat_device.h" +#include "qat_qp.h" #include "adf_transport_access_macros.h" diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h index 49d9f29d3..73888b805 100644 --- a/drivers/crypto/qat/qat_qp.h +++ b/drivers/crypto/qat/qat_qp.h @@ -5,9 +5,10 @@ #define _QAT_QP_H_ #include "qat_common.h" -#include #include "adf_transport_access_macros.h" +struct qat_pci_device; + #define QAT_CSR_HEAD_WRITE_THRESH 32U /* number of requests to accumulate before writing head CSR */ #define QAT_CSR_TAIL_WRITE_THRESH 32U @@ -76,9 +77,9 @@ struct qat_queue { struct qat_qp { void *mmap_bar_addr; uint16_t inflights16; - struct qat_queue tx_q; - struct qat_queue rx_q; - struct qat_common_stats stats; + struct qat_queue tx_q; + struct qat_queue rx_q; + struct qat_common_stats stats; struct rte_mempool *op_cookie_pool; void **op_cookies; uint32_t nb_descriptors; diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c index 8007e25d6..15244d113 100644 --- a/drivers/crypto/qat/qat_sym.c +++ b/drivers/crypto/qat/qat_sym.c @@ -2,6 +2,8 @@ * Copyright(c) 2015-2018 Intel Corporation */ +#include + #include #include #include @@ -9,14 +11,10 @@ #include #include -#include - #include "qat_logs.h" #include "qat_sym_session.h" #include "qat_sym.h" -#include "qat_qp.h" -#include "adf_transport_access_macros.h" -#include "qat_device.h" +#include "qat_sym_pmd.h" #define BYTE_LENGTH 8 /* bpi is only used for partial blocks of DES and AES @@ -82,9 +80,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst, return -EINVAL; } -/** Creates a context in either AES or DES in ECB mode - * Depends on openssl libcrypto - */ static inline uint32_t qat_bpicipher_preprocess(struct qat_sym_session *ctx, @@ -197,57 +192,6 @@ qat_bpicipher_postprocess(struct qat_sym_session *ctx, return sym_op->cipher.data.length - last_block_len; } -uint16_t -qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops, - uint16_t nb_ops) -{ - return qat_enqueue_op_burst(qp, (void **)ops, nb_ops); -} - -static int -qat_sym_process_response(void **op, uint8_t *resp, - __rte_unused void *op_cookie, - __rte_unused enum qat_device_gen qat_dev_gen) -{ - - struct icp_qat_fw_comn_resp *resp_msg = - (struct icp_qat_fw_comn_resp *)resp; - struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t) - (resp_msg->opaque_data); - -#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX - rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg, - sizeof(struct icp_qat_fw_comn_resp)); -#endif - - if (ICP_QAT_FW_COMN_STATUS_FLAG_OK != - ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET( - resp_msg->comn_hdr.comn_status)) { - - rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; - } else { - struct qat_sym_session *sess = (struct qat_sym_session *) - get_session_private_data( - rx_op->sym->session, - cryptodev_qat_driver_id); - - if (sess->bpi_ctx) - qat_bpicipher_postprocess(sess, rx_op); - rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS; - } - *op = (void *)rx_op; - - return 0; -} - - -uint16_t -qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops, - uint16_t nb_ops) -{ - return qat_dequeue_op_burst(qp, (void **)ops, nb_ops); -} - static inline void set_cipher_iv(uint16_t iv_length, uint16_t iv_offset, struct icp_qat_fw_la_cipher_req_params *cipher_param, @@ -293,7 +237,7 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset, iv_length); } -static int +int qat_sym_build_request(void *in_op, uint8_t *out_msg, void *op_cookie, enum qat_device_gen qat_dev_gen) { @@ -716,168 +660,38 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg, return 0; } - -static void qat_stats_get(struct qat_pci_device *dev, - struct qat_common_stats *stats, - enum qat_service_type service) -{ - int i; - struct qat_qp **qp; - - if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) { - PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d", - stats, dev, service); - return; - } - - qp = dev->qps_in_use[service]; - for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) { - if (qp[i] == NULL) { - PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d", - service, i); - continue; - } - - stats->enqueued_count += qp[i]->stats.enqueued_count; - stats->dequeued_count += qp[i]->stats.dequeued_count; - stats->enqueue_err_count += qp[i]->stats.enqueue_err_count; - stats->dequeue_err_count += qp[i]->stats.dequeue_err_count; - } -} - -void qat_sym_stats_get(struct rte_cryptodev *dev, - struct rte_cryptodev_stats *stats) -{ - struct qat_common_stats qat_stats = {0}; - struct qat_sym_dev_private *qat_priv; - - if (stats == NULL || dev == NULL) { - PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev); - return; - } - qat_priv = dev->data->dev_private; - - qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC); - stats->enqueued_count = qat_stats.enqueued_count; - stats->dequeued_count = qat_stats.dequeued_count; - stats->enqueue_err_count = qat_stats.enqueue_err_count; - stats->dequeue_err_count = qat_stats.dequeue_err_count; -} - -static void qat_stats_reset(struct qat_pci_device *dev, - enum qat_service_type service) -{ - int i; - struct qat_qp **qp; - - if (dev == NULL || service >= QAT_SERVICE_INVALID) { - PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d", - dev, service); - return; - } - - qp = dev->qps_in_use[service]; - for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) { - if (qp[i] == NULL) { - PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d", - service, i); - continue; - } - memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats)); - } - - PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service); -} - -void qat_sym_stats_reset(struct rte_cryptodev *dev) -{ - struct qat_sym_dev_private *qat_priv; - - if (dev == NULL) { - PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev); - return; - } - qat_priv = dev->data->dev_private; - - qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC); - -} - -int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id) +int +qat_sym_process_response(void **op, uint8_t *resp, + __rte_unused void *op_cookie, + __rte_unused enum qat_device_gen qat_dev_gen) { - struct qat_sym_dev_private *qat_private = dev->data->dev_private; - PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d", - queue_pair_id, dev->data->dev_id); - - qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id] - = NULL; - - return qat_qp_release((struct qat_qp **) - &(dev->data->queue_pairs[queue_pair_id])); -} - -int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, - const struct rte_cryptodev_qp_conf *qp_conf, - int socket_id, struct rte_mempool *session_pool __rte_unused) -{ - struct qat_qp *qp; - int ret = 0; - uint32_t i; - struct qat_qp_config qat_qp_conf; - - struct qat_qp **qp_addr = - (struct qat_qp **)&(dev->data->queue_pairs[qp_id]); - struct qat_sym_dev_private *qat_private = dev->data->dev_private; - const struct qat_qp_hw_data *sym_hw_qps = - qp_gen_config[qat_private->qat_dev->qat_dev_gen] - .qp_hw_data[QAT_SERVICE_SYMMETRIC]; - const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id; - - /* If qp is already in use free ring memory and qp metadata. */ - if (*qp_addr != NULL) { - ret = qat_sym_qp_release(dev, qp_id); - if (ret < 0) - return ret; - } - if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) { - PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id); - return -EINVAL; - } - - qat_qp_conf.hw = qp_hw_data; - qat_qp_conf.build_request = qat_sym_build_request; - qat_qp_conf.process_response = qat_sym_process_response; - qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie); - qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors; - qat_qp_conf.socket_id = socket_id; - qat_qp_conf.service_str = "sym"; - - ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf); - if (ret != 0) - return ret; - - /* store a link to the qp in the qat_pci_device */ - qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id] - = *qp_addr; - - qp = (struct qat_qp *)*qp_addr; + struct icp_qat_fw_comn_resp *resp_msg = + (struct icp_qat_fw_comn_resp *)resp; + struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t) + (resp_msg->opaque_data); - for (i = 0; i < qp->nb_descriptors; i++) { +#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX + rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg, + sizeof(struct icp_qat_fw_comn_resp)); +#endif - struct qat_sym_op_cookie *sql_cookie = - qp->op_cookies[i]; + if (ICP_QAT_FW_COMN_STATUS_FLAG_OK != + ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET( + resp_msg->comn_hdr.comn_status)) { - sql_cookie->qat_sgl_src_phys_addr = - rte_mempool_virt2iova(sql_cookie) + - offsetof(struct qat_sym_op_cookie, - qat_sgl_src); + rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; + } else { + struct qat_sym_session *sess = (struct qat_sym_session *) + get_session_private_data( + rx_op->sym->session, + cryptodev_qat_driver_id); - sql_cookie->qat_sgl_dst_phys_addr = - rte_mempool_virt2iova(sql_cookie) + - offsetof(struct qat_sym_op_cookie, - qat_sgl_dst); + if (sess->bpi_ctx) + qat_bpicipher_postprocess(sess, rx_op); + rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS; } + *op = (void *)rx_op; - return ret; + return 0; } diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h index 78b40e378..d887dc126 100644 --- a/drivers/crypto/qat/qat_sym.h +++ b/drivers/crypto/qat/qat_sym.h @@ -6,11 +6,8 @@ #define _QAT_SYM_H_ #include -#include #include "qat_common.h" -#include "qat_device.h" -#include "qat_crypto_capabilities.h" /* * This macro rounds up a number to a be a multiple of @@ -29,23 +26,12 @@ struct qat_sym_op_cookie { phys_addr_t qat_sgl_dst_phys_addr; }; -void qat_sym_stats_get(struct rte_cryptodev *dev, - struct rte_cryptodev_stats *stats); -void qat_sym_stats_reset(struct rte_cryptodev *dev); - -int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id, - const struct rte_cryptodev_qp_conf *rx_conf, int socket_id, - struct rte_mempool *session_pool); -int qat_sym_qp_release(struct rte_cryptodev *dev, - uint16_t queue_pair_id); - - -uint16_t -qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops, - uint16_t nb_ops); - -uint16_t -qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops, - uint16_t nb_ops); +int +qat_sym_build_request(void *in_op, uint8_t *out_msg, + void *op_cookie, enum qat_device_gen qat_dev_gen); +int +qat_sym_process_response(void **op, uint8_t *resp, + __rte_unused void *op_cookie, + __rte_unused enum qat_device_gen qat_dev_gen); #endif /* _QAT_SYM_H_ */ diff --git a/drivers/crypto/qat/qat_crypto_capabilities.h b/drivers/crypto/qat/qat_sym_capabilities.h similarity index 99% rename from drivers/crypto/qat/qat_crypto_capabilities.h rename to drivers/crypto/qat/qat_sym_capabilities.h index 001c32c5d..d10a95ecb 100644 --- a/drivers/crypto/qat/qat_crypto_capabilities.h +++ b/drivers/crypto/qat/qat_sym_capabilities.h @@ -2,8 +2,8 @@ * Copyright(c) 2017-2018 Intel Corporation */ -#ifndef _QAT_CRYPTO_CAPABILITIES_H_ -#define _QAT_CRYPTO_CAPABILITIES_H_ +#ifndef _QAT_SYM_CAPABILITIES_H_ +#define _QAT_SYM_CAPABILITIES_H_ #define QAT_BASE_GEN1_SYM_CAPABILITIES \ { /* SHA1 HMAC */ \ @@ -554,4 +554,4 @@ }, } \ } -#endif /* _QAT_CRYPTO_CAPABILITIES_H_ */ +#endif /* _QAT_SYM_CAPABILITIES_H_ */ diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c new file mode 100644 index 000000000..aa71b4641 --- /dev/null +++ b/drivers/crypto/qat/qat_sym_pmd.c @@ -0,0 +1,326 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2015-2018 Intel Corporation + */ + +#include +#include +#include +#include +#include +#include + +#include "qat_logs.h" +#include "qat_sym.h" +#include "qat_sym_session.h" +#include "qat_sym_pmd.h" + +uint8_t cryptodev_qat_driver_id; + +static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = { + QAT_BASE_GEN1_SYM_CAPABILITIES, + RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() +}; + +static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = { + QAT_BASE_GEN1_SYM_CAPABILITIES, + QAT_EXTRA_GEN2_SYM_CAPABILITIES, + RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() +}; + +static int qat_sym_qp_release(struct rte_cryptodev *dev, + uint16_t queue_pair_id); + +static int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev, + __rte_unused struct rte_cryptodev_config *config) +{ + PMD_INIT_FUNC_TRACE(); + return 0; +} + +static int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev) +{ + PMD_INIT_FUNC_TRACE(); + return 0; +} + +static void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev) +{ + PMD_INIT_FUNC_TRACE(); +} + +static int qat_sym_dev_close(struct rte_cryptodev *dev) +{ + int i, ret; + + PMD_INIT_FUNC_TRACE(); + + for (i = 0; i < dev->data->nb_queue_pairs; i++) { + ret = qat_sym_qp_release(dev, i); + if (ret < 0) + return ret; + } + + return 0; +} + +static void qat_sym_dev_info_get(struct rte_cryptodev *dev, + struct rte_cryptodev_info *info) +{ + struct qat_sym_dev_private *internals = dev->data->dev_private; + const struct qat_qp_hw_data *sym_hw_qps = + qp_gen_config[internals->qat_dev->qat_dev_gen] + .qp_hw_data[QAT_SERVICE_SYMMETRIC]; + + PMD_INIT_FUNC_TRACE(); + if (info != NULL) { + info->max_nb_queue_pairs = + qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC); + info->feature_flags = dev->feature_flags; + info->capabilities = internals->qat_dev_capabilities; + info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS; + info->driver_id = cryptodev_qat_driver_id; + info->pci_dev = RTE_DEV_TO_PCI(dev->device); + } +} + +static void qat_sym_stats_get(struct rte_cryptodev *dev, + struct rte_cryptodev_stats *stats) +{ + struct qat_common_stats qat_stats = {0}; + struct qat_sym_dev_private *qat_priv; + + if (stats == NULL || dev == NULL) { + PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev); + return; + } + qat_priv = dev->data->dev_private; + + qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC); + stats->enqueued_count = qat_stats.enqueued_count; + stats->dequeued_count = qat_stats.dequeued_count; + stats->enqueue_err_count = qat_stats.enqueue_err_count; + stats->dequeue_err_count = qat_stats.dequeue_err_count; +} + +static void qat_sym_stats_reset(struct rte_cryptodev *dev) +{ + struct qat_sym_dev_private *qat_priv; + + if (dev == NULL) { + PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev); + return; + } + qat_priv = dev->data->dev_private; + + qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC); + +} + +static int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id) +{ + struct qat_sym_dev_private *qat_private = dev->data->dev_private; + + PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d", + queue_pair_id, dev->data->dev_id); + + qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id] + = NULL; + + return qat_qp_release((struct qat_qp **) + &(dev->data->queue_pairs[queue_pair_id])); +} + +static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, + const struct rte_cryptodev_qp_conf *qp_conf, + int socket_id, struct rte_mempool *session_pool __rte_unused) +{ + struct qat_qp *qp; + int ret = 0; + uint32_t i; + struct qat_qp_config qat_qp_conf; + + struct qat_qp **qp_addr = + (struct qat_qp **)&(dev->data->queue_pairs[qp_id]); + struct qat_sym_dev_private *qat_private = dev->data->dev_private; + const struct qat_qp_hw_data *sym_hw_qps = + qp_gen_config[qat_private->qat_dev->qat_dev_gen] + .qp_hw_data[QAT_SERVICE_SYMMETRIC]; + const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id; + + /* If qp is already in use free ring memory and qp metadata. */ + if (*qp_addr != NULL) { + ret = qat_sym_qp_release(dev, qp_id); + if (ret < 0) + return ret; + } + if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) { + PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id); + return -EINVAL; + } + + qat_qp_conf.hw = qp_hw_data; + qat_qp_conf.build_request = qat_sym_build_request; + qat_qp_conf.process_response = qat_sym_process_response; + qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie); + qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors; + qat_qp_conf.socket_id = socket_id; + qat_qp_conf.service_str = "sym"; + + ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf); + if (ret != 0) + return ret; + + /* store a link to the qp in the qat_pci_device */ + qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id] + = *qp_addr; + + qp = (struct qat_qp *)*qp_addr; + + for (i = 0; i < qp->nb_descriptors; i++) { + + struct qat_sym_op_cookie *sql_cookie = + qp->op_cookies[i]; + + sql_cookie->qat_sgl_src_phys_addr = + rte_mempool_virt2iova(sql_cookie) + + offsetof(struct qat_sym_op_cookie, + qat_sgl_src); + + sql_cookie->qat_sgl_dst_phys_addr = + rte_mempool_virt2iova(sql_cookie) + + offsetof(struct qat_sym_op_cookie, + qat_sgl_dst); + } + + return ret; +} + +static struct rte_cryptodev_ops crypto_qat_ops = { + + /* Device related operations */ + .dev_configure = qat_sym_dev_config, + .dev_start = qat_sym_dev_start, + .dev_stop = qat_sym_dev_stop, + .dev_close = qat_sym_dev_close, + .dev_infos_get = qat_sym_dev_info_get, + + .stats_get = qat_sym_stats_get, + .stats_reset = qat_sym_stats_reset, + .queue_pair_setup = qat_sym_qp_setup, + .queue_pair_release = qat_sym_qp_release, + .queue_pair_start = NULL, + .queue_pair_stop = NULL, + .queue_pair_count = NULL, + + /* Crypto related operations */ + .session_get_size = qat_sym_session_get_private_size, + .session_configure = qat_sym_session_configure, + .session_clear = qat_sym_session_clear +}; + +static uint16_t +qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + return qat_enqueue_op_burst(qp, (void **)ops, nb_ops); +} + +static uint16_t +qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + return qat_dequeue_op_burst(qp, (void **)ops, nb_ops); +} + +int +qat_sym_dev_create(struct qat_pci_device *qat_pci_dev) +{ + struct rte_cryptodev_pmd_init_params init_params = { + .name = "", + .socket_id = qat_pci_dev->pci_dev->device.numa_node, + .private_data_size = sizeof(struct qat_sym_dev_private), + .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS + }; + char name[RTE_CRYPTODEV_NAME_MAX_LEN]; + struct rte_cryptodev *cryptodev; + struct qat_sym_dev_private *internals; + + snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s", + qat_pci_dev->name, "sym"); + PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name); + + cryptodev = rte_cryptodev_pmd_create(name, + &qat_pci_dev->pci_dev->device, &init_params); + + if (cryptodev == NULL) + return -ENODEV; + + cryptodev->driver_id = cryptodev_qat_driver_id; + cryptodev->dev_ops = &crypto_qat_ops; + + cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst; + cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst; + + cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | + RTE_CRYPTODEV_FF_HW_ACCELERATED | + RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | + RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER; + + internals = cryptodev->data->dev_private; + internals->qat_dev = qat_pci_dev; + qat_pci_dev->sym_dev = internals; + + internals->sym_dev_id = cryptodev->data->dev_id; + switch (qat_pci_dev->qat_dev_gen) { + case QAT_GEN1: + internals->qat_dev_capabilities = qat_gen1_sym_capabilities; + break; + case QAT_GEN2: + internals->qat_dev_capabilities = qat_gen2_sym_capabilities; + break; + default: + internals->qat_dev_capabilities = qat_gen2_sym_capabilities; + PMD_DRV_LOG(DEBUG, + "QAT gen %d capabilities unknown, default to GEN2", + qat_pci_dev->qat_dev_gen); + break; + } + + PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d", + name, internals->sym_dev_id); + return 0; +} + +int +qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev) +{ + struct rte_cryptodev *cryptodev; + + if (qat_pci_dev == NULL) + return -ENODEV; + if (qat_pci_dev->sym_dev == NULL) + return 0; + + /* free crypto device */ + cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id); + rte_cryptodev_pmd_destroy(cryptodev); + qat_pci_dev->sym_dev = NULL; + + return 0; +} + + +/* An rte_driver is needed in the registration of both the device and the driver + * with cryptodev. + * The actual qat pci's rte_driver can't be used as its name represents + * the whole pci device with all services. Think of this as a holder for a name + * for the crypto part of the pci device. + */ +static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD); +static struct rte_driver cryptodev_qat_sym_driver = { + .name = qat_sym_drv_name, + .alias = qat_sym_drv_name +}; +static struct cryptodev_driver qat_crypto_drv; +RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver, + cryptodev_qat_driver_id); diff --git a/drivers/crypto/qat/qat_sym_pmd.h b/drivers/crypto/qat/qat_sym_pmd.h new file mode 100644 index 000000000..efa3b0775 --- /dev/null +++ b/drivers/crypto/qat/qat_sym_pmd.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2015-2018 Intel Corporation + */ + +#ifndef _QAT_SYM_PMD_H_ +#define _QAT_SYM_PMD_H_ + +#include + +#include "qat_sym_capabilities.h" +#include "qat_device.h" + + +/**< Intel(R) QAT Symmetric Crypto PMD device name */ +#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat + +extern uint8_t cryptodev_qat_driver_id; + +/** private data structure for a QAT device. + * This QAT device is a device offering only symmetric crypto service, + * there can be one of these on each qat_pci_device (VF), + * in future there may also be private data structures for other services. + */ +struct qat_sym_dev_private { + struct qat_pci_device *qat_dev; + /**< The qat pci device hosting the service */ + uint8_t sym_dev_id; + /**< Device instance for this rte_cryptodev */ + const struct rte_cryptodev_capabilities *qat_dev_capabilities; + /* QAT device symmetric crypto capabilities */ +}; + + +int +qat_sym_dev_create(struct qat_pci_device *qat_pci_dev); + +int +qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev); +#endif /* _QAT_SYM_PMD_H_ */ diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c index 68d7773a2..689596d51 100644 --- a/drivers/crypto/qat/qat_sym_session.c +++ b/drivers/crypto/qat/qat_sym_session.c @@ -1,6 +1,12 @@ /* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0) * Copyright(c) 2015-2018 Intel Corporation */ + +#include /* Needed to calculate pre-compute values */ +#include /* Needed to calculate pre-compute values */ +#include /* Needed to calculate pre-compute values */ +#include /* Needed for bpi runt block processing */ + #include #include #include @@ -10,14 +16,8 @@ #include #include "qat_logs.h" -#include "qat_device.h" - -#include /* Needed to calculate pre-compute values */ -#include /* Needed to calculate pre-compute values */ -#include /* Needed to calculate pre-compute values */ -#include - #include "qat_sym_session.h" +#include "qat_sym_pmd.h" /** Frees a context previously created * Depends on openssl libcrypto @@ -29,6 +29,9 @@ bpi_cipher_ctx_free(void *bpi_ctx) EVP_CIPHER_CTX_free((EVP_CIPHER_CTX *)bpi_ctx); } +/** Creates a context in either AES or DES in ECB mode + * Depends on openssl libcrypto + */ static int bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo, enum rte_crypto_cipher_operation direction __rte_unused, diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h index d3d27ff46..18d77e990 100644 --- a/drivers/crypto/qat/qat_sym_session.h +++ b/drivers/crypto/qat/qat_sym_session.h @@ -1,5 +1,5 @@ /* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation + * Copyright(c) 2015-2018 Intel Corporation */ #ifndef _QAT_SYM_SESSION_H_ #define _QAT_SYM_SESSION_H_ diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c deleted file mode 100644 index 91bb1e590..000000000 --- a/drivers/crypto/qat/rte_qat_cryptodev.c +++ /dev/null @@ -1,258 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2015-2018 Intel Corporation - */ - -#include -#include -#include -#include -#include -#include - -#include "qat_sym.h" -#include "qat_sym_session.h" -#include "qat_logs.h" - -uint8_t cryptodev_qat_driver_id; - -static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = { - QAT_BASE_GEN1_SYM_CAPABILITIES, - RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() -}; - -static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = { - QAT_BASE_GEN1_SYM_CAPABILITIES, - QAT_EXTRA_GEN2_SYM_CAPABILITIES, - RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() -}; - -static struct rte_cryptodev_ops crypto_qat_ops = { - - /* Device related operations */ - .dev_configure = qat_sym_dev_config, - .dev_start = qat_sym_dev_start, - .dev_stop = qat_sym_dev_stop, - .dev_close = qat_sym_dev_close, - .dev_infos_get = qat_sym_dev_info_get, - - .stats_get = qat_sym_stats_get, - .stats_reset = qat_sym_stats_reset, - .queue_pair_setup = qat_sym_qp_setup, - .queue_pair_release = qat_sym_qp_release, - .queue_pair_start = NULL, - .queue_pair_stop = NULL, - .queue_pair_count = NULL, - - /* Crypto related operations */ - .session_get_size = qat_sym_session_get_private_size, - .session_configure = qat_sym_session_configure, - .session_clear = qat_sym_session_clear -}; - -/* - * The set of PCI devices this driver supports - */ - -static const struct rte_pci_id pci_id_qat_map[] = { - { - RTE_PCI_DEVICE(0x8086, 0x0443), - }, - { - RTE_PCI_DEVICE(0x8086, 0x37c9), - }, - { - RTE_PCI_DEVICE(0x8086, 0x19e3), - }, - { - RTE_PCI_DEVICE(0x8086, 0x6f55), - }, - {.device_id = 0}, -}; - - - -static int -qat_sym_dev_create(struct qat_pci_device *qat_pci_dev) -{ - struct rte_cryptodev_pmd_init_params init_params = { - .name = "", - .socket_id = qat_pci_dev->pci_dev->device.numa_node, - .private_data_size = sizeof(struct qat_sym_dev_private), - .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS - }; - char name[RTE_CRYPTODEV_NAME_MAX_LEN]; - struct rte_cryptodev *cryptodev; - struct qat_sym_dev_private *internals; - - snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s", - qat_pci_dev->name, "sym"); - PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name); - - cryptodev = rte_cryptodev_pmd_create(name, - &qat_pci_dev->pci_dev->device, &init_params); - - if (cryptodev == NULL) - return -ENODEV; - - cryptodev->driver_id = cryptodev_qat_driver_id; - cryptodev->dev_ops = &crypto_qat_ops; - - cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst; - cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst; - - cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | - RTE_CRYPTODEV_FF_HW_ACCELERATED | - RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | - RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER; - - internals = cryptodev->data->dev_private; - internals->qat_dev = qat_pci_dev; - qat_pci_dev->sym_dev = internals; - - internals->sym_dev_id = cryptodev->data->dev_id; - switch (qat_pci_dev->qat_dev_gen) { - case QAT_GEN1: - internals->qat_dev_capabilities = qat_gen1_sym_capabilities; - break; - case QAT_GEN2: - internals->qat_dev_capabilities = qat_gen2_sym_capabilities; - break; - default: - internals->qat_dev_capabilities = qat_gen2_sym_capabilities; - PMD_DRV_LOG(DEBUG, - "QAT gen %d capabilities unknown, default to GEN2", - qat_pci_dev->qat_dev_gen); - break; - } - - PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d", - name, internals->sym_dev_id); - return 0; -} - -static int -qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev) -{ - struct rte_cryptodev *cryptodev; - - if (qat_pci_dev == NULL) - return -ENODEV; - if (qat_pci_dev->sym_dev == NULL) - return 0; - - /* free crypto device */ - cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id); - rte_cryptodev_pmd_destroy(cryptodev); - qat_pci_dev->sym_dev = NULL; - - return 0; -} - -static int -qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused) -{ - return 0; -} - -static int -qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused) -{ - return 0; -} - -static int -qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused) -{ - return 0; -} - -static int -qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused) -{ - return 0; -} - -static int -qat_pci_dev_destroy(struct qat_pci_device *qat_pci_dev, - struct rte_pci_device *pci_dev) -{ - qat_sym_dev_destroy(qat_pci_dev); - qat_comp_dev_destroy(qat_pci_dev); - qat_asym_dev_destroy(qat_pci_dev); - return qat_pci_device_release(pci_dev); -} - -static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) -{ - int ret = 0; - struct qat_pci_device *qat_pci_dev; - - PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x", - pci_dev->addr.bus, - pci_dev->addr.devid, - pci_dev->addr.function); - - qat_pci_dev = qat_pci_device_allocate(pci_dev); - if (qat_pci_dev == NULL) - return -ENODEV; - - ret = qat_sym_dev_create(qat_pci_dev); - if (ret != 0) - goto error_out; - - ret = qat_comp_dev_create(qat_pci_dev); - if (ret != 0) - goto error_out; - - ret = qat_asym_dev_create(qat_pci_dev); - if (ret != 0) - goto error_out; - - return 0; - -error_out: - qat_pci_dev_destroy(qat_pci_dev, pci_dev); - return ret; - -} - -static int qat_pci_remove(struct rte_pci_device *pci_dev) -{ - struct qat_pci_device *qat_pci_dev; - - if (pci_dev == NULL) - return -EINVAL; - - qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev); - if (qat_pci_dev == NULL) - return 0; - - return qat_pci_dev_destroy(qat_pci_dev, pci_dev); - -} - - -/* An rte_driver is needed in the registration of both the device and the driver - * with cryptodev. - * The actual qat pci's rte_driver can't be used as its name represents - * the whole pci device with all services. Think of this as a holder for a name - * for the crypto part of the pci device. - */ -static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD); -static struct rte_driver cryptodev_qat_sym_driver = { - .name = qat_sym_drv_name, - .alias = qat_sym_drv_name -}; -static struct cryptodev_driver qat_crypto_drv; -RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver, - cryptodev_qat_driver_id); - -static struct rte_pci_driver rte_qat_pmd = { - .id_table = pci_id_qat_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING, - .probe = qat_pci_probe, - .remove = qat_pci_remove -}; -RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd); -RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map);