[v2,01/20] crypto/cnxk: add driver skeleton
Checks
Commit Message
From: Ankur Dwivedi <adwivedi@marvell.com>
Add driver skeleton for crypto_cn9k & crypto_cn10k PMDs leveraging cnxk
common framework.
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Archana Muniganti <marchana@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
MAINTAINERS | 9 +++
doc/guides/cryptodevs/cnxk.rst | 126 +++++++++++++++++++++++++++++++
doc/guides/cryptodevs/features/cn10k.ini | 21 ++++++
doc/guides/cryptodevs/features/cn9k.ini | 21 ++++++
drivers/crypto/cnxk/cn10k_cryptodev.c | 42 +++++++++++
drivers/crypto/cnxk/cn10k_cryptodev.h | 13 ++++
drivers/crypto/cnxk/cn9k_cryptodev.c | 40 ++++++++++
drivers/crypto/cnxk/cn9k_cryptodev.h | 13 ++++
drivers/crypto/cnxk/meson.build | 16 ++++
drivers/crypto/cnxk/version.map | 3 +
drivers/crypto/meson.build | 1 +
11 files changed, 305 insertions(+)
create mode 100644 doc/guides/cryptodevs/cnxk.rst
create mode 100644 doc/guides/cryptodevs/features/cn10k.ini
create mode 100644 doc/guides/cryptodevs/features/cn9k.ini
create mode 100644 drivers/crypto/cnxk/cn10k_cryptodev.c
create mode 100644 drivers/crypto/cnxk/cn10k_cryptodev.h
create mode 100644 drivers/crypto/cnxk/cn9k_cryptodev.c
create mode 100644 drivers/crypto/cnxk/cn9k_cryptodev.h
create mode 100644 drivers/crypto/cnxk/meson.build
create mode 100644 drivers/crypto/cnxk/version.map
Comments
Hi,
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Anoob Joseph
> Sent: Friday, June 25, 2021 8:56 AM
> To: Akhil Goyal <gakhil@marvell.com>; NBU-Contact-Thomas Monjalon
> <thomas@monjalon.net>
> Cc: Ankur Dwivedi <adwivedi@marvell.com>; Jerin Jacob
> <jerinj@marvell.com>; Tejasree Kondoj <ktejasree@marvell.com>;
> dev@dpdk.org; Anoob Joseph <anoobj@marvell.com>; Archana Muniganti
> <marchana@marvell.com>
> Subject: [dpdk-dev] [PATCH v2 01/20] crypto/cnxk: add driver skeleton
>
This patch is causing the following exception with debug mode in main (02edbfab1e):
"""
meson --werror --buildtype=debug build && ninja -C build
...
ValueError: Buffer size too small (4 instead of at least 12 bytes)
...
"""
OS: Ubuntu 18.04.5
Meson: 0.58.1
Ninja: 1.8.2
Gcc: 7.5.0
Regards,
Ali
On Thu, Jul 8, 2021 at 7:08 PM Ali Alnubani <alialnu@oss.nvidia.com> wrote:
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Anoob Joseph
> > Sent: Friday, June 25, 2021 8:56 AM
> > To: Akhil Goyal <gakhil@marvell.com>; NBU-Contact-Thomas Monjalon
> > <thomas@monjalon.net>
> > Cc: Ankur Dwivedi <adwivedi@marvell.com>; Jerin Jacob
> > <jerinj@marvell.com>; Tejasree Kondoj <ktejasree@marvell.com>;
> > dev@dpdk.org; Anoob Joseph <anoobj@marvell.com>; Archana Muniganti
> > <marchana@marvell.com>
> > Subject: [dpdk-dev] [PATCH v2 01/20] crypto/cnxk: add driver skeleton
> >
>
> This patch is causing the following exception with debug mode in main (02edbfab1e):
> """
> meson --werror --buildtype=debug build && ninja -C build
> ...
> ValueError: Buffer size too small (4 instead of at least 12 bytes)
> ...
> """
- Reproduced the issue in Ubuntu 18.04 GHA vms:
FAILED: drivers/rte_crypto_cnxk.pmd.c
/usr/bin/python3 ../buildtools/gen-pmdinfo-cfile.py
/home/runner/work/dpdk/dpdk/build/buildtools ar
/home/runner/work/dpdk/dpdk/build/drivers/libtmp_rte_crypto_cnxk.a
drivers/rte_crypto_cnxk.pmd.c /usr/bin/python3
../buildtools/pmdinfogen.py elf
Traceback (most recent call last):
File "../buildtools/pmdinfogen.py", line 274, in <module>
main()
File "../buildtools/pmdinfogen.py", line 269, in main
drivers = load_drivers(image)
File "../buildtools/pmdinfogen.py", line 203, in load_drivers
drivers.append(Driver.load(image, symbol))
File "../buildtools/pmdinfogen.py", line 157, in load
driver.pci_ids = cls._load_pci_ids(image, pci_table_name_symbol)
File "../buildtools/pmdinfogen.py", line 177, in _load_pci_ids
pci_id = rte_pci_id.from_buffer_copy(data)
ValueError: Buffer size too small (4 instead of at least 12 bytes)
Traceback (most recent call last):
File "../buildtools/gen-pmdinfo-cfile.py", line 20, in <module>
subprocess.run(pmdinfogen + paths + [output], check=True)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/python3',
'../buildtools/pmdinfogen.py', 'elf',
'/home/runner/work/dpdk/dpdk/build/buildtools/tmp1ajyjflf/crypto_cnxk_cn9k_cryptodev.c.o',
'/home/runner/work/dpdk/dpdk/build/buildtools/tmp1ajyjflf/crypto_cnxk_cn9k_cryptodev_ops.c.o',
'/home/runner/work/dpdk/dpdk/build/buildtools/tmp1ajyjflf/crypto_cnxk_cn10k_cryptodev.c.o',
'/home/runner/work/dpdk/dpdk/build/buildtools/tmp1ajyjflf/crypto_cnxk_cn10k_cryptodev_ops.c.o',
'/home/runner/work/dpdk/dpdk/build/buildtools/tmp1ajyjflf/crypto_cnxk_cn10k_ipsec.c.o',
'/home/runner/work/dpdk/dpdk/build/buildtools/tmp1ajyjflf/crypto_cnxk_cnxk_cryptodev.c.o',
'/home/runner/work/dpdk/dpdk/build/buildtools/tmp1ajyjflf/crypto_cnxk_cnxk_cryptodev_capabilities.c.o',
'/home/runner/work/dpdk/dpdk/build/buildtools/tmp1ajyjflf/crypto_cnxk_cnxk_cryptodev_ops.c.o',
'/home/runner/work/dpdk/dpdk/build/buildtools/tmp1ajyjflf/crypto_cnxk_cnxk_cryptodev_sec.c.o',
'drivers/rte_crypto_cnxk.pmd.c']' returned non-zero exit status 1.
I don't get the relation with the debug mode, but there must be
something different with the pci table symbols for this driver.
My bet is on drivers/crypto/cnxk/cn9k_cryptodev.c:
static struct rte_pci_id pci_id_cpt_table[] = {
{
},
/* sentinel */
{
.device_id = 0
},
};
Adding some pci id, like the cn9k vf cpt pci id, will likely fix the issue.
And as a bonus, this will make this driver actually do something.
- There is also another issue with clang + debug.
This seems rather old, nobody complained.
Fixed with:
Author: David Marchand <david.marchand@redhat.com>
Date: Thu Jul 8 22:10:11 2021 +0200
net/octeontx/base: fix debug build with clang
Remove conflicting declaration of this symbol.
Fixes: d0d654986018 ("net/octeontx: support event Rx adapter")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
diff --git a/drivers/net/octeontx/base/octeontx_pkivf.h
b/drivers/net/octeontx/base/octeontx_pkivf.h
index d41eaa57ed..8c86841ea6 100644
--- a/drivers/net/octeontx/base/octeontx_pkivf.h
+++ b/drivers/net/octeontx/base/octeontx_pkivf.h
@@ -362,7 +362,6 @@ int octeontx_pki_port_open(int port);
int octeontx_pki_port_hash_config(int port, pki_hash_cfg_t *hash_cfg);
int octeontx_pki_port_pktbuf_config(int port, pki_pktbuf_cfg_t *buf_cfg);
int octeontx_pki_port_create_qos(int port, pki_qos_cfg_t *qos_cfg);
-int octeontx_pki_port_close(int port);
int octeontx_pki_port_errchk_config(int port, pki_errchk_cfg_t *cfg);
int octeontx_pki_port_vlan_fltr_config(int port,
pki_port_vlan_filter_config_t *fltr_cfg);
Hello,
On Thu, Jul 8, 2021 at 10:15 PM David Marchand
<david.marchand@redhat.com> wrote:
> - Reproduced the issue in Ubuntu 18.04 GHA vms:
>
> FAILED: drivers/rte_crypto_cnxk.pmd.c
> /usr/bin/python3 ../buildtools/gen-pmdinfo-cfile.py
> /home/runner/work/dpdk/dpdk/build/buildtools ar
> /home/runner/work/dpdk/dpdk/build/drivers/libtmp_rte_crypto_cnxk.a
> drivers/rte_crypto_cnxk.pmd.c /usr/bin/python3
> ../buildtools/pmdinfogen.py elf
> Traceback (most recent call last):
> File "../buildtools/pmdinfogen.py", line 274, in <module>
> main()
> File "../buildtools/pmdinfogen.py", line 269, in main
> drivers = load_drivers(image)
> File "../buildtools/pmdinfogen.py", line 203, in load_drivers
> drivers.append(Driver.load(image, symbol))
> File "../buildtools/pmdinfogen.py", line 157, in load
> driver.pci_ids = cls._load_pci_ids(image, pci_table_name_symbol)
> File "../buildtools/pmdinfogen.py", line 177, in _load_pci_ids
> pci_id = rte_pci_id.from_buffer_copy(data)
> ValueError: Buffer size too small (4 instead of at least 12 bytes)
I spent some time trying to understand this error, there is probably
multiple bug(s) somewhere between pyelftools/ctype/pmdinfogen.
In any case, on my f32, before
https://git.dpdk.org/dpdk/commit/?id=b146c30d3c0e (that hides the
issue), with a more "recent" pyelftools + python than Ubuntu 18.04, I
can see:
$ cat $HOME/builds/build-clang-shared/drivers/rte_crypto_cnxk.pmd.c
static __attribute__((unused)) const char *generator =
"../../dpdk/buildtools/pmdinfogen.py";
const char crypto_cn9k_pmd_info[] __attribute__((used)) =
"PMD_INFO_STRING= {\"name\": \"crypto_cn9k\", \"kmod\": \"vfio-pci\",
\"pci_ids\": [[10272, 25926, 28516, 24946], [11824, 11569, 11827,
25446]]}";
const char crypto_cn10k_pmd_info[] __attribute__((used)) =
"PMD_INFO_STRING= {\"name\": \"crypto_cn10k\", \"kmod\": \"vfio-pci\",
\"pci_ids\": [[6013, 41203, 65535, 65535]]}";
$ nm -S /home/dmarchan/builds/build-clang-shared/drivers/librte_crypto_cnxk.so
|grep cpt_table
000000000001d400 0000000000000018 b pci_id_cpt_table
000000000001d2f0 0000000000000018 d pci_id_cpt_table
You'll notice the former is located in bss.
$ hexdump -s $(($(printf "%d" 0x000000000001d400) - $(printf "%d"
0x1000))) -n $(printf "%d" 0x18)
/home/dmarchan/builds/build-clang-shared/drivers/librte_crypto_cnxk.so
001c400 6e61 2067 6576 7372 6f69 206e 3031 302e
001c410 312e 2820 6546 6f64
001c418
$ hexdump -s $(($(printf "%d" 0x000000000001d2f0) - $(printf "%d"
0x1000))) -n $(printf "%d" 0x18)
/home/dmarchan/builds/build-clang-shared/drivers/librte_crypto_cnxk.so
001c2f0 ffff 00ff 177d a0f3 ffff ffff 0000 0000
001c300 0000 0000 0000 0000
001c308
Those values match what the pmd.c file contains.
Are we missing a "memset()" for such symbols in pmdinfogen?
Or maybe a warning that we are adding an empty pci ids table..
There is no urgency, but if you find some time to look at this. Thanks.
@@ -1080,6 +1080,15 @@ F: drivers/crypto/octeontx2/
F: doc/guides/cryptodevs/octeontx2.rst
F: doc/guides/cryptodevs/features/octeontx2.ini
+Marvell cnxk crypto
+M: Ankur Dwivedi <adwivedi@marvell.com>
+M: Anoob Joseph <anoobj@marvell.com>
+M: Tejasree Kondoj <ktejasree@marvell.com>
+F: drivers/crypto/cnxk/
+F: doc/guides/cryptodevs/cnxk.rst
+F: doc/guides/cryptodevs/features/cn9k.ini
+F: doc/guides/cryptodevs/features/cn10k.ini
+
Null Crypto
M: Declan Doherty <declan.doherty@intel.com>
F: drivers/crypto/null/
new file mode 100644
@@ -0,0 +1,126 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2021 Marvell.
+
+Marvell cnxk Crypto Poll Mode Driver
+====================================
+
+The cnxk crypto poll mode driver provides support for offloading
+cryptographic operations to cryptographic accelerator units on the
+**Marvell OCTEON cnxk** SoC family.
+
+The cnxk crypto PMD code is organized into different sets of files.
+The file names starting with cn9k and cn10k provides support for CN9XX
+and CN10XX respectively. The common code between the SoCs is present
+in file names starting with cnxk.
+
+More information about OCTEON cnxk SoCs may be obtained from `<https://www.marvell.com>`_
+
+Supported OCTEON cnxk SoCs
+--------------------------
+
+- CN9XX
+- CN10XX
+
+Installation
+------------
+
+The OCTEON cnxk crypto PMD may be compiled natively on an OCTEON cnxk platform
+or cross-compiled on an x86 platform.
+
+Refer to :doc:`../platform/cnxk` for instructions to build your DPDK
+application.
+
+.. note::
+
+ The OCTEON cnxk crypto PMD uses services from the kernel mode OCTEON cnxk
+ crypto PF driver in linux. This driver is included in the OCTEON TX SDK.
+
+Initialization
+--------------
+
+``CN9K Initialization``
+
+List the CPT PF devices available on cn9k platform:
+
+.. code-block:: console
+
+ lspci -d:a0fd
+
+``a0fd`` is the CPT PF device id. You should see output similar to:
+
+.. code-block:: console
+
+ 0002:10:00.0 Class 1080: Device 177d:a0fd
+
+Set ``sriov_numvfs`` on the CPT PF device, to create a VF:
+
+.. code-block:: console
+
+ echo 1 > /sys/bus/pci/devices/0002:10:00.0/sriov_numvfs
+
+Bind the CPT VF device to the vfio_pci driver:
+
+.. code-block:: console
+
+ cd <dpdk directory>
+ ./usertools/dpdk-devbind.py -u 0002:10:00.1
+ ./usertools/dpdk-devbind.py -b vfio-pci 0002:10.00.1
+
+.. note::
+
+ * For CN98xx SoC, it is recommended to use even and odd DBDF VFs to achieve
+ higher performance as even VF uses one crypto engine and odd one uses
+ another crypto engine.
+
+ * Ensure that sufficient huge pages are available for your application::
+
+ dpdk-hugepages.py --setup 4G --pagesize 512M
+
+ Refer to :ref:`linux_gsg_hugepages` for more details.
+
+``CN10K Initialization``
+
+List the CPT PF devices available on cn10k platform:
+
+.. code-block:: console
+
+ lspci -d:a0f2
+
+``a0f2`` is the CPT PF device id. You should see output similar to:
+
+.. code-block:: console
+
+ 0002:20:00.0 Class 1080: Device 177d:a0f2
+
+Set ``sriov_numvfs`` on the CPT PF device, to create a VF:
+
+.. code-block:: console
+
+ echo 1 > /sys/bus/pci/devices/0002:20:00.0/sriov_numvfs
+
+Bind the CPT VF device to the vfio_pci driver:
+
+.. code-block:: console
+
+ cd <dpdk directory>
+ ./usertools/dpdk-devbind.py -u 0002:20:00.1
+ ./usertools/dpdk-devbind.py -b vfio-pci 0002:20:00.1
+
+Debugging Options
+-----------------
+
+.. _table_octeon_cnxk_crypto_debug_options:
+
+.. table:: OCTEON cnxk crypto PMD debug options
+
+ +---+------------+-------------------------------------------------------+
+ | # | Component | EAL log command |
+ +===+============+=======================================================+
+ | 1 | CPT | --log-level='pmd\.crypto\.cnxk,8' |
+ +---+------------+-------------------------------------------------------+
+
+Limitations
+-----------
+
+Multiple lcores may not operate on the same crypto queue pair. The lcore that
+enqueues to a queue pair is the one that must dequeue from it.
new file mode 100644
@@ -0,0 +1,21 @@
+;
+; Supported features of the 'cn10k' crypto driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+
+;
+; Supported crypto algorithms of 'cn10k' crypto driver.
+;
+[Cipher]
+
+;
+; Supported authentication algorithms of 'cn10k' crypto driver.
+;
+[Auth]
+
+;
+; Supported AEAD algorithms of 'cn10k' crypto driver.
+;
+[AEAD]
new file mode 100644
@@ -0,0 +1,21 @@
+;
+; Supported features of the 'cn9k' crypto driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+
+;
+; Supported crypto algorithms of 'cn9k' crypto driver.
+;
+[Cipher]
+
+;
+; Supported authentication algorithms of 'cn9k' crypto driver.
+;
+[Auth]
+
+;
+; Supported AEAD algorithms of 'cn9k' crypto driver.
+;
+[AEAD]
new file mode 100644
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2021 Marvell.
+ */
+
+#include <rte_bus_pci.h>
+#include <rte_common.h>
+#include <rte_crypto.h>
+#include <rte_cryptodev.h>
+#include <rte_cryptodev_pmd.h>
+#include <rte_dev.h>
+#include <rte_pci.h>
+
+#include "cn10k_cryptodev.h"
+#include "roc_api.h"
+
+uint8_t cn10k_cryptodev_driver_id;
+
+static struct rte_pci_id pci_id_cpt_table[] = {
+ {
+ RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVID_CN10K_RVU_CPT_VF)
+ },
+ /* sentinel */
+ {
+ .device_id = 0
+ },
+};
+
+static struct rte_pci_driver cn10k_cryptodev_pmd = {
+ .id_table = pci_id_cpt_table,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA,
+ .probe = NULL,
+ .remove = NULL,
+};
+
+static struct cryptodev_driver cn10k_cryptodev_drv;
+
+RTE_PMD_REGISTER_PCI(CRYPTODEV_NAME_CN10K_PMD, cn10k_cryptodev_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(CRYPTODEV_NAME_CN10K_PMD, pci_id_cpt_table);
+RTE_PMD_REGISTER_KMOD_DEP(CRYPTODEV_NAME_CN10K_PMD, "vfio-pci");
+RTE_PMD_REGISTER_CRYPTO_DRIVER(cn10k_cryptodev_drv, cn10k_cryptodev_pmd.driver,
+ cn10k_cryptodev_driver_id);
new file mode 100644
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2021 Marvell.
+ */
+
+#ifndef _CN10K_CRYPTODEV_H_
+#define _CN10K_CRYPTODEV_H_
+
+/* Marvell OCTEON CN10K Crypto PMD device name */
+#define CRYPTODEV_NAME_CN10K_PMD crypto_cn10k
+
+extern uint8_t cn10k_cryptodev_driver_id;
+
+#endif /* _CN10K_CRYPTODEV_H_ */
new file mode 100644
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2021 Marvell.
+ */
+
+#include <rte_bus_pci.h>
+#include <rte_common.h>
+#include <rte_crypto.h>
+#include <rte_cryptodev.h>
+#include <rte_cryptodev_pmd.h>
+#include <rte_dev.h>
+#include <rte_pci.h>
+
+#include "cn9k_cryptodev.h"
+#include "roc_api.h"
+
+uint8_t cn9k_cryptodev_driver_id;
+
+static struct rte_pci_id pci_id_cpt_table[] = {
+ {
+ },
+ /* sentinel */
+ {
+ .device_id = 0
+ },
+};
+
+static struct rte_pci_driver cn9k_cryptodev_pmd = {
+ .id_table = pci_id_cpt_table,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA,
+ .probe = NULL,
+ .remove = NULL,
+};
+
+static struct cryptodev_driver cn9k_cryptodev_drv;
+
+RTE_PMD_REGISTER_PCI(CRYPTODEV_NAME_CN9K_PMD, cn9k_cryptodev_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(CRYPTODEV_NAME_CN9K_PMD, pci_id_cpt_table);
+RTE_PMD_REGISTER_KMOD_DEP(CRYPTODEV_NAME_CN9K_PMD, "vfio-pci");
+RTE_PMD_REGISTER_CRYPTO_DRIVER(cn9k_cryptodev_drv, cn9k_cryptodev_pmd.driver,
+ cn9k_cryptodev_driver_id);
new file mode 100644
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2021 Marvell.
+ */
+
+#ifndef _CN9K_CRYPTODEV_H_
+#define _CN9K_CRYPTODEV_H_
+
+/* Marvell OCTEON CN9K Crypto PMD device name */
+#define CRYPTODEV_NAME_CN9K_PMD crypto_cn9k
+
+extern uint8_t cn9k_cryptodev_driver_id;
+
+#endif /* _CN9K_CRYPTODEV_H_ */
new file mode 100644
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(C) 2021 Marvell.
+#
+
+if not is_linux or not dpdk_conf.get('RTE_ARCH_64')
+ build = false
+ reason = 'only supported on 64-bit Linux'
+ subdir_done()
+endif
+
+sources = files(
+ 'cn9k_cryptodev.c',
+ 'cn10k_cryptodev.c',
+)
+
+deps += ['bus_pci', 'common_cnxk']
new file mode 100644
@@ -0,0 +1,3 @@
+INTERNAL {
+ local: *;
+};
@@ -12,6 +12,7 @@ drivers = [
'bcmfs',
'caam_jr',
'ccp',
+ 'cnxk',
'dpaa_sec',
'dpaa2_sec',
'kasumi',