Patch Detail

HTTP 200 OK
Allow: GET, PUT, PATCH, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "id": 8375,
    "url": "http://patches.dpdk.org/api/patches/8375/?format=api",
    "web_url": "http://patches.dpdk.org/project/dpdk/patch/1446209986-29137-5-git-send-email-declan.doherty@intel.com/",
    "project": {
        "id": 1,
        "url": "http://patches.dpdk.org/api/projects/1/?format=api",
        "name": "DPDK",
        "link_name": "dpdk",
        "list_id": "dev.dpdk.org",
        "list_email": "dev@dpdk.org",
        "web_url": "http://core.dpdk.org",
        "scm_url": "git://dpdk.org/dpdk",
        "webscm_url": "http://git.dpdk.org/dpdk",
        "list_archive_url": "https://inbox.dpdk.org/dev",
        "list_archive_url_format": "https://inbox.dpdk.org/dev/{}",
        "commit_url_format": ""
    },
    "msgid": "<1446209986-29137-5-git-send-email-declan.doherty@intel.com>",
    "list_archive_url": "https://inbox.dpdk.org/dev/1446209986-29137-5-git-send-email-declan.doherty@intel.com",
    "date": "2015-10-30T12:59:44",
    "name": "[dpdk-dev,v2,4/6] aesni_mb_pmd: Initial implementation of multi buffer based crypto device",
    "commit_ref": null,
    "pull_url": null,
    "state": "superseded",
    "archived": true,
    "hash": "f5ba9aa6bb0bedd215638fcb882e998e96fb662e",
    "submitter": {
        "id": 11,
        "url": "http://patches.dpdk.org/api/people/11/?format=api",
        "name": "Doherty, Declan",
        "email": "declan.doherty@intel.com"
    },
    "delegate": null,
    "mbox": "http://patches.dpdk.org/project/dpdk/patch/1446209986-29137-5-git-send-email-declan.doherty@intel.com/mbox/",
    "series": [],
    "comments": "http://patches.dpdk.org/api/patches/8375/comments/",
    "check": "pending",
    "checks": "http://patches.dpdk.org/api/patches/8375/checks/",
    "tags": {},
    "related": [],
    "headers": {
        "Return-Path": "<dev-bounces@dpdk.org>",
        "X-Original-To": "patchwork@dpdk.org",
        "Delivered-To": "patchwork@dpdk.org",
        "Received": [
            "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 093118E86;\n\tFri, 30 Oct 2015 13:55:28 +0100 (CET)",
            "from mga14.intel.com (mga14.intel.com [192.55.52.115])\n\tby dpdk.org (Postfix) with ESMTP id 2DF168E81\n\tfor <dev@dpdk.org>; Fri, 30 Oct 2015 13:55:19 +0100 (CET)",
            "from fmsmga003.fm.intel.com ([10.253.24.29])\n\tby fmsmga103.fm.intel.com with ESMTP; 30 Oct 2015 05:55:18 -0700",
            "from dwdohert-dpdk-fedora-20.ir.intel.com ([163.33.213.96])\n\tby FMSMGA003.fm.intel.com with ESMTP; 30 Oct 2015 05:55:17 -0700"
        ],
        "X-ExtLoop1": "1",
        "X-IronPort-AV": "E=Sophos;i=\"5.20,218,1444719600\"; d=\"scan'208\";a=\"590996786\"",
        "From": "Declan Doherty <declan.doherty@intel.com>",
        "To": "dev@dpdk.org",
        "Date": "Fri, 30 Oct 2015 12:59:44 +0000",
        "Message-Id": "<1446209986-29137-5-git-send-email-declan.doherty@intel.com>",
        "X-Mailer": "git-send-email 2.4.3",
        "In-Reply-To": "<1446209986-29137-1-git-send-email-declan.doherty@intel.com>",
        "References": "<1443826867-21004-1-git-send-email-declan.doherty@intel.com>\n\t<1446209986-29137-1-git-send-email-declan.doherty@intel.com>",
        "MIME-Version": "1.0",
        "Content-Type": "text/plain; charset=UTF-8",
        "Content-Transfer-Encoding": "8bit",
        "Subject": "[dpdk-dev] [PATCH v2 4/6] aesni_mb_pmd: Initial implementation of\n\tmulti buffer based crypto device",
        "X-BeenThere": "dev@dpdk.org",
        "X-Mailman-Version": "2.1.15",
        "Precedence": "list",
        "List-Id": "patches and discussions about DPDK <dev.dpdk.org>",
        "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>",
        "List-Archive": "<http://dpdk.org/ml/archives/dev/>",
        "List-Post": "<mailto:dev@dpdk.org>",
        "List-Help": "<mailto:dev-request@dpdk.org?subject=help>",
        "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>",
        "Errors-To": "dev-bounces@dpdk.org",
        "Sender": "\"dev\" <dev-bounces@dpdk.org>"
    },
    "content": "This patch provides the initial implementation of the AES-NI multi-buffer\nbased crypto poll mode driver using DPDK's new cryptodev framework.\n\nThis PMD is dependent on Intel's multibuffer library, see the whitepaper\n\"Fast Multi-buffer IPsec Implementations on Intel® Architecture\nProcessors\", see ref 1 for details on the library's design and ref 2 to\ndownload the library itself. This initial implementation is limited to\nsupporting the chained operations of \"hash then cipher\" or \"cipher then\nhash\" for the following cipher and hash algorithms:\n\nCipher algorithms:\n  - RTE_CRYPTO_CIPHER_AES128_CBC\n  - RTE_CRYPTO_CIPHER_AES256_CBC\n  - RTE_CRYPTO_CIPHER_AES512_CBC\n \nHash algorithms:\n  - RTE_CRYPTO_AUTH_SHA1_HMAC\n  - RTE_CRYPTO_AUTH_SHA256_HMAC\n  - RTE_CRYPTO_AUTH_SHA512_HMAC\n  - RTE_CRYPTO_AUTH_AES_XCBC_MAC\n\nImportant Note:\nDue to the fact that the multi-buffer library is designed for\naccelerating IPsec crypto oepration, the digest's generated for the HMAC\nfunctions are truncated to lengths specified by IPsec RFC's, ie RFC2404\nfor using HMAC-SHA-1 with IPsec specifies that the digest is truncate\nfrom 20 to 12 bytes.\n\nBuild instructions:\nTo build DPKD with the AESNI_MB_PMD the user is required to download\n(ref 2) and compile the multi-buffer library on there user system before\nbuilding DPDK. The environmental variable AESNI_MULTI_BUFFER_LIB_PATH\nmust be exported with the path where you extracted and built the multi\nbuffer library and finally set CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y in\nconfig/common_linuxapp.\n\nCurrent status: It's doesn't support crypto operation\nacross chained mbufs, or cipher only or hash only operations.\n\nref 1:\nhttps://www-ssl.intel.com/content/www/us/en/intelligent-systems/intel-technology/fast-multi-buffer-ipsec-implementations-ia-processors-p\n\nref 2: https://downloadcenter.intel.com/download/22972\n\nSigned-off-by: Declan Doherty <declan.doherty@intel.com>\n---\n config/common_bsdapp                               |   7 +\n config/common_linuxapp                             |   6 +\n doc/guides/cryptodevs/aesni_mb.rst                 |  76 ++\n doc/guides/cryptodevs/index.rst                    |   1 +\n drivers/crypto/Makefile                            |   1 +\n drivers/crypto/aesni_mb/Makefile                   |  67 ++\n drivers/crypto/aesni_mb/aesni_mb_ops.h             | 212 ++++++\n drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         | 790 +++++++++++++++++++++\n drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c     | 296 ++++++++\n drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h | 230 ++++++\n drivers/crypto/aesni_mb/rte_pmd_aesni_version.map  |   3 +\n mk/rte.app.mk                                      |   4 +\n 12 files changed, 1693 insertions(+)\n create mode 100644 doc/guides/cryptodevs/aesni_mb.rst\n create mode 100644 drivers/crypto/aesni_mb/Makefile\n create mode 100644 drivers/crypto/aesni_mb/aesni_mb_ops.h\n create mode 100644 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c\n create mode 100644 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c\n create mode 100644 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h\n create mode 100644 drivers/crypto/aesni_mb/rte_pmd_aesni_version.map",
    "diff": "diff --git a/config/common_bsdapp b/config/common_bsdapp\nindex 02f10a3..5a177c3 100644\n--- a/config/common_bsdapp\n+++ b/config/common_bsdapp\n@@ -168,6 +168,13 @@ CONFIG_RTE_LIBRTE_QAT_DEBUG_DRIVER=n\n #\n CONFIG_RTE_MAX_QAT_SESSIONS=200\n \n+\n+#\n+# Compile PMD for AESNI backed device\n+#\n+CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n\n+CONFIG_RTE_LIBRTE_AESNI_MB_DEBUG=n\n+\n #\n # Support NIC bypass logic\n #\ndiff --git a/config/common_linuxapp b/config/common_linuxapp\nindex cae80a5..c1054e8 100644\n--- a/config/common_linuxapp\n+++ b/config/common_linuxapp\n@@ -166,6 +166,12 @@ CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=n\n #\n CONFIG_RTE_LIBRTE_PMD_QAT_MAX_SESSIONS=2048\n \n+# Compile PMD for AESNI backed device\n+#\n+CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y\n+CONFIG_RTE_LIBRTE_PMD_AESNI_MB_DEBUG=n\n+CONFIG_RTE_LIBRTE_PMD_AESNI_MB_MAX_SESSIONS=2048\n+\n #\n # Support NIC bypass logic\n #\ndiff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst\nnew file mode 100644\nindex 0000000..826b632\n--- /dev/null\n+++ b/doc/guides/cryptodevs/aesni_mb.rst\n@@ -0,0 +1,76 @@\n+..  BSD LICENSE\n+    Copyright(c) 2015 Intel Corporation. All rights reserved.\n+\n+    Redistribution and use in source and binary forms, with or without\n+    modification, are permitted provided that the following conditions\n+    are met:\n+\n+    * Redistributions of source code must retain the above copyright\n+    notice, this list of conditions and the following disclaimer.\n+    * Redistributions in binary form must reproduce the above copyright\n+    notice, this list of conditions and the following disclaimer in\n+    the documentation and/or other materials provided with the\n+    distribution.\n+    * Neither the name of Intel Corporation nor the names of its\n+    contributors may be used to endorse or promote products derived\n+    from this software without specific prior written permission.\n+\n+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+    \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+\n+AESN-NI Multi Buffer Crytpo Poll Mode Driver\n+============================================\n+\n+\n+The AESNI MB PMD (**librte_pmd_aesni_mb**) provides poll mode crypto driver\n+support for utilising Intel multi buffer library, see the white paper\n+`Fast Multi-buffer IPsec Implementations on Intel® Architecture Processors\n+<https://www-ssl.intel.com/content/www/us/en/intelligent-systems/intel-technology/fast-multi-buffer-ipsec-implementations-ia-processors-paper.html?wapkw=multi+buffer>`_.\n+\n+The AES-NI MB PMD has current only been tested on Fedora 21 64-bit with gcc.\n+\n+Features\n+--------\n+\n+AESNI MB PMD has support for:\n+\n+Cipher algorithms:\n+\n+* RTE_CRYPTO_SYM_CIPHER_AES128_CBC\n+* RTE_CRYPTO_SYM_CIPHER_AES256_CBC\n+* RTE_CRYPTO_SYM_CIPHER_AES512_CBC\n+\n+Hash algorithms:\n+\n+* RTE_CRYPTO_SYM_HASH_SHA1_HMAC\n+* RTE_CRYPTO_SYM_HASH_SHA256_HMAC\n+* RTE_CRYPTO_SYM_HASH_SHA512_HMAC\n+\n+Limitations\n+-----------\n+\n+* Chained mbufs are not supported.\n+* Hash only is not supported.\n+* Cipher only is not supported.\n+* Only in-place is currently supported (destination address is the same as source address).\n+* Only supports session-oriented API implementation (session-less APIs are not supported).\n+*  Not performance tuned.\n+\n+Installation\n+------------\n+\n+To build DPKD with the AESNI_MB_PMD the user is required to download the library\n+from `here <https://downloadcenter.intel.com/download/22972>`_ and compile it on\n+their user system before building DPDK. The environmental variable\n+AESNI_MULTI_BUFFER_LIB_PATH must be exported with the path where you extracted\n+and built the multi buffer library and finally set\n+CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y in config/common_linuxapp.\ndiff --git a/doc/guides/cryptodevs/index.rst b/doc/guides/cryptodevs/index.rst\nindex 1c31697..8949fd0 100644\n--- a/doc/guides/cryptodevs/index.rst\n+++ b/doc/guides/cryptodevs/index.rst\n@@ -39,4 +39,5 @@ Crypto Device Drivers\n     :maxdepth: 2\n     :numbered:\n \n+    aesni_mb\n     qat\ndiff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile\nindex 9529f30..26325b0 100644\n--- a/drivers/crypto/Makefile\n+++ b/drivers/crypto/Makefile\n@@ -31,6 +31,7 @@\n \n include $(RTE_SDK)/mk/rte.vars.mk\n \n+DIRS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += aesni_mb\n DIRS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat\n \n include $(RTE_SDK)/mk/rte.sharelib.mk\ndiff --git a/drivers/crypto/aesni_mb/Makefile b/drivers/crypto/aesni_mb/Makefile\nnew file mode 100644\nindex 0000000..62f51ce\n--- /dev/null\n+++ b/drivers/crypto/aesni_mb/Makefile\n@@ -0,0 +1,67 @@\n+#   BSD LICENSE\n+#\n+#   Copyright(c) 2015 Intel Corporation. All rights reserved.\n+#\n+#   Redistribution and use in source and binary forms, with or without\n+#   modification, are permitted provided that the following conditions\n+#   are met:\n+#\n+#     * Redistributions of source code must retain the above copyright\n+#       notice, this list of conditions and the following disclaimer.\n+#     * Redistributions in binary form must reproduce the above copyright\n+#       notice, this list of conditions and the following disclaimer in\n+#       the documentation and/or other materials provided with the\n+#       distribution.\n+#     * Neither the name of Intel Corporation nor the names of its\n+#       contributors may be used to endorse or promote products derived\n+#       from this software without specific prior written permission.\n+#\n+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+#   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+\n+include $(RTE_SDK)/mk/rte.vars.mk\n+\n+ifeq ($(AESNI_MULTI_BUFFER_LIB_PATH),)\n+$(error \"Please define AESNI_MULTI_BUFFER_LIB_PATH environment variable\")\n+endif\n+\n+# library name\n+LIB = librte_pmd_aesni_mb.a\n+\n+# build flags\n+CFLAGS += -O3\n+CFLAGS += $(WERROR_FLAGS)\n+\n+# library version\n+LIBABIVER := 1\n+\n+# versioning export map\n+EXPORT_MAP := rte_pmd_aesni_version.map\n+\n+# external library include paths\n+CFLAGS += -I$(AESNI_MULTI_BUFFER_LIB_PATH)\n+CFLAGS += -I$(AESNI_MULTI_BUFFER_LIB_PATH)/include\n+\n+# library source files\n+SRCS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += rte_aesni_mb_pmd.c\n+SRCS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += rte_aesni_mb_pmd_ops.c\n+\n+# export include files\n+SYMLINK-y-include +=\n+\n+\n+# library dependencies\n+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += lib/librte_eal\n+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += lib/librte_mbuf\n+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += lib/librte_cryptodev\n+\n+include $(RTE_SDK)/mk/rte.lib.mk\ndiff --git a/drivers/crypto/aesni_mb/aesni_mb_ops.h b/drivers/crypto/aesni_mb/aesni_mb_ops.h\nnew file mode 100644\nindex 0000000..3d15a68\n--- /dev/null\n+++ b/drivers/crypto/aesni_mb/aesni_mb_ops.h\n@@ -0,0 +1,212 @@\n+/*-\n+ *   BSD LICENSE\n+ *\n+ *   Copyright(c) 2015 Intel Corporation. All rights reserved.\n+ *\n+ *   Redistribution and use in source and binary forms, with or without\n+ *   modification, are permitted provided that the following conditions\n+ *   are met:\n+ *\n+ *     * Redistributions of source code must retain the above copyright\n+ *       notice, this list of conditions and the following disclaimer.\n+ *     * Redistributions in binary form must reproduce the above copyright\n+ *       notice, this list of conditions and the following disclaimer in\n+ *       the documentation and/or other materials provided with the\n+ *       distribution.\n+ *     * Neither the name of Intel Corporation nor the names of its\n+ *       contributors may be used to endorse or promote products derived\n+ *       from this software without specific prior written permission.\n+ *\n+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ *   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ */\n+\n+#ifndef _AESNI_MB_OPS_H_\n+#define _AESNI_MB_OPS_H_\n+\n+#ifndef LINUX\n+#define LINUX\n+#endif\n+\n+#include <mb_mgr.h>\n+#include <aux_funcs.h>\n+#include <gcm_defines.h>\n+\n+enum aesni_mb_vector_mode {\n+\tRTE_AESNI_MB_NOT_SUPPORTED = 0,\n+\tRTE_AESNI_MB_SSE,\n+\tRTE_AESNI_MB_AVX,\n+\tRTE_AESNI_MB_AVX2\n+};\n+\n+typedef void (*md5_one_block_t)(void *data, void *digest);\n+typedef void (*sha1_one_block_t)(void *data, void *digest);\n+typedef void (*sha224_one_block_t)(void *data, void *digest);\n+typedef void (*sha256_one_block_t)(void *data, void *digest);\n+typedef void (*sha384_one_block_t)(void *data, void *digest);\n+typedef void (*sha512_one_block_t)(void *data, void *digest);\n+\n+typedef void (*aes_keyexp_128_t)(void *key, void *enc_exp_keys, void *dec_exp_keys);\n+typedef void (*aes_keyexp_128_enc_t)(void *key, void *enc_exp_keys);\n+typedef void (*aes_keyexp_192_t)(void *key, void *enc_exp_keys, void *dec_exp_keys);\n+typedef void (*aes_keyexp_256_t)(void *key, void *enc_exp_keys, void *dec_exp_keys);\n+\n+typedef void (*aes_xcbc_expand_key_t)(void *key, void *exp_k1, void *k2, void *k3);\n+\n+typedef void (*aesni_gcm_t)(gcm_data *my_ctx_data, u8 *out, const u8 *in,\n+\t\tu64 plaintext_len, u8 *iv, const u8 *aad, u64 aad_len,\n+\t\tu8 *auth_tag, u64 auth_tag_len);\n+\n+typedef void (*aesni_gcm_precomp_t)(gcm_data *my_ctx_data, u8 *hash_subkey);\n+\n+/** Multi-buffer library function pointer table */\n+struct aesni_mb_ops {\n+\tstruct {\n+\t\tinit_mb_mgr_t init_mgr;\t\t/**< Initialise scheduler  */\n+\t\tget_next_job_t get_next;\t/**< Get next free job structure */\n+\t\tsubmit_job_t submit;\t\t/**< Submit job to scheduler */\n+\t\tget_completed_job_t get_completed_job; /**< Get completed job */\n+\t\tflush_job_t flush_job;\t\t/**< flush jobs from manager */\n+\t} job; /**< multi buffer manager functions */\n+\tstruct {\n+\t\tstruct {\n+\t\t\tmd5_one_block_t md5;\t\t/**< MD5 one block hash */\n+\t\t\tsha1_one_block_t sha1;\t\t/**< SHA1 one block hash */\n+\t\t\tsha224_one_block_t sha224;\t/**< SHA224 one block hash */\n+\t\t\tsha256_one_block_t sha256;\t/**< SHA256 one block hash */\n+\t\t\tsha384_one_block_t sha384;\t/**< SHA384 one block hash */\n+\t\t\tsha512_one_block_t sha512;\t/**< SHA512 one block hash */\n+\t\t} one_block; /**< one block hash functions */\n+\t\tstruct {\n+\t\t\taes_keyexp_128_t aes128;\t/**< AES128 key expansions */\n+\t\t\taes_keyexp_128_enc_t aes128_enc;/**< AES128 enc key expansion */\n+\t\t\taes_keyexp_192_t aes192;\t/**< AES192 key expansions */\n+\t\t\taes_keyexp_256_t aes256;\t/**< AES256 key expansions */\n+\t\t\taes_xcbc_expand_key_t aes_xcbc;\t/**< AES XCBC key expansions */\n+\t\t} keyexp;\t/**< Key expansion functions */\n+\t} aux; /**< Auxiliary functions */\n+\tstruct {\n+\n+\t\taesni_gcm_t enc;\t\t/**< MD5 encode */\n+\t\taesni_gcm_t dec;\t\t/**< GCM decode */\n+\t\taesni_gcm_precomp_t precomp;\t/**< GCM pre-compute */\n+\t} gcm; /**< GCM functions */\n+};\n+\n+\n+static const struct aesni_mb_ops job_ops[] = {\n+\t\t[RTE_AESNI_MB_NOT_SUPPORTED] = {\n+\t\t\t.job = { NULL },\n+\t\t\t.aux = {\n+\t\t\t\t.one_block = { NULL },\n+\t\t\t\t.keyexp = { NULL }\n+\t\t\t},\n+\t\t\t.gcm = { NULL\n+\t\t\t}\n+\t\t},\n+\t\t[RTE_AESNI_MB_SSE] = {\n+\t\t\t.job = {\n+\t\t\t\tinit_mb_mgr_sse,\n+\t\t\t\tget_next_job_sse,\n+\t\t\t\tsubmit_job_sse,\n+\t\t\t\tget_completed_job_sse,\n+\t\t\t\tflush_job_sse\n+\t\t\t},\n+\t\t\t.aux = {\n+\t\t\t\t.one_block = {\n+\t\t\t\t\tmd5_one_block_sse,\n+\t\t\t\t\tsha1_one_block_sse,\n+\t\t\t\t\tsha224_one_block_sse,\n+\t\t\t\t\tsha256_one_block_sse,\n+\t\t\t\t\tsha384_one_block_sse,\n+\t\t\t\t\tsha512_one_block_sse\n+\t\t\t\t},\n+\t\t\t\t.keyexp = {\n+\t\t\t\t\taes_keyexp_128_sse,\n+\t\t\t\t\taes_keyexp_128_enc_sse,\n+\t\t\t\t\taes_keyexp_192_sse,\n+\t\t\t\t\taes_keyexp_256_sse,\n+\t\t\t\t\taes_xcbc_expand_key_sse\n+\t\t\t\t}\n+\t\t\t},\n+\t\t\t.gcm = {\n+\t\t\t\taesni_gcm_enc_sse,\n+\t\t\t\taesni_gcm_dec_sse,\n+\t\t\t\taesni_gcm_precomp_sse\n+\t\t\t}\n+\t\t},\n+\t\t[RTE_AESNI_MB_AVX] = {\n+\t\t\t\t.job = {\n+\t\t\t\t\tinit_mb_mgr_avx,\n+\t\t\t\t\tget_next_job_avx,\n+\t\t\t\t\tsubmit_job_avx,\n+\t\t\t\t\tget_completed_job_avx,\n+\t\t\t\t\tflush_job_avx\n+\t\t\t\t},\n+\t\t\t\t.aux = {\n+\t\t\t\t\t.one_block = {\n+\t\t\t\t\t\tmd5_one_block_avx,\n+\t\t\t\t\t\tsha1_one_block_avx,\n+\t\t\t\t\t\tsha224_one_block_avx,\n+\t\t\t\t\t\tsha256_one_block_avx,\n+\t\t\t\t\t\tsha384_one_block_avx,\n+\t\t\t\t\t\tsha512_one_block_avx\n+\t\t\t\t\t},\n+\t\t\t\t\t.keyexp = {\n+\t\t\t\t\t\taes_keyexp_128_avx,\n+\t\t\t\t\t\taes_keyexp_128_enc_avx,\n+\t\t\t\t\t\taes_keyexp_192_avx,\n+\t\t\t\t\t\taes_keyexp_256_avx,\n+\t\t\t\t\t\taes_xcbc_expand_key_avx\n+\t\t\t\t\t}\n+\t\t\t\t},\n+\t\t\t\t.gcm = {\n+\t\t\t\t\taesni_gcm_enc_avx_gen2,\n+\t\t\t\t\taesni_gcm_dec_avx_gen2,\n+\t\t\t\t\taesni_gcm_precomp_avx_gen2\n+\t\t\t\t}\n+\t\t},\n+\t\t[RTE_AESNI_MB_AVX2] = {\n+\t\t\t\t.job = {\n+\t\t\t\t\tinit_mb_mgr_avx2,\n+\t\t\t\t\tget_next_job_avx2,\n+\t\t\t\t\tsubmit_job_avx2,\n+\t\t\t\t\tget_completed_job_avx2,\n+\t\t\t\t\tflush_job_avx2\n+\t\t\t\t},\n+\t\t\t\t.aux = {\n+\t\t\t\t\t.one_block = {\n+\t\t\t\t\t\tmd5_one_block_avx2,\n+\t\t\t\t\t\tsha1_one_block_avx2,\n+\t\t\t\t\t\tsha224_one_block_avx2,\n+\t\t\t\t\t\tsha256_one_block_avx2,\n+\t\t\t\t\t\tsha384_one_block_avx2,\n+\t\t\t\t\t\tsha512_one_block_avx2\n+\t\t\t\t\t},\n+\t\t\t\t\t.keyexp = {\n+\t\t\t\t\t\taes_keyexp_128_avx2,\n+\t\t\t\t\t\taes_keyexp_128_enc_avx2,\n+\t\t\t\t\t\taes_keyexp_192_avx2,\n+\t\t\t\t\t\taes_keyexp_256_avx2,\n+\t\t\t\t\t\taes_xcbc_expand_key_avx2\n+\t\t\t\t\t}\n+\t\t\t\t},\n+\t\t\t\t.gcm = {\n+\t\t\t\t\taesni_gcm_enc_avx_gen4,\n+\t\t\t\t\taesni_gcm_dec_avx_gen4,\n+\t\t\t\t\taesni_gcm_precomp_avx_gen4\n+\t\t\t\t}\n+\t\t},\n+};\n+\n+\n+#endif /* _AESNI_MB_OPS_H_ */\ndiff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c\nnew file mode 100644\nindex 0000000..e469f6d\n--- /dev/null\n+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c\n@@ -0,0 +1,790 @@\n+/*-\n+ *   BSD LICENSE\n+ *\n+ *   Copyright(c) 2015 Intel Corporation. All rights reserved.\n+ *\n+ *   Redistribution and use in source and binary forms, with or without\n+ *   modification, are permitted provided that the following conditions\n+ *   are met:\n+ *\n+ *     * Redistributions of source code must retain the above copyright\n+ *       notice, this list of conditions and the following disclaimer.\n+ *     * Redistributions in binary form must reproduce the above copyright\n+ *       notice, this list of conditions and the following disclaimer in\n+ *       the documentation and/or other materials provided with the\n+ *       distribution.\n+ *     * Neither the name of Intel Corporation nor the names of its\n+ *       contributors may be used to endorse or promote products derived\n+ *       from this software without specific prior written permission.\n+ *\n+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ *   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ */\n+\n+#include <rte_common.h>\n+#include <rte_config.h>\n+#include <rte_hexdump.h>\n+#include <rte_cryptodev.h>\n+#include <rte_cryptodev_pmd.h>\n+#include <rte_dev.h>\n+#include <rte_malloc.h>\n+#include <rte_cpuflags.h>\n+#include <rte_mbuf_offload.h>\n+\n+#include \"rte_aesni_mb_pmd_private.h\"\n+\n+/**\n+ * Global static parameter used to create a unique name for each AES-NI multi\n+ * buffer crypto device.\n+ */\n+static unsigned unique_name_id;\n+\n+static inline int\n+create_unique_device_name(char *name, size_t size)\n+{\n+\tint ret;\n+\n+\tif (name == NULL)\n+\t\treturn -EINVAL;\n+\n+\tret = snprintf(name, size, \"%s_%u\", CRYPTODEV_NAME_AESNI_MB_PMD,\n+\t\t\tunique_name_id++);\n+\tif (ret < 0)\n+\t\treturn ret;\n+\treturn 0;\n+}\n+\n+typedef void (*hash_one_block_t)(void *data, void *digest);\n+typedef void (*aes_keyexp_t)(void *key, void *enc_exp_keys, void *dec_exp_keys);\n+\n+/**\n+ * Calculate the authentication pre-computes\n+ *\n+ * @param one_block_hash\tFunction pointer to calculate digest on ipad/opad\n+ * @param ipad\t\t\tInner pad output byte array\n+ * @param opad\t\t\tOuter pad output byte array\n+ * @param hkey\t\t\tAuthentication key\n+ * @param hkey_len\t\tAuthentication key length\n+ * @param blocksize\t\tBlock size of selected hash algo\n+ */\n+static void\n+calculate_auth_precomputes(hash_one_block_t one_block_hash,\n+\t\tuint8_t *ipad, uint8_t *opad,\n+\t\tuint8_t *hkey, uint16_t hkey_len,\n+\t\tuint16_t blocksize)\n+{\n+\tunsigned i, length;\n+\n+\tuint8_t ipad_buf[blocksize] __rte_aligned(16);\n+\tuint8_t opad_buf[blocksize] __rte_aligned(16);\n+\n+\t/* Setup inner and outer pads */\n+\tmemset(ipad_buf, HMAC_IPAD_VALUE, blocksize);\n+\tmemset(opad_buf, HMAC_OPAD_VALUE, blocksize);\n+\n+\t/* XOR hash key with inner and outer pads */\n+\tlength = hkey_len > blocksize ? blocksize : hkey_len;\n+\n+\tfor (i = 0; i < length; i++) {\n+\t\tipad_buf[i] ^= hkey[i];\n+\t\topad_buf[i] ^= hkey[i];\n+\t}\n+\n+\t/* Compute partial hashes */\n+\t(*one_block_hash)(ipad_buf, ipad);\n+\t(*one_block_hash)(opad_buf, opad);\n+\n+\t/* Clean up stack */\n+\tmemset(ipad_buf, 0, blocksize);\n+\tmemset(opad_buf, 0, blocksize);\n+}\n+\n+/** Get xform chain order */\n+static int\n+aesni_mb_get_chain_order(const struct rte_crypto_xform *xform)\n+{\n+\t/* multi-buffer only supports HASH_CIPHER or CIPHER_HASH chained\n+\t * operations, all other options are invalid, so we must have exactly\n+\t * 2 xform structs chained together */\n+\tif (xform->next == NULL || xform->next->next != NULL)\n+\t\treturn -1;\n+\n+\tif (xform->type == RTE_CRYPTO_XFORM_AUTH &&\n+\t\t\txform->next->type == RTE_CRYPTO_XFORM_CIPHER)\n+\t\treturn HASH_CIPHER;\n+\n+\tif (xform->type == RTE_CRYPTO_XFORM_CIPHER &&\n+\t\t\t\txform->next->type == RTE_CRYPTO_XFORM_AUTH)\n+\t\treturn CIPHER_HASH;\n+\n+\treturn -1;\n+}\n+\n+/** Set session authentication parameters */\n+static int\n+aesni_mb_set_session_auth_parameters(const struct aesni_mb_ops *mb_ops,\n+\t\tstruct aesni_mb_session *sess,\n+\t\tconst struct rte_crypto_xform *xform)\n+{\n+\thash_one_block_t hash_oneblock_fn;\n+\n+\tif (xform->type != RTE_CRYPTO_XFORM_AUTH) {\n+\t\tMB_LOG_ERR(\"Crypto xform struct not of type auth\");\n+\t\treturn -1;\n+\t}\n+\n+\t/* Set Authentication Parameters */\n+\tif (xform->auth.algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC) {\n+\t\tsess->auth.algo = AES_XCBC;\n+\t\t(*mb_ops->aux.keyexp.aes_xcbc)(xform->auth.key.data,\n+\t\t\t\tsess->auth.xcbc.k1_expanded,\n+\t\t\t\tsess->auth.xcbc.k2, sess->auth.xcbc.k3);\n+\t\treturn 0;\n+\t}\n+\n+\tswitch (xform->auth.algo) {\n+\tcase RTE_CRYPTO_AUTH_MD5_HMAC:\n+\t\tsess->auth.algo = MD5;\n+\t\thash_oneblock_fn = mb_ops->aux.one_block.md5;\n+\t\tbreak;\n+\tcase RTE_CRYPTO_AUTH_SHA1_HMAC:\n+\t\tsess->auth.algo = SHA1;\n+\t\thash_oneblock_fn = mb_ops->aux.one_block.sha1;\n+\t\tbreak;\n+\tcase RTE_CRYPTO_AUTH_SHA224_HMAC:\n+\t\tsess->auth.algo = SHA_224;\n+\t\thash_oneblock_fn = mb_ops->aux.one_block.sha224;\n+\t\tbreak;\n+\tcase RTE_CRYPTO_AUTH_SHA256_HMAC:\n+\t\tsess->auth.algo = SHA_256;\n+\t\thash_oneblock_fn = mb_ops->aux.one_block.sha256;\n+\t\tbreak;\n+\tcase RTE_CRYPTO_AUTH_SHA384_HMAC:\n+\t\tsess->auth.algo = SHA_384;\n+\t\thash_oneblock_fn = mb_ops->aux.one_block.sha384;\n+\t\tbreak;\n+\tcase RTE_CRYPTO_AUTH_SHA512_HMAC:\n+\t\tsess->auth.algo = SHA_512;\n+\t\thash_oneblock_fn = mb_ops->aux.one_block.sha512;\n+\t\tbreak;\n+\tdefault:\n+\t\tMB_LOG_ERR(\"Unsupported authentication algorithm selection\");\n+\t\treturn -1;\n+\t}\n+\n+\t/* Calculate Authentication precomputes */\n+\tcalculate_auth_precomputes(hash_oneblock_fn,\n+\t\t\tsess->auth.pads.inner, sess->auth.pads.outer,\n+\t\t\txform->auth.key.data,\n+\t\t\txform->auth.key.length,\n+\t\t\tget_auth_algo_blocksize(sess->auth.algo));\n+\n+\treturn 0;\n+}\n+\n+/** Set session cipher parameters */\n+static int\n+aesni_mb_set_session_cipher_parameters(const struct aesni_mb_ops *mb_ops,\n+\t\tstruct aesni_mb_session *sess,\n+\t\tconst struct rte_crypto_xform *xform)\n+{\n+\taes_keyexp_t aes_keyexp_fn;\n+\n+\tif (xform->type != RTE_CRYPTO_XFORM_CIPHER) {\n+\t\tMB_LOG_ERR(\"Crypto xform struct not of type cipher\");\n+\t\treturn -1;\n+\t}\n+\n+\t/* Select cipher direction */\n+\tswitch (xform->cipher.op) {\n+\tcase RTE_CRYPTO_CIPHER_OP_ENCRYPT:\n+\t\tsess->cipher.direction = ENCRYPT;\n+\t\tbreak;\n+\tcase RTE_CRYPTO_CIPHER_OP_DECRYPT:\n+\t\tsess->cipher.direction = DECRYPT;\n+\t\tbreak;\n+\tdefault:\n+\t\tMB_LOG_ERR(\"Unsupported cipher operation parameter\");\n+\t\treturn -1;\n+\t}\n+\n+\n+\t/* Select cipher mode */\n+\tswitch (xform->cipher.algo) {\n+\tcase RTE_CRYPTO_CIPHER_AES_CBC:\n+\t\tsess->cipher.mode = CBC;\n+\t\tbreak;\n+\tdefault:\n+\t\tMB_LOG_ERR(\"Unsupported cipher mode parameter\");\n+\t\treturn -1;\n+\t}\n+\n+\t/* Check key length and choose key expansion function */\n+\tswitch (xform->cipher.key.length) {\n+\tcase AES_128_BYTES:\n+\t\tsess->cipher.key_length_in_bytes = AES_128_BYTES;\n+\t\taes_keyexp_fn = mb_ops->aux.keyexp.aes128;\n+\t\tbreak;\n+\tcase AES_192_BYTES:\n+\t\tsess->cipher.key_length_in_bytes = AES_192_BYTES;\n+\t\taes_keyexp_fn = mb_ops->aux.keyexp.aes192;\n+\t\tbreak;\n+\tcase AES_256_BYTES:\n+\t\tsess->cipher.key_length_in_bytes = AES_256_BYTES;\n+\t\taes_keyexp_fn = mb_ops->aux.keyexp.aes256;\n+\t\tbreak;\n+\tdefault:\n+\t\tMB_LOG_ERR(\"Unsupported cipher key length\");\n+\t\treturn -1;\n+\t}\n+\n+\t/* Expanded cipher keys */\n+\t(*aes_keyexp_fn)(xform->cipher.key.data,\n+\t\t\tsess->cipher.expanded_aes_keys.encode,\n+\t\t\tsess->cipher.expanded_aes_keys.decode);\n+\n+\treturn 0;\n+}\n+\n+/** Parse crypto xform chain and set private session parameters */\n+int\n+aesni_mb_set_session_parameters(const struct aesni_mb_ops *mb_ops,\n+\t\tstruct aesni_mb_session *sess,\n+\t\tconst struct rte_crypto_xform *xform)\n+{\n+\tconst struct rte_crypto_xform *auth_xform = NULL;\n+\tconst struct rte_crypto_xform *cipher_xform = NULL;\n+\n+\t/* Select Crypto operation - hash then cipher / cipher then hash */\n+\tswitch (aesni_mb_get_chain_order(xform)) {\n+\tcase HASH_CIPHER:\n+\t\tsess->chain_order = HASH_CIPHER;\n+\t\tauth_xform = xform;\n+\t\tcipher_xform = xform->next;\n+\t\tbreak;\n+\tcase CIPHER_HASH:\n+\t\tsess->chain_order = CIPHER_HASH;\n+\t\tauth_xform = xform->next;\n+\t\tcipher_xform = xform;\n+\t\tbreak;\n+\tdefault:\n+\t\tMB_LOG_ERR(\"Unsupported operation chain order parameter\");\n+\t\treturn -1;\n+\t}\n+\n+\tif (aesni_mb_set_session_auth_parameters(mb_ops, sess, auth_xform)) {\n+\t\tMB_LOG_ERR(\"Invalid/unsupported authentication parameters\");\n+\t\treturn -1;\n+\t}\n+\n+\tif (aesni_mb_set_session_cipher_parameters(mb_ops, sess, cipher_xform)) {\n+\t\tMB_LOG_ERR(\"Invalid/unsupported cipher parameters\");\n+\t\treturn -1;\n+\t}\n+\treturn 0;\n+}\n+\n+/** Get multi buffer session */\n+static struct aesni_mb_session *\n+aesni_mb_get_session(struct aesni_mb_qp *qp, struct rte_crypto_op *crypto_op)\n+{\n+\tstruct aesni_mb_session *sess;\n+\n+\tif (crypto_op->type == RTE_CRYPTO_OP_WITH_SESSION) {\n+\t\tif (unlikely(crypto_op->session->type !=\n+\t\t\t\tRTE_CRYPTODEV_AESNI_MB_PMD))\n+\t\t\treturn NULL;\n+\n+\t\tsess = (struct aesni_mb_session *)crypto_op->session->_private;\n+\t} else  {\n+\t\tstruct rte_cryptodev_session *c_sess = NULL;\n+\n+\t\tif (rte_mempool_get(qp->sess_mp, (void **)&c_sess))\n+\t\t\treturn NULL;\n+\n+\t\tsess = (struct aesni_mb_session *)c_sess->_private;\n+\n+\t\tif (unlikely(aesni_mb_set_session_parameters(qp->mb_ops,\n+\t\t\t\tsess, crypto_op->xform) != 0))\n+\t\t\treturn NULL;\n+\t}\n+\n+\treturn sess;\n+}\n+\n+/**\n+ * Process a crypto operation and complete a JOB_AES_HMAC job structure for\n+ * submission to the multi buffer library for processing.\n+ *\n+ * @param\tqp\tqueue pair\n+ * @param\tjob\tJOB_AES_HMAC structure to fill\n+ * @param\tm\tmbuf to process\n+ *\n+ * @return\n+ * - Completed JOB_AES_HMAC structure pointer on success\n+ * - NULL pointer if completion of JOB_AES_HMAC structure isn't possible\n+ */\n+static JOB_AES_HMAC *\n+process_mb_crypto_op(struct aesni_mb_qp *qp, struct rte_mbuf *m,\n+\t\tstruct rte_crypto_op *c_op, struct aesni_mb_session *session)\n+{\n+\tJOB_AES_HMAC *job;\n+\n+\tjob = (*qp->mb_ops->job.get_next)(&qp->mb_mgr);\n+\tif (unlikely(job == NULL))\n+\t\treturn job;\n+\n+\t/* Set crypto operation */\n+\tjob->chain_order = session->chain_order;\n+\n+\t/* Set cipher parameters */\n+\tjob->cipher_direction = session->cipher.direction;\n+\tjob->cipher_mode = session->cipher.mode;\n+\n+\tjob->aes_key_len_in_bytes = session->cipher.key_length_in_bytes;\n+\tjob->aes_enc_key_expanded = session->cipher.expanded_aes_keys.encode;\n+\tjob->aes_dec_key_expanded = session->cipher.expanded_aes_keys.decode;\n+\n+\n+\t/* Set authentication parameters */\n+\tjob->hash_alg = session->auth.algo;\n+\tif (job->hash_alg == AES_XCBC) {\n+\t\tjob->_k1_expanded = session->auth.xcbc.k1_expanded;\n+\t\tjob->_k2 = session->auth.xcbc.k2;\n+\t\tjob->_k3 = session->auth.xcbc.k3;\n+\t} else {\n+\t\tjob->hashed_auth_key_xor_ipad = session->auth.pads.inner;\n+\t\tjob->hashed_auth_key_xor_opad = session->auth.pads.outer;\n+\t}\n+\n+\t/* Mutable crypto operation parameters */\n+\n+\t/* Set digest output location */\n+\tif (job->cipher_direction == DECRYPT) {\n+\t\tjob->auth_tag_output = (uint8_t *)rte_pktmbuf_append(m,\n+\t\t\t\tget_digest_byte_length(job->hash_alg));\n+\n+\t\tif (job->auth_tag_output)\n+\t\t\tmemset(job->auth_tag_output, 0,\n+\t\t\t\tsizeof(get_digest_byte_length(job->hash_alg)));\n+\t\telse\n+\t\t\treturn NULL;\n+\t} else {\n+\t\tjob->auth_tag_output = c_op->digest.data;\n+\t}\n+\n+\t/* Multiple buffer library current only support returning a truncated digest length\n+\t * as specified in the relevant IPsec RFCs */\n+\tjob->auth_tag_output_len_in_bytes =\n+\t\t\tget_truncated_digest_byte_length(job->hash_alg);\n+\n+\t/* Set IV parameters */\n+\tjob->iv = c_op->iv.data;\n+\tjob->iv_len_in_bytes = c_op->iv.length;\n+\n+\t/* Data  Parameter */\n+\tjob->src = rte_pktmbuf_mtod(m, uint8_t *);\n+\tjob->dst = c_op->dst.m ?\n+\t\t\trte_pktmbuf_mtod(c_op->dst.m, uint8_t *) +\n+\t\t\tc_op->dst.offset :\n+\t\t\trte_pktmbuf_mtod(m, uint8_t *) +\n+\t\t\tc_op->data.to_cipher.offset;\n+\n+\tjob->cipher_start_src_offset_in_bytes = c_op->data.to_cipher.offset;\n+\tjob->msg_len_to_cipher_in_bytes = c_op->data.to_cipher.length;\n+\n+\tjob->hash_start_src_offset_in_bytes = c_op->data.to_hash.offset;\n+\tjob->msg_len_to_hash_in_bytes = c_op->data.to_hash.length;\n+\n+\t/* Set user data to be crypto operation data struct */\n+\tjob->user_data = m;\n+\tjob->user_data2 = c_op;\n+\n+\treturn job;\n+}\n+\n+/**\n+ * Process a crypto operation and complete a JOB_AES_HMAC job structure for\n+ * submission to the multi buffer library for processing.\n+ *\n+ * @param\tqp\tqueue pair\n+ * @param\tjob\tJOB_AES_HMAC structure to fill\n+ * @param\tm\tmbuf to process\n+ *\n+ * @return\n+ *\n+ */\n+static int\n+process_gcm_crypto_op(struct aesni_mb_qp *qp, struct rte_mbuf *m,\n+\t\tstruct rte_crypto_op *c_op, struct aesni_mb_session *session)\n+{\n+\tuint8_t *src, *dst;\n+\n+\tsrc = rte_pktmbuf_mtod(m, uint8_t *) + c_op->data.to_cipher.offset;\n+\tdst = c_op->dst.m ?\n+\t\t\trte_pktmbuf_mtod(c_op->dst.m, uint8_t *) +\n+\t\t\tc_op->dst.offset :\n+\t\t\trte_pktmbuf_mtod(m, uint8_t *) +\n+\t\t\tc_op->data.to_cipher.offset;\n+\n+\tif (session->cipher.direction == ENCRYPT) {\n+\n+\t\t(*qp->mb_ops->gcm.enc)(&session->gdata, dst, src,\n+\t\t\t\t(uint64_t)c_op->data.to_cipher.length,\n+\t\t\t\tc_op->iv.data,\n+\t\t\t\tc_op->additional_auth.data,\n+\t\t\t\t(uint64_t)c_op->additional_auth.length,\n+\t\t\t\tc_op->digest.data,\n+\t\t\t\t(uint64_t)c_op->digest.length);\n+\t} else {\n+\t\tuint8_t *auth_tag = (uint8_t *)rte_pktmbuf_append(m,\n+\t\t\t\tc_op->digest.length);\n+\n+\t\tif (!auth_tag)\n+\t\t\treturn -1;\n+\n+\t\t(*qp->mb_ops->gcm.dec)(&session->gdata, dst, src,\n+\t\t\t\t(uint64_t)c_op->data.to_cipher.length,\n+\t\t\t\tc_op->iv.data,\n+\t\t\t\tc_op->additional_auth.data,\n+\t\t\t\t(uint64_t)c_op->additional_auth.length,\n+\t\t\t\tauth_tag,\n+\t\t\t\t(uint64_t)c_op->digest.length);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/**\n+ * Process a completed job and return rte_mbuf which job processed\n+ *\n+ * @param job\tJOB_AES_HMAC job to process\n+ *\n+ * @return\n+ * - Returns processed mbuf which is trimmed of output digest used in\n+ * verification of supplied digest in the case of a HASH_CIPHER operation\n+ * - Returns NULL on invalid job\n+ */\n+static struct rte_mbuf *\n+post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job)\n+{\n+\tstruct rte_mbuf *m;\n+\tstruct rte_crypto_op *c_op;\n+\n+\tif (job->user_data == NULL)\n+\t\treturn NULL;\n+\n+\t/* handled retrieved job */\n+\tm = (struct rte_mbuf *)job->user_data;\n+\tc_op = (struct rte_crypto_op *)job->user_data2;\n+\n+\t/* check if job has been processed  */\n+\tif (unlikely(job->status != STS_COMPLETED)) {\n+\t\tc_op->status = RTE_CRYPTO_OP_STATUS_ERROR;\n+\t\treturn m;\n+\t} else if (job->chain_order == HASH_CIPHER) {\n+\t\t/* Verify digest if required */\n+\t\tif (memcmp(job->auth_tag_output, c_op->digest.data,\n+\t\t\t\tjob->auth_tag_output_len_in_bytes) != 0)\n+\t\t\tc_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;\n+\t\telse\n+\t\t\tc_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;\n+\n+\t\t/* trim area used for digest from mbuf */\n+\t\trte_pktmbuf_trim(m, get_digest_byte_length(job->hash_alg));\n+\t} else {\n+\t\tc_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;\n+\t}\n+\n+\t/* Free session if a session-less crypto op */\n+\tif (c_op->type == RTE_CRYPTO_OP_SESSIONLESS) {\n+\t\trte_mempool_put(qp->sess_mp, c_op->session);\n+\t\tc_op->session = NULL;\n+\t}\n+\n+\treturn m;\n+}\n+\n+/**\n+ * Process a completed JOB_AES_HMAC job and keep processing jobs until\n+ * get_completed_job return NULL\n+ *\n+ * @param qp\t\tQueue Pair to process\n+ * @param job\t\tJOB_AES_HMAC job\n+ *\n+ * @return\n+ * - Number of processed jobs\n+ */\n+static unsigned\n+handle_completed_mb_jobs(struct aesni_mb_qp *qp, JOB_AES_HMAC *job)\n+{\n+\tstruct rte_mbuf *m = NULL;\n+\tunsigned processed_jobs = 0;\n+\n+\twhile (job) {\n+\t\tprocessed_jobs++;\n+\t\tm = post_process_mb_job(qp, job);\n+\t\tif (m)\n+\t\t\trte_ring_enqueue(qp->processed_pkts, (void *)m);\n+\t\telse\n+\t\t\tqp->qp_stats.dequeue_err_count++;\n+\n+\t\tjob = (*qp->mb_ops->job.get_completed_job)(&qp->mb_mgr);\n+\t}\n+\n+\treturn processed_jobs;\n+}\n+\n+/**\n+ * Process a completed job and return rte_mbuf which job processed\n+ *\n+ * @param job\tJOB_AES_HMAC job to process\n+ *\n+ * @return\n+ * - Returns processed mbuf which is trimmed of output digest used in\n+ * verification of supplied digest in the case of a HASH_CIPHER operation\n+ * - Returns NULL on invalid job\n+ */\n+static struct rte_mbuf *\n+post_process_gcm_crypto_op(struct rte_mbuf *m, struct rte_crypto_op *c_op)\n+{\n+\tstruct aesni_mb_session *session =\n+\t\t\t(struct aesni_mb_session *)c_op->session->_private;\n+\n+\t/* Verify digest if required */\n+\tif (session->cipher.direction == DECRYPT) {\n+\n+\t\tuint8_t *auth_tag = rte_pktmbuf_mtod_offset(m, uint8_t *,\n+\t\t\t\tm->data_len - c_op->digest.length);\n+\n+\t\tif (memcmp(auth_tag, c_op->digest.data,\n+\t\t\t\tc_op->digest.length) != 0)\n+\t\t\tc_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;\n+\t\telse\n+\t\t\tc_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;\n+\n+\t\t/* trim area used for digest from mbuf */\n+\t\trte_pktmbuf_trim(m, c_op->digest.length);\n+\t} else {\n+\t\tc_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;\n+\t}\n+\n+\treturn m;\n+}\n+\n+/**\n+ * Process a completed GCM request\n+ *\n+ * @param qp\t\tQueue Pair to process\n+ * @param job\t\tJOB_AES_HMAC job\n+ *\n+ * @return\n+ * - Number of processed jobs\n+ */\n+static unsigned\n+handle_completed_gcm_crypto_op(struct aesni_mb_qp *qp, struct rte_mbuf *m,\n+\t\tstruct rte_crypto_op *c_op)\n+{\n+\tm = post_process_gcm_crypto_op(m, c_op);\n+\n+\t/* Free session if a session-less crypto op */\n+\tif (c_op->type == RTE_CRYPTO_OP_SESSIONLESS) {\n+\t\trte_mempool_put(qp->sess_mp, c_op->session);\n+\t\tc_op->session = NULL;\n+\t}\n+\n+\trte_ring_enqueue(qp->processed_pkts, (void *)m);\n+\n+\treturn 0;\n+}\n+\n+static uint16_t\n+aesni_mb_pmd_enqueue_burst(void *queue_pair, struct rte_mbuf **bufs,\n+\t\tuint16_t nb_bufs)\n+{\n+\tstruct rte_mbuf_offload *ol;\n+\tstruct rte_crypto_op *c_op;\n+\n+\tstruct aesni_mb_session *sess;\n+\tstruct aesni_mb_qp *qp = queue_pair;\n+\tJOB_AES_HMAC *job = NULL;\n+\n+\tint i, retval, processed_jobs = 0;\n+\n+\tfor (i = 0; i < nb_bufs; i++) {\n+\t\tol = rte_pktmbuf_offload_get(bufs[i], RTE_PKTMBUF_OL_CRYPTO);\n+\t\tif (unlikely(ol == NULL)) {\n+\t\t\tqp->qp_stats.enqueue_err_count++;\n+\t\t\tgoto flush_jobs;\n+\t\t}\n+\t\tc_op = &ol->op.crypto;\n+\n+\t\tsess = aesni_mb_get_session(qp, c_op);\n+\t\tif (unlikely(sess == NULL)) {\n+\t\t\tqp->qp_stats.enqueue_err_count++;\n+\t\t\tgoto flush_jobs;\n+\t\t}\n+\n+\t\tif (sess->gcm_session) {\n+\t\t\tretval = process_gcm_crypto_op(qp, bufs[i], c_op, sess);\n+\t\t\tif (retval < 0) {\n+\t\t\t\tqp->qp_stats.enqueue_err_count++;\n+\t\t\t\tgoto flush_jobs;\n+\t\t\t}\n+\n+\t\t\thandle_completed_gcm_crypto_op(qp, bufs[i], c_op);\n+\t\t\tprocessed_jobs++;\n+\t\t} else {\n+\t\t\tjob = process_mb_crypto_op(qp, bufs[i], c_op, sess);\n+\t\t\tif (unlikely(job == NULL)) {\n+\t\t\t\tqp->qp_stats.enqueue_err_count++;\n+\t\t\t\tgoto flush_jobs;\n+\t\t\t}\n+\n+\t\t\t/* Submit Job */\n+\t\t\tjob = (*qp->mb_ops->job.submit)(&qp->mb_mgr);\n+\n+\t\t\t/* If submit returns a processed job then handle it,\n+\t\t\t * before submitting subsequent jobs */\n+\t\t\tif (job)\n+\t\t\t\tprocessed_jobs +=\n+\t\t\t\t\thandle_completed_mb_jobs(qp, job);\n+\t\t}\n+\t}\n+\n+\tif (processed_jobs == 0)\n+\t\tgoto flush_jobs;\n+\telse\n+\t\tqp->qp_stats.enqueued_count += processed_jobs;\n+\t\treturn i;\n+\n+flush_jobs:\n+\t/* if we haven't processed any jobs in submit loop, then flush jobs\n+\t * queue to stop the output stalling */\n+\tjob = (*qp->mb_ops->job.flush_job)(&qp->mb_mgr);\n+\tif (job)\n+\t\tqp->qp_stats.enqueued_count +=\n+\t\t\t\thandle_completed_mb_jobs(qp, job);\n+\n+\treturn i;\n+}\n+\n+static uint16_t\n+aesni_mb_pmd_dequeue_burst(void *queue_pair,\n+\t\tstruct rte_mbuf **bufs,\tuint16_t nb_bufs)\n+{\n+\tstruct aesni_mb_qp *qp = queue_pair;\n+\n+\tunsigned nb_dequeued;\n+\n+\tnb_dequeued = rte_ring_dequeue_burst(qp->processed_pkts,\n+\t\t\t(void **)bufs, nb_bufs);\n+\tqp->qp_stats.dequeued_count += nb_dequeued;\n+\n+\treturn nb_dequeued;\n+}\n+\n+\n+static int cryptodev_aesni_mb_uninit(const char *name);\n+\n+static int\n+cryptodev_aesni_mb_create(const char *name, unsigned socket_id)\n+{\n+\tstruct rte_cryptodev *dev;\n+\tchar crypto_dev_name[RTE_CRYPTODEV_NAME_MAX_LEN];\n+\tstruct aesni_mb_private *internals;\n+\tenum aesni_mb_vector_mode vector_mode;\n+\n+\t/* Check CPU for support for AES instruction set */\n+\tif (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {\n+\t\tMB_LOG_ERR(\"AES instructions not supported by CPU\");\n+\t\treturn -EFAULT;\n+\t}\n+\n+\t/* Check CPU for supported vector instruction set */\n+\tif (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2))\n+\t\tvector_mode = RTE_AESNI_MB_AVX2;\n+\telse if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX))\n+\t\tvector_mode = RTE_AESNI_MB_AVX;\n+\telse if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_SSE4_1))\n+\t\tvector_mode = RTE_AESNI_MB_SSE;\n+\telse {\n+\t\tMB_LOG_ERR(\"Vector instructions are not supported by CPU\");\n+\t\treturn -EFAULT;\n+\t}\n+\n+\t/* create a unique device name */\n+\tif (create_unique_device_name(crypto_dev_name,\n+\t\t\tRTE_CRYPTODEV_NAME_MAX_LEN) != 0) {\n+\t\tMB_LOG_ERR(\"failed to create unique cryptodev name\");\n+\t\treturn -EINVAL;\n+\t}\n+\n+\n+\tdev = rte_cryptodev_pmd_virtual_dev_init(crypto_dev_name,\n+\t\t\tsizeof(struct aesni_mb_private), socket_id);\n+\tif (dev == NULL) {\n+\t\tMB_LOG_ERR(\"failed to create cryptodev vdev\");\n+\t\tgoto init_error;\n+\t}\n+\n+\tdev->dev_type = RTE_CRYPTODEV_AESNI_MB_PMD;\n+\tdev->dev_ops = rte_aesni_mb_pmd_ops;\n+\n+\t/* register rx/tx burst functions for data path */\n+\tdev->dequeue_burst = aesni_mb_pmd_dequeue_burst;\n+\tdev->enqueue_burst = aesni_mb_pmd_enqueue_burst;\n+\n+\t/* Set vector instructions mode supported */\n+\tinternals = dev->data->dev_private;\n+\n+\tinternals->vector_mode = vector_mode;\n+\tinternals->max_nb_qpairs = AESNI_MB_MAX_NB_QUEUE_PAIRS;\n+\n+\treturn dev->data->dev_id;\n+init_error:\n+\tMB_LOG_ERR(\"driver %s: cryptodev_aesni_create failed\", name);\n+\n+\tcryptodev_aesni_mb_uninit(crypto_dev_name);\n+\treturn -EFAULT;\n+}\n+\n+\n+static int\n+cryptodev_aesni_mb_init(const char *name,\n+\t\tconst char *params __rte_unused)\n+{\n+\tRTE_LOG(INFO, PMD, \"Initialising %s\\n\", name);\n+\n+\treturn cryptodev_aesni_mb_create(name, rte_socket_id());\n+}\n+\n+static int\n+cryptodev_aesni_mb_uninit(const char *name)\n+{\n+\tif (name == NULL)\n+\t\treturn -EINVAL;\n+\n+\tRTE_LOG(INFO, PMD, \"Closing AESNI crypto device %s on numa socket %u\\n\",\n+\t\t\tname, rte_socket_id());\n+\n+\treturn 0;\n+}\n+\n+static struct rte_driver cryptodev_aesni_mb_pmd_drv = {\n+\t.name = CRYPTODEV_NAME_AESNI_MB_PMD,\n+\t.type = PMD_VDEV,\n+\t.init = cryptodev_aesni_mb_init,\n+\t.uninit = cryptodev_aesni_mb_uninit\n+};\n+\n+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv);\ndiff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c\nnew file mode 100644\nindex 0000000..41b8d04\n--- /dev/null\n+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c\n@@ -0,0 +1,296 @@\n+/*-\n+ *   BSD LICENSE\n+ *\n+ *   Copyright(c) 2015 Intel Corporation. All rights reserved.\n+ *\n+ *   Redistribution and use in source and binary forms, with or without\n+ *   modification, are permitted provided that the following conditions\n+ *   are met:\n+ *\n+ *     * Redistributions of source code must retain the above copyright\n+ *       notice, this list of conditions and the following disclaimer.\n+ *     * Redistributions in binary form must reproduce the above copyright\n+ *       notice, this list of conditions and the following disclaimer in\n+ *       the documentation and/or other materials provided with the\n+ *       distribution.\n+ *     * Neither the name of Intel Corporation nor the names of its\n+ *       contributors may be used to endorse or promote products derived\n+ *       from this software without specific prior written permission.\n+ *\n+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ *   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ */\n+\n+#include <string.h>\n+\n+#include <rte_common.h>\n+#include <rte_malloc.h>\n+#include <rte_cryptodev_pmd.h>\n+\n+#include \"rte_aesni_mb_pmd_private.h\"\n+\n+/** Configure device */\n+static int\n+aesni_mb_pmd_config(__rte_unused struct rte_cryptodev *dev)\n+{\n+\treturn 0;\n+}\n+\n+/** Start device */\n+static int\n+aesni_mb_pmd_start(__rte_unused struct rte_cryptodev *dev)\n+{\n+\treturn 0;\n+}\n+\n+/** Stop device */\n+static void\n+aesni_mb_pmd_stop(__rte_unused struct rte_cryptodev *dev)\n+{\n+}\n+\n+/** Close device */\n+static int\n+aesni_mb_pmd_close(__rte_unused struct rte_cryptodev *dev)\n+{\n+\treturn 0;\n+}\n+\n+\n+/** Get device statistics */\n+static void\n+aesni_mb_pmd_stats_get(struct rte_cryptodev *dev,\n+\t\tstruct rte_cryptodev_stats *stats)\n+{\n+\tint qp_id;\n+\n+\tfor (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {\n+\t\tstruct aesni_mb_qp *qp = dev->data->queue_pairs[qp_id];\n+\n+\t\tstats->enqueued_count += qp->qp_stats.enqueued_count;\n+\t\tstats->dequeued_count += qp->qp_stats.dequeued_count;\n+\n+\t\tstats->enqueue_err_count += qp->qp_stats.enqueue_err_count;\n+\t\tstats->dequeue_err_count += qp->qp_stats.dequeue_err_count;\n+\t}\n+}\n+\n+/** Reset device statistics */\n+static void\n+aesni_mb_pmd_stats_reset(struct rte_cryptodev *dev)\n+{\n+\tint qp_id;\n+\n+\tfor (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {\n+\t\tstruct aesni_mb_qp *qp = dev->data->queue_pairs[qp_id];\n+\n+\t\tmemset(&qp->qp_stats, 0, sizeof(qp->qp_stats));\n+\t}\n+}\n+\n+\n+/** Get device info */\n+static void\n+aesni_mb_pmd_info_get(struct rte_cryptodev *dev,\n+\t\tstruct rte_cryptodev_info *dev_info)\n+{\n+\tstruct aesni_mb_private *internals = dev->data->dev_private;\n+\n+\tif (dev_info != NULL) {\n+\t\tdev_info->dev_type = dev->dev_type;\n+\t\tdev_info->max_queue_pairs = internals->max_nb_qpairs;\n+\t}\n+}\n+\n+/** Release queue pair */\n+static int\n+aesni_mb_pmd_qp_release(struct rte_cryptodev *dev, uint16_t qp_id)\n+{\n+\tif (dev->data->queue_pairs[qp_id] != NULL) {\n+\t\trte_free(dev->data->queue_pairs[qp_id]);\n+\t\tdev->data->queue_pairs[qp_id] = NULL;\n+\t}\n+\treturn 0;\n+}\n+\n+/** set a unique name for the queue pair based on it's name, dev_id and qp_id */\n+static int\n+aesni_mb_pmd_qp_set_unique_name(struct rte_cryptodev *dev,\n+\t\tstruct aesni_mb_qp *qp)\n+{\n+\tunsigned n = snprintf(qp->name, sizeof(qp->name),\n+\t\t\t\"aesni_mb_pmd_%u_qp_%u\",\n+\t\t\tdev->data->dev_id, qp->id);\n+\n+\tif (n > sizeof(qp->name))\n+\t\treturn -1;\n+\n+\treturn 0;\n+}\n+\n+/** Create a ring to place process packets on */\n+static struct rte_ring *\n+aesni_mb_pmd_qp_create_processed_pkts_ring(struct aesni_mb_qp *qp,\n+\t\tunsigned ring_size, int socket_id)\n+{\n+\tstruct rte_ring *r;\n+\n+\tr = rte_ring_lookup(qp->name);\n+\tif (r) {\n+\t\tif (r->prod.size >= ring_size) {\n+\t\t\tMB_LOG_INFO(\"Reusing existing ring %s for processed packets\",\n+\t\t\t\t\t qp->name);\n+\t\t\treturn r;\n+\t\t}\n+\n+\t\tMB_LOG_ERR(\"Unable to reuse existing ring %s for processed packets\",\n+\t\t\t\t qp->name);\n+\t\treturn NULL;\n+\t}\n+\n+\treturn rte_ring_create(qp->name, ring_size, socket_id,\n+\t\t\tRING_F_SP_ENQ | RING_F_SC_DEQ);\n+}\n+\n+/** Setup a queue pair */\n+static int\n+aesni_mb_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,\n+\t\tconst struct rte_cryptodev_qp_conf *qp_conf,\n+\t\t int socket_id)\n+{\n+\tstruct aesni_mb_qp *qp = NULL;\n+\tstruct aesni_mb_private *internals = dev->data->dev_private;\n+\n+\t/* Free memory prior to re-allocation if needed. */\n+\tif (dev->data->queue_pairs[qp_id] != NULL)\n+\t\taesni_mb_pmd_qp_release(dev, qp_id);\n+\n+\t/* Allocate the queue pair data structure. */\n+\tqp = rte_zmalloc_socket(\"AES-NI PMD Queue Pair\", sizeof(*qp),\n+\t\t\t\t\tRTE_CACHE_LINE_SIZE, socket_id);\n+\tif (qp == NULL)\n+\t\treturn (-ENOMEM);\n+\n+\tqp->id = qp_id;\n+\tdev->data->queue_pairs[qp_id] = qp;\n+\n+\tif (aesni_mb_pmd_qp_set_unique_name(dev, qp))\n+\t\tgoto qp_setup_cleanup;\n+\n+\tqp->mb_ops = &job_ops[internals->vector_mode];\n+\n+\tqp->processed_pkts = aesni_mb_pmd_qp_create_processed_pkts_ring(qp,\n+\t\t\tqp_conf->nb_descriptors, socket_id);\n+\tif (qp->processed_pkts == NULL)\n+\t\tgoto qp_setup_cleanup;\n+\n+\tqp->sess_mp = dev->data->session_pool;\n+\n+\tmemset(&qp->qp_stats, 0, sizeof(qp->qp_stats));\n+\n+\t/* Initialise multi-buffer manager */\n+\t(*qp->mb_ops->job.init_mgr)(&qp->mb_mgr);\n+\n+\treturn 0;\n+\n+qp_setup_cleanup:\n+\tif (qp)\n+\t\trte_free(qp);\n+\n+\treturn -1;\n+}\n+\n+/** Start queue pair */\n+static int\n+aesni_mb_pmd_qp_start(__rte_unused struct rte_cryptodev *dev,\n+\t\t__rte_unused uint16_t queue_pair_id)\n+{\n+\treturn -ENOTSUP;\n+}\n+\n+/** Stop queue pair */\n+static int\n+aesni_mb_pmd_qp_stop(__rte_unused struct rte_cryptodev *dev,\n+\t\t__rte_unused uint16_t queue_pair_id)\n+{\n+\treturn -ENOTSUP;\n+}\n+\n+/** Return the number of allocated queue pairs */\n+static uint32_t\n+aesni_mb_pmd_qp_count(struct rte_cryptodev *dev)\n+{\n+\treturn dev->data->nb_queue_pairs;\n+}\n+\n+/** Returns the size of the aesni multi-buffer session structure */\n+static unsigned\n+aesni_mb_pmd_session_get_size(struct rte_cryptodev *dev __rte_unused)\n+{\n+\treturn sizeof(struct aesni_mb_session);\n+}\n+\n+/** Configure a aesni multi-buffer session from a crypto xform chain */\n+static void *\n+aesni_mb_pmd_session_configure(struct rte_cryptodev *dev,\n+\t\tstruct rte_crypto_xform *xform,\tvoid *sess)\n+{\n+\tstruct aesni_mb_private *internals = dev->data->dev_private;\n+\n+\tif (unlikely(sess == NULL)) {\n+\t\tMB_LOG_ERR(\"invalid session struct\");\n+\t\treturn NULL;\n+\t}\n+\n+\tif (aesni_mb_set_session_parameters(&job_ops[internals->vector_mode],\n+\t\t\tsess, xform) != 0) {\n+\t\tMB_LOG_ERR(\"failed configure session parameters\");\n+\t\treturn NULL;\n+\t}\n+\n+\treturn sess;\n+}\n+\n+/** Clear the memory of session so it doesn't leave key material behind */\n+static void\n+aesni_mb_pmd_session_clear(struct rte_cryptodev *dev __rte_unused, void *sess)\n+{\n+\t/* Current just resetting the whole data structure, need to investigate\n+\t * whether a more selective reset of key would be more performant */\n+\tif (sess)\n+\t\tmemset(sess, 0, sizeof(struct aesni_mb_session));\n+}\n+\n+struct rte_cryptodev_ops aesni_mb_pmd_ops = {\n+\t\t.dev_configure\t\t= aesni_mb_pmd_config,\n+\t\t.dev_start\t\t= aesni_mb_pmd_start,\n+\t\t.dev_stop\t\t= aesni_mb_pmd_stop,\n+\t\t.dev_close\t\t= aesni_mb_pmd_close,\n+\n+\t\t.stats_get\t\t= aesni_mb_pmd_stats_get,\n+\t\t.stats_reset\t\t= aesni_mb_pmd_stats_reset,\n+\n+\t\t.dev_infos_get\t\t= aesni_mb_pmd_info_get,\n+\n+\t\t.queue_pair_setup\t= aesni_mb_pmd_qp_setup,\n+\t\t.queue_pair_release\t= aesni_mb_pmd_qp_release,\n+\t\t.queue_pair_start\t= aesni_mb_pmd_qp_start,\n+\t\t.queue_pair_stop\t= aesni_mb_pmd_qp_stop,\n+\t\t.queue_pair_count\t= aesni_mb_pmd_qp_count,\n+\n+\t\t.session_get_size\t= aesni_mb_pmd_session_get_size,\n+\t\t.session_configure\t= aesni_mb_pmd_session_configure,\n+\t\t.session_clear\t\t= aesni_mb_pmd_session_clear\n+};\n+\n+struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops = &aesni_mb_pmd_ops;\n+\ndiff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h\nnew file mode 100644\nindex 0000000..e5e317b\n--- /dev/null\n+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h\n@@ -0,0 +1,230 @@\n+/*-\n+ *   BSD LICENSE\n+ *\n+ *   Copyright(c) 2015 Intel Corporation. All rights reserved.\n+ *\n+ *   Redistribution and use in source and binary forms, with or without\n+ *   modification, are permitted provided that the following conditions\n+ *   are met:\n+ *\n+ *     * Redistributions of source code must retain the above copyright\n+ *       notice, this list of conditions and the following disclaimer.\n+ *     * Redistributions in binary form must reproduce the above copyright\n+ *       notice, this list of conditions and the following disclaimer in\n+ *       the documentation and/or other materials provided with the\n+ *       distribution.\n+ *     * Neither the name of Intel Corporation nor the names of its\n+ *       contributors may be used to endorse or promote products derived\n+ *       from this software without specific prior written permission.\n+ *\n+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ *   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ */\n+\n+#ifndef _RTE_AESNI_MB_PMD_PRIVATE_H_\n+#define _RTE_AESNI_MB_PMD_PRIVATE_H_\n+\n+#include \"aesni_mb_ops.h\"\n+\n+#define MB_LOG_ERR(fmt, args...) \\\n+\tRTE_LOG(ERR, CRYPTODEV, \"[%s] %s() line %u: \" fmt \"\\n\",  \\\n+\t\t\tCRYPTODEV_NAME_AESNI_MB_PMD, \\\n+\t\t\t__func__, __LINE__, ## args)\n+\n+#ifdef RTE_LIBRTE_AESNI_MB_DEBUG\n+#define MB_LOG_INFO(fmt, args...) \\\n+\tRTE_LOG(INFO, CRYPTODEV, \"[%s] %s() line %u: \" fmt \"\\n\", \\\n+\t\t\tCRYPTODEV_NAME_AESNI_MB_PMD, \\\n+\t\t\t__func__, __LINE__, ## args)\n+\n+#define MB_LOG_DBG(fmt, args...) \\\n+\tRTE_LOG(DEBUG, CRYPTODEV, \"[%s] %s() line %u: \" fmt \"\\n\", \\\n+\t\t\tCRYPTODEV_NAME_AESNI_MB_PMD, \\\n+\t\t\t__func__, __LINE__, ## args)\n+#else\n+#define MB_LOG_INFO(fmt, args...)\n+#define MB_LOG_DBG(fmt, args...)\n+#endif\n+\n+#define AESNI_MB_NAME_MAX_LENGTH\t(64)\n+#define AESNI_MB_MAX_NB_QUEUE_PAIRS\t(4)\n+\n+#define HMAC_IPAD_VALUE\t\t\t(0x36)\n+#define HMAC_OPAD_VALUE\t\t\t(0x5C)\n+\n+static const unsigned auth_blocksize[] = {\n+\t\t[MD5]\t\t= 64,\n+\t\t[SHA1]\t\t= 64,\n+\t\t[SHA_224]\t= 64,\n+\t\t[SHA_256]\t= 64,\n+\t\t[SHA_384]\t= 128,\n+\t\t[SHA_512]\t= 128,\n+\t\t[AES_XCBC]\t= 16,\n+};\n+\n+/**\n+ * Get the blocksize in bytes for a specified authentication algorithm\n+ *\n+ * @Note: this function will not return a valid value for a non-valid\n+ * authentication algorithm\n+ */\n+static inline unsigned\n+get_auth_algo_blocksize(JOB_HASH_ALG algo)\n+{\n+\treturn auth_blocksize[algo];\n+}\n+\n+static const unsigned auth_truncated_digest_byte_lengths[] = {\n+\t\t[MD5]\t\t= 12,\n+\t\t[SHA1]\t\t= 12,\n+\t\t[SHA_224]\t= 14,\n+\t\t[SHA_256]\t= 16,\n+\t\t[SHA_384]\t= 24,\n+\t\t[SHA_512]\t= 32,\n+\t\t[AES_XCBC]\t= 12,\n+};\n+\n+/**\n+ * Get the IPsec specified truncated length in bytes of the HMAC digest for a\n+ * specified authentication algorithm\n+ *\n+ * @Note: this function will not return a valid value for a non-valid\n+ * authentication algorithm\n+ */\n+static inline unsigned\n+get_truncated_digest_byte_length(JOB_HASH_ALG algo)\n+{\n+\treturn auth_truncated_digest_byte_lengths[algo];\n+}\n+\n+static const unsigned auth_digest_byte_lengths[] = {\n+\t\t[MD5]\t\t= 16,\n+\t\t[SHA1]\t\t= 20,\n+\t\t[SHA_224]\t= 28,\n+\t\t[SHA_256]\t= 32,\n+\t\t[SHA_384]\t= 48,\n+\t\t[SHA_512]\t= 64,\n+\t\t[AES_XCBC]\t= 16,\n+};\n+\n+/**\n+ * Get the output digest size in bytes for a specified authentication algorithm\n+ *\n+ * @Note: this function will not return a valid value for a non-valid\n+ * authentication algorithm\n+ */\n+static inline unsigned\n+get_digest_byte_length(JOB_HASH_ALG algo)\n+{\n+\treturn auth_digest_byte_lengths[algo];\n+}\n+\n+\n+/** private data structure for each virtual AESNI device */\n+struct aesni_mb_private {\n+\tenum aesni_mb_vector_mode vector_mode;\n+\n+\tunsigned max_nb_qpairs;\n+};\n+\n+struct aesni_mb_qp {\n+\tuint16_t id;\t\t\t\t/**< Queue Pair Identifier */\n+\tchar name[AESNI_MB_NAME_MAX_LENGTH];\t/**< Unique Queue Pair Name */\n+\tconst struct aesni_mb_ops *mb_ops;\t/**< Architecture dependent\n+\t\t\t\t\t\t * function pointer table of\n+\t\t\t\t\t\t * the multi-buffer APIs */\n+\tMB_MGR mb_mgr;\t\t\t\t/**< Multi-buffer instance */\n+\tstruct rte_ring *processed_pkts;\t/**< Ring for placing process packets */\n+\n+\tstruct rte_mempool *sess_mp;\t\t/**< Session Mempool */\n+\tstruct rte_cryptodev_stats qp_stats;\t/**< Queue pair statistics */\n+} __rte_cache_aligned;\n+\n+\n+/** AES-NI multi-buffer private session structure */\n+struct aesni_mb_session {\n+\tJOB_CHAIN_ORDER chain_order;\n+\n+\tunsigned gcm_session:1;\n+\n+\t/** Cipher Parameters */\n+\tstruct {\n+\t\t/** Cipher direction - encrypt / decrypt */\n+\t\tJOB_CIPHER_DIRECTION direction;\n+\t\t/** Cipher mode - CBC / Counter */\n+\t\tJOB_CIPHER_MODE mode;\n+\n+\t\tuint64_t key_length_in_bytes;\n+\n+\t\tstruct {\n+\t\t\tuint32_t encode[60] __rte_aligned(16);\n+\t\t\t/**< encode key */\n+\t\t\tuint32_t decode[60] __rte_aligned(16);\n+\t\t\t/**< decode key */\n+\t\t} expanded_aes_keys;\n+\t\t/**< Expanded AES keys - Allocating space to\n+\t\t * contain the maximum expanded key size which\n+\t\t * is 240 bytes for 256 bit AES, calculate by:\n+\t\t * ((key size (bytes)) *\n+\t\t * ((number of rounds) + 1)) */\n+\t} cipher;\n+\n+\tunion {\n+\t\t/** Authentication Parameters */\n+\t\tstruct {\n+\t\t\tJOB_HASH_ALG algo; /**< Authentication Algorithm */\n+\t\t\tunion {\n+\t\t\t\tstruct {\n+\t\t\t\t\tuint8_t inner[128] __rte_aligned(16);\n+\t\t\t\t\t/**< inner pad */\n+\t\t\t\t\tuint8_t outer[128] __rte_aligned(16);\n+\t\t\t\t\t/**< outer pad */\n+\t\t\t\t} pads;\n+\t\t\t\t/**< HMAC Authentication pads -\n+\t\t\t\t * allocating space for the maximum pad\n+\t\t\t\t * size supported which is 128 bytes for\n+\t\t\t\t * SHA512 */\n+\n+\t\t\t\tstruct {\n+\t\t\t\t    uint32_t k1_expanded[44] __rte_aligned(16);\n+\t\t\t\t    /**< k1 (expanded key). */\n+\t\t\t\t    uint8_t k2[16] __rte_aligned(16);\n+\t\t\t\t    /**< k2. */\n+\t\t\t\t    uint8_t k3[16] __rte_aligned(16);\n+\t\t\t\t    /**< k3. */\n+\t\t\t\t} xcbc;\n+\t\t\t\t/**< Expanded XCBC authentication keys */\n+\t\t\t};\n+\t\t} auth;\n+\n+\t\t/** GCM parameters */\n+\t\tstruct gcm_data gdata;\n+\t};\n+} __rte_cache_aligned;\n+\n+\n+/**\n+ *\n+ */\n+extern int\n+aesni_mb_set_session_parameters(const struct aesni_mb_ops *mb_ops,\n+\t\tstruct aesni_mb_session *sess,\n+\t\tconst struct rte_crypto_xform *xform);\n+\n+\n+/** device specific operations function pointer structure */\n+extern struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops;\n+\n+\n+\n+#endif /* _RTE_AESNI_MB_PMD_PRIVATE_H_ */\n+\ndiff --git a/drivers/crypto/aesni_mb/rte_pmd_aesni_version.map b/drivers/crypto/aesni_mb/rte_pmd_aesni_version.map\nnew file mode 100644\nindex 0000000..ad607bb\n--- /dev/null\n+++ b/drivers/crypto/aesni_mb/rte_pmd_aesni_version.map\n@@ -0,0 +1,3 @@\n+DPDK_2.2 {\n+\tlocal: *;\n+};\ndiff --git a/mk/rte.app.mk b/mk/rte.app.mk\nindex 5d960cd..6255d4e 100644\n--- a/mk/rte.app.mk\n+++ b/mk/rte.app.mk\n@@ -148,6 +148,10 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null\n # QAT PMD has a dependency on libcrypto (from openssl) for calculating HMAC precomputes\n _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat -lcrypto\n \n+# AESNI MULTI BUFFER is dependent on the IPSec_MB library\n+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -lrte_pmd_aesni_mb\n+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB\n+\n endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)\n \n endif # ! CONFIG_RTE_BUILD_COMBINE_LIBS\n",
    "prefixes": [
        "dpdk-dev",
        "v2",
        "4/6"
    ]
}