Patch Detail

HTTP 200 OK
Allow: GET, PUT, PATCH, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "id": 27867,
    "url": "https://patches.dpdk.org/api/patches/27867/?format=api",
    "web_url": "https://patches.dpdk.org/project/dpdk/patch/1503582789-121358-2-git-send-email-cristian.dumitrescu@intel.com/",
    "project": {
        "id": 1,
        "url": "https://patches.dpdk.org/api/projects/1/?format=api",
        "name": "DPDK",
        "link_name": "dpdk",
        "list_id": "dev.dpdk.org",
        "list_email": "dev@dpdk.org",
        "web_url": "http://core.dpdk.org",
        "scm_url": "git://dpdk.org/dpdk",
        "webscm_url": "http://git.dpdk.org/dpdk",
        "list_archive_url": "https://inbox.dpdk.org/dev",
        "list_archive_url_format": "https://inbox.dpdk.org/dev/{}",
        "commit_url_format": ""
    },
    "msgid": "<1503582789-121358-2-git-send-email-cristian.dumitrescu@intel.com>",
    "list_archive_url": "https://inbox.dpdk.org/dev/1503582789-121358-2-git-send-email-cristian.dumitrescu@intel.com",
    "date": "2017-08-24T13:53:05",
    "name": "[dpdk-dev,1/5] table: add key mask for hash tables",
    "commit_ref": null,
    "pull_url": null,
    "state": "superseded",
    "archived": true,
    "hash": "ff623790d02c3d08779a3e5ce4172c371b126037",
    "submitter": {
        "id": 19,
        "url": "https://patches.dpdk.org/api/people/19/?format=api",
        "name": "Cristian Dumitrescu",
        "email": "cristian.dumitrescu@intel.com"
    },
    "delegate": null,
    "mbox": "https://patches.dpdk.org/project/dpdk/patch/1503582789-121358-2-git-send-email-cristian.dumitrescu@intel.com/mbox/",
    "series": [],
    "comments": "https://patches.dpdk.org/api/patches/27867/comments/",
    "check": "warning",
    "checks": "https://patches.dpdk.org/api/patches/27867/checks/",
    "tags": {},
    "related": [],
    "headers": {
        "Return-Path": "<dev-bounces@dpdk.org>",
        "X-Original-To": "patchwork@dpdk.org",
        "Delivered-To": "patchwork@dpdk.org",
        "Received": [
            "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 1FC017CEC;\n\tThu, 24 Aug 2017 15:53:13 +0200 (CEST)",
            "from mga03.intel.com (mga03.intel.com [134.134.136.65])\n\tby dpdk.org (Postfix) with ESMTP id 923927CE1\n\tfor <dev@dpdk.org>; Thu, 24 Aug 2017 15:53:10 +0200 (CEST)",
            "from orsmga005.jf.intel.com ([10.7.209.41])\n\tby orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t24 Aug 2017 06:53:10 -0700",
            "from silpixa00382658.ir.intel.com ([10.237.223.29])\n\tby orsmga005.jf.intel.com with ESMTP; 24 Aug 2017 06:53:08 -0700"
        ],
        "X-ExtLoop1": "1",
        "X-IronPort-AV": "E=Sophos;i=\"5.41,421,1498546800\"; d=\"scan'208\";a=\"141567945\"",
        "From": "Cristian Dumitrescu <cristian.dumitrescu@intel.com>",
        "To": "dev@dpdk.org",
        "Date": "Thu, 24 Aug 2017 14:53:05 +0100",
        "Message-Id": "<1503582789-121358-2-git-send-email-cristian.dumitrescu@intel.com>",
        "X-Mailer": "git-send-email 2.7.4",
        "In-Reply-To": "<1503582789-121358-1-git-send-email-cristian.dumitrescu@intel.com>",
        "References": "<1503582789-121358-1-git-send-email-cristian.dumitrescu@intel.com>",
        "Subject": "[dpdk-dev] [PATCH 1/5] table: add key mask for hash tables",
        "X-BeenThere": "dev@dpdk.org",
        "X-Mailman-Version": "2.1.15",
        "Precedence": "list",
        "List-Id": "DPDK patches and discussions <dev.dpdk.org>",
        "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>",
        "List-Archive": "<http://dpdk.org/ml/archives/dev/>",
        "List-Post": "<mailto:dev@dpdk.org>",
        "List-Help": "<mailto:dev-request@dpdk.org?subject=help>",
        "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>",
        "Errors-To": "dev-bounces@dpdk.org",
        "Sender": "\"dev\" <dev-bounces@dpdk.org>"
    },
    "content": "Main changes:\n\n1. The key_mask parameter is added to all the hash tables that were\n   previously missing it, as well to the hash compute function. This was\n   first started in DPDK 2.0, but was only implemented for a couple of\n   hash tables. The benefit of this approach is that it allows for better\n   performance for large keys (bigger than 16 bytes), while it preserves\n   the same performance for small keys [Q&A1].\n\n2. The precomputed key signature (i.e. non-\"do-sig\") versions have been\n   removed for all the hash tables, so now the key signature is always\n   computed on every lookup. Note that this approach also allows for the\n   precomputed key signature scheme [Q&A2].\n\n3. API cleanup: single parameter structure common for all hash tables.\n\nQ&A:\n\nQ1: How is better lookup performance achieved by using key mask approach\n   for hash tables?\nA1: This approach eliminates the need to consolidate the lookup key in a\n   single contiguous buffer where the relevant packet fields are written\n   one by one, which is a very costly operation that also hash strong data\n   dependencies.\n\nQ2: How can the pre-computed key signature scheme be implemented with\n    current approach?\nA2: The application can implement a straightforward custom hash function\n    that simply reads the pre-computed key signature from a given offset\n    in the input key buffer where it has been stored prior to the lookup\n    operation.\n\nSigned-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>\n---\n lib/librte_table/rte_table_hash.h        | 302 +-------------\n lib/librte_table/rte_table_hash_cuckoo.c | 200 ++++-----\n lib/librte_table/rte_table_hash_ext.c    | 407 ++++++------------\n lib/librte_table/rte_table_hash_key16.c  | 691 ++++++++-----------------------\n lib/librte_table/rte_table_hash_key32.c  | 381 +++++++++--------\n lib/librte_table/rte_table_hash_key8.c   | 654 +++++++----------------------\n lib/librte_table/rte_table_hash_lru.c    | 506 +++++++++-------------\n 7 files changed, 981 insertions(+), 2160 deletions(-)",
    "diff": "diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h\nindex 57505a6..978ae98 100644\n--- a/lib/librte_table/rte_table_hash.h\n+++ b/lib/librte_table/rte_table_hash.h\n@@ -1,7 +1,7 @@\n /*-\n  *   BSD LICENSE\n  *\n- *   Copyright(c) 2010-2016 Intel Corporation. All rights reserved.\n+ *   Copyright(c) 2010-2017 Intel Corporation. All rights reserved.\n  *   All rights reserved.\n  *\n  *   Redistribution and use in source and binary forms, with or without\n@@ -74,19 +74,7 @@ extern \"C\" {\n  *        4 keys, potentially until all keys in this bucket are examined. The\n  *        extendible bucket logic requires maintaining specific data structures\n  *        per table and per each bucket.\n- * 2. Key signature computation:\n- *     a. Pre-computed key signature: The key lookup operation is split between\n- *        two CPU cores. The first CPU core (typically the CPU core performing\n- *        packet RX) extracts the key from the input packet, computes the key\n- *        signature and saves both the key and the key signature in the packet\n- *        buffer as packet meta-data. The second CPU core reads both the key and\n- *        the key signature from the packet meta-data and performs the bucket\n- *        search step of the key lookup operation.\n- *     b. Key signature computed on lookup (do-sig): The same CPU core reads\n- *        the key from the packet meta-data, uses it to compute the key\n- *        signature and also performs the bucket search step of the key lookup\n- *        operation.\n- * 3. Key size:\n+ * 2. Key size:\n  *     a. Configurable key size\n  *     b. Single key size (8-byte, 16-byte or 32-byte key size)\n  *\n@@ -98,59 +86,28 @@ extern \"C\" {\n /** Hash function */\n typedef uint64_t (*rte_table_hash_op_hash)(\n \tvoid *key,\n+\tvoid *key_mask,\n \tuint32_t key_size,\n \tuint64_t seed);\n \n-/**\n- * Hash tables with configurable key size\n- *\n- */\n-/** Extendible bucket hash table parameters */\n-struct rte_table_hash_ext_params {\n+/** Hash table parameters */\n+struct rte_table_hash_params {\n+\t/** Name */\n+\tconst char *name;\n+\n \t/** Key size (number of bytes) */\n \tuint32_t key_size;\n \n-\t/** Maximum number of keys */\n-\tuint32_t n_keys;\n-\n-\t/** Number of hash table buckets. Each bucket stores up to 4 keys. */\n-\tuint32_t n_buckets;\n-\n-\t/** Number of hash table bucket extensions. Each bucket extension has\n-\tspace for 4 keys and each bucket can have 0, 1 or more extensions. */\n-\tuint32_t n_buckets_ext;\n-\n-\t/** Hash function */\n-\trte_table_hash_op_hash f_hash;\n-\n-\t/** Seed value for the hash function */\n-\tuint64_t seed;\n-\n-\t/** Byte offset within packet meta-data where the 4-byte key signature\n-\tis located. Valid for pre-computed key signature tables, ignored for\n-\tdo-sig tables. */\n-\tuint32_t signature_offset;\n-\n \t/** Byte offset within packet meta-data where the key is located */\n \tuint32_t key_offset;\n-};\n \n-/** Extendible bucket hash table operations for pre-computed key signature */\n-extern struct rte_table_ops rte_table_hash_ext_ops;\n-\n-/** Extendible bucket hash table operations for key signature computed on\n-\tlookup (\"do-sig\") */\n-extern struct rte_table_ops rte_table_hash_ext_dosig_ops;\n-\n-/** LRU hash table parameters */\n-struct rte_table_hash_lru_params {\n-\t/** Key size (number of bytes) */\n-\tuint32_t key_size;\n+\t/** Key mask */\n+\tuint8_t *key_mask;\n \n-\t/** Maximum number of keys */\n+\t/** Number of keys */\n \tuint32_t n_keys;\n \n-\t/** Number of hash table buckets. Each bucket stores up to 4 keys. */\n+\t/** Number of buckets */\n \tuint32_t n_buckets;\n \n \t/** Hash function */\n@@ -158,239 +115,22 @@ struct rte_table_hash_lru_params {\n \n \t/** Seed value for the hash function */\n \tuint64_t seed;\n-\n-\t/** Byte offset within packet meta-data where the 4-byte key signature\n-\tis located. Valid for pre-computed key signature tables, ignored for\n-\tdo-sig tables. */\n-\tuint32_t signature_offset;\n-\n-\t/** Byte offset within packet meta-data where the key is located */\n-\tuint32_t key_offset;\n-};\n-\n-/** LRU hash table operations for pre-computed key signature */\n-extern struct rte_table_ops rte_table_hash_lru_ops;\n-\n-/** LRU hash table operations for key signature computed on lookup (\"do-sig\") */\n-extern struct rte_table_ops rte_table_hash_lru_dosig_ops;\n-\n-/**\n- * 8-byte key hash tables\n- *\n- */\n-/** LRU hash table parameters */\n-struct rte_table_hash_key8_lru_params {\n-\t/** Maximum number of entries (and keys) in the table */\n-\tuint32_t n_entries;\n-\n-\t/** Hash function */\n-\trte_table_hash_op_hash f_hash;\n-\n-\t/** Seed for the hash function */\n-\tuint64_t seed;\n-\n-\t/** Byte offset within packet meta-data where the 4-byte key signature\n-\tis located. Valid for pre-computed key signature tables, ignored for\n-\tdo-sig tables. */\n-\tuint32_t signature_offset;\n-\n-\t/** Byte offset within packet meta-data where the key is located */\n-\tuint32_t key_offset;\n-\n-\t/** Bit-mask to be AND-ed to the key on lookup */\n-\tuint8_t *key_mask;\n };\n \n-/** LRU hash table operations for pre-computed key signature */\n-extern struct rte_table_ops rte_table_hash_key8_lru_ops;\n-\n-/** LRU hash table operations for key signature computed on lookup (\"do-sig\") */\n-extern struct rte_table_ops rte_table_hash_key8_lru_dosig_ops;\n-\n-/** Extendible bucket hash table parameters */\n-struct rte_table_hash_key8_ext_params {\n-\t/** Maximum number of entries (and keys) in the table */\n-\tuint32_t n_entries;\n-\n-\t/** Number of entries (and keys) for hash table bucket extensions. Each\n-\t\tbucket is extended in increments of 4 keys. */\n-\tuint32_t n_entries_ext;\n-\n-\t/** Hash function */\n-\trte_table_hash_op_hash f_hash;\n-\n-\t/** Seed for the hash function */\n-\tuint64_t seed;\n-\n-\t/** Byte offset within packet meta-data where the 4-byte key signature\n-\tis located. Valid for pre-computed key signature tables, ignored for\n-\tdo-sig tables. */\n-\tuint32_t signature_offset;\n-\n-\t/** Byte offset within packet meta-data where the key is located */\n-\tuint32_t key_offset;\n-\n-\t/** Bit-mask to be AND-ed to the key on lookup */\n-\tuint8_t *key_mask;\n-};\n-\n-/** Extendible bucket hash table operations for pre-computed key signature */\n+/** Extendible bucket hash table operations */\n+extern struct rte_table_ops rte_table_hash_ext_ops;\n extern struct rte_table_ops rte_table_hash_key8_ext_ops;\n-\n-/** Extendible bucket hash table operations for key signature computed on\n-    lookup (\"do-sig\") */\n-extern struct rte_table_ops rte_table_hash_key8_ext_dosig_ops;\n-\n-/**\n- * 16-byte key hash tables\n- *\n- */\n-/** LRU hash table parameters */\n-struct rte_table_hash_key16_lru_params {\n-\t/** Maximum number of entries (and keys) in the table */\n-\tuint32_t n_entries;\n-\n-\t/** Hash function */\n-\trte_table_hash_op_hash f_hash;\n-\n-\t/** Seed for the hash function */\n-\tuint64_t seed;\n-\n-\t/** Byte offset within packet meta-data where the 4-byte key signature\n-\tis located. Valid for pre-computed key signature tables, ignored for\n-\tdo-sig tables. */\n-\tuint32_t signature_offset;\n-\n-\t/** Byte offset within packet meta-data where the key is located */\n-\tuint32_t key_offset;\n-\n-\t/** Bit-mask to be AND-ed to the key on lookup */\n-\tuint8_t *key_mask;\n-};\n-\n-/** LRU hash table operations for pre-computed key signature */\n-extern struct rte_table_ops rte_table_hash_key16_lru_ops;\n-\n-/** LRU hash table operations for key signature computed on lookup\n-    (\"do-sig\") */\n-extern struct rte_table_ops rte_table_hash_key16_lru_dosig_ops;\n-\n-/** Extendible bucket hash table parameters */\n-struct rte_table_hash_key16_ext_params {\n-\t/** Maximum number of entries (and keys) in the table */\n-\tuint32_t n_entries;\n-\n-\t/** Number of entries (and keys) for hash table bucket extensions. Each\n-\tbucket is extended in increments of 4 keys. */\n-\tuint32_t n_entries_ext;\n-\n-\t/** Hash function */\n-\trte_table_hash_op_hash f_hash;\n-\n-\t/** Seed for the hash function */\n-\tuint64_t seed;\n-\n-\t/** Byte offset within packet meta-data where the 4-byte key signature\n-\tis located. Valid for pre-computed key signature tables, ignored for\n-\tdo-sig tables. */\n-\tuint32_t signature_offset;\n-\n-\t/** Byte offset within packet meta-data where the key is located */\n-\tuint32_t key_offset;\n-\n-\t/** Bit-mask to be AND-ed to the key on lookup */\n-\tuint8_t *key_mask;\n-};\n-\n-/** Extendible bucket operations for pre-computed key signature */\n extern struct rte_table_ops rte_table_hash_key16_ext_ops;\n-\n-/** Extendible bucket hash table operations for key signature computed on\n-    lookup (\"do-sig\") */\n-extern struct rte_table_ops rte_table_hash_key16_ext_dosig_ops;\n-\n-/**\n- * 32-byte key hash tables\n- *\n- */\n-/** LRU hash table parameters */\n-struct rte_table_hash_key32_lru_params {\n-\t/** Maximum number of entries (and keys) in the table */\n-\tuint32_t n_entries;\n-\n-\t/** Hash function */\n-\trte_table_hash_op_hash f_hash;\n-\n-\t/** Seed for the hash function */\n-\tuint64_t seed;\n-\n-\t/** Byte offset within packet meta-data where the 4-byte key signature\n-\tis located. Valid for pre-computed key signature tables, ignored for\n-\tdo-sig tables. */\n-\tuint32_t signature_offset;\n-\n-\t/** Byte offset within packet meta-data where the key is located */\n-\tuint32_t key_offset;\n-};\n-\n-/** LRU hash table operations for pre-computed key signature */\n-extern struct rte_table_ops rte_table_hash_key32_lru_ops;\n-\n-/** Extendible bucket hash table parameters */\n-struct rte_table_hash_key32_ext_params {\n-\t/** Maximum number of entries (and keys) in the table */\n-\tuint32_t n_entries;\n-\n-\t/** Number of entries (and keys) for hash table bucket extensions. Each\n-\t\tbucket is extended in increments of 4 keys. */\n-\tuint32_t n_entries_ext;\n-\n-\t/** Hash function */\n-\trte_table_hash_op_hash f_hash;\n-\n-\t/** Seed for the hash function */\n-\tuint64_t seed;\n-\n-\t/** Byte offset within packet meta-data where the 4-byte key signature\n-\tis located. Valid for pre-computed key signature tables, ignored for\n-\tdo-sig tables. */\n-\tuint32_t signature_offset;\n-\n-\t/** Byte offset within packet meta-data where the key is located */\n-\tuint32_t key_offset;\n-};\n-\n-/** Extendible bucket hash table operations */\n extern struct rte_table_ops rte_table_hash_key32_ext_ops;\n \n-/** Cuckoo hash table parameters */\n-struct rte_table_hash_cuckoo_params {\n-    /** Key size (number of bytes */\n-\t\tuint32_t key_size;\n-\n-\t/** Maximum number of hash table entries */\n-\tuint32_t n_keys;\n-\n-\t/** Hash function used to calculate hash */\n-\trte_table_hash_op_hash f_hash;\n-\n-\t/** Seed value or Init value used by f_hash */\n-\tuint32_t seed;\n-\n-\t/** Byte offset within packet meta-data where the 4-byte key signature\n-\tis located. Valid for pre-computed key signature tables, ignored for\n-\tdo-sig tables. */\n-\tuint32_t signature_offset;\n-\n-\t/** Byte offset within packet meta-data where the key is located */\n-\tuint32_t key_offset;\n-\n-\t/** Hash table name */\n-\tconst char *name;\n-};\n+/** LRU hash table operations */\n+extern struct rte_table_ops rte_table_hash_lru_ops;\n+extern struct rte_table_ops rte_table_hash_key8_lru_ops;\n+extern struct rte_table_ops rte_table_hash_key16_lru_ops;\n+extern struct rte_table_ops rte_table_hash_key32_lru_ops;\n \n-/** Cuckoo hash table operations */\n-extern struct rte_table_ops rte_table_hash_cuckoo_dosig_ops;\n+/* Cuckoo hash table operations */\n+extern struct rte_table_ops rte_table_hash_cuckoo_ops;\n \n #ifdef __cplusplus\n }\ndiff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c\nindex da1597f..b3e8eb9 100644\n--- a/lib/librte_table/rte_table_hash_cuckoo.c\n+++ b/lib/librte_table/rte_table_hash_cuckoo.c\n@@ -1,34 +1,34 @@\n /*-\n- *\t BSD LICENSE\n+ *   BSD LICENSE\n  *\n- *\t Copyright(c) 2016 Intel Corporation. All rights reserved.\n- *\t All rights reserved.\n+ *   Copyright(c) 2010-2017 Intel Corporation. All rights reserved.\n+ *   All rights reserved.\n  *\n- *\t Redistribution and use in source and binary forms, with or without\n- *\t modification, are permitted provided that the following conditions\n- *\t are met:\n+ *   Redistribution and use in source and binary forms, with or without\n+ *   modification, are permitted provided that the following conditions\n+ *   are met:\n  *\n- *\t* Redistributions of source code must retain the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer.\n- *\t* Redistributions in binary form must reproduce the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer in\n- *\t\t the documentation and/or other materials provided with the\n- *\t\t distribution.\n- *\t* Neither the name of Intel Corporation nor the names of its\n- *\t\t contributors may be used to endorse or promote products derived\n- *\t\t from this software without specific prior written permission.\n+ *     * Redistributions of source code must retain the above copyright\n+ *       notice, this list of conditions and the following disclaimer.\n+ *     * Redistributions in binary form must reproduce the above copyright\n+ *       notice, this list of conditions and the following disclaimer in\n+ *       the documentation and/or other materials provided with the\n+ *       distribution.\n+ *     * Neither the name of Intel Corporation nor the names of its\n+ *       contributors may be used to endorse or promote products derived\n+ *       from this software without specific prior written permission.\n  *\n- *\t THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n- *\t \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n- *\t LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n- *\t A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n- *\t OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n- *\t SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n- *\t LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n- *\t DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n- *\t THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n- *\t (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n- *\t OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ *   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n  */\n #include <string.h>\n #include <stdio.h>\n@@ -66,25 +66,28 @@ struct rte_table_hash {\n \tuint32_t n_keys;\n \trte_table_hash_op_hash f_hash;\n \tuint32_t seed;\n-\tuint32_t signature_offset;\n \tuint32_t key_offset;\n-\tconst char *name;\n \n \t/* cuckoo hash table object */\n \tstruct rte_hash *h_table;\n \n \t/* Lookup table */\n-\tuint8_t memory[0] __rte_cache_aligned; };\n+\tuint8_t memory[0] __rte_cache_aligned;\n+};\n \n static int\n-check_params_create_hash_cuckoo(const struct\n-rte_table_hash_cuckoo_params *params) {\n-\t/* Check for valid parameters */\n+check_params_create_hash_cuckoo(struct rte_table_hash_params *params)\n+{\n \tif (params == NULL) {\n \t\tRTE_LOG(ERR, TABLE, \"NULL Input Parameters.\\n\");\n \t\treturn -EINVAL;\n \t}\n \n+\tif (params->name == NULL) {\n+\t\tRTE_LOG(ERR, TABLE, \"Table name is NULL.\\n\");\n+\t\treturn -EINVAL;\n+\t}\n+\n \tif (params->key_size == 0) {\n \t\tRTE_LOG(ERR, TABLE, \"Invalid key_size.\\n\");\n \t\treturn -EINVAL;\n@@ -100,11 +103,6 @@ rte_table_hash_cuckoo_params *params) {\n \t\treturn -EINVAL;\n \t}\n \n-\tif (params->name == NULL) {\n-\t\tRTE_LOG(ERR, TABLE, \"Table name is NULL.\\n\");\n-\t\treturn -EINVAL;\n-\t}\n-\n \treturn 0;\n }\n \n@@ -113,34 +111,25 @@ rte_table_hash_cuckoo_create(void *params,\n \t\t\tint socket_id,\n \t\t\tuint32_t entry_size)\n {\n-\tstruct rte_hash *rte_hash_handle;\n+\tstruct rte_table_hash_params *p = params;\n+\tstruct rte_hash *h_table;\n \tstruct rte_table_hash *t;\n \tuint32_t total_size, total_cl_size;\n \n \t/* Check input parameters */\n-\tstruct rte_table_hash_cuckoo_params *p =\n-\t\t(struct rte_table_hash_cuckoo_params *) params;\n-\n \tif (check_params_create_hash_cuckoo(params))\n \t\treturn NULL;\n \n \t/* Memory allocation */\n-\ttotal_cl_size =\n-\t\t(sizeof(struct rte_table_hash) +\n-\t\t RTE_CACHE_LINE_SIZE) / RTE_CACHE_LINE_SIZE;\n-\ttotal_cl_size += (p->n_keys * entry_size +\n-\t\t\tRTE_CACHE_LINE_SIZE) / RTE_CACHE_LINE_SIZE;\n+\ttotal_cl_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));\n+\ttotal_cl_size += RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);\n \ttotal_size = total_cl_size * RTE_CACHE_LINE_SIZE;\n \n-\tt = rte_zmalloc_socket(\"TABLE\",\n-\t\t\ttotal_size,\n-\t\t\tRTE_CACHE_LINE_SIZE,\n-\t\t\tsocket_id);\n+\tt = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);\n \tif (t == NULL) {\n \t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\"%s: Cannot allocate %u bytes for Cuckoo hash table\\n\",\n-\t\t\t__func__,\n-\t\t\t(uint32_t)sizeof(struct rte_table_hash));\n+\t\t\t\"%s: Cannot allocate %u bytes for cuckoo hash table %s\\n\",\n+\t\t\t__func__, total_size, p->name);\n \t\treturn NULL;\n \t}\n \n@@ -154,13 +143,13 @@ rte_table_hash_cuckoo_create(void *params,\n \t\t.name = p->name\n \t};\n \n-\trte_hash_handle = rte_hash_find_existing(p->name);\n-\tif (rte_hash_handle == NULL) {\n-\t\trte_hash_handle = rte_hash_create(&hash_cuckoo_params);\n-\t\tif (NULL == rte_hash_handle) {\n+\th_table = rte_hash_find_existing(p->name);\n+\tif (h_table == NULL) {\n+\t\th_table = rte_hash_create(&hash_cuckoo_params);\n+\t\tif (h_table == NULL) {\n \t\t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\t\"%s: failed to create cuckoo hash table. keysize: %u\",\n-\t\t\t\t__func__, hash_cuckoo_params.key_len);\n+\t\t\t\t\"%s: failed to create cuckoo hash table %s\\n\",\n+\t\t\t\t__func__, p->name);\n \t\t\trte_free(t);\n \t\t\treturn NULL;\n \t\t}\n@@ -172,26 +161,22 @@ rte_table_hash_cuckoo_create(void *params,\n \tt->n_keys = p->n_keys;\n \tt->f_hash = p->f_hash;\n \tt->seed = p->seed;\n-\tt->signature_offset = p->signature_offset;\n \tt->key_offset = p->key_offset;\n-\tt->name = p->name;\n-\tt->h_table = rte_hash_handle;\n+\tt->h_table = h_table;\n \n \tRTE_LOG(INFO, TABLE,\n-\t\t\"%s: Cuckoo Hash table memory footprint is %u bytes\\n\",\n-\t\t__func__, total_size);\n+\t\t\"%s: Cuckoo hash table %s memory footprint is %u bytes\\n\",\n+\t\t__func__, p->name, total_size);\n \treturn t;\n }\n \n static int\n rte_table_hash_cuckoo_free(void *table) {\n-\tif (table == NULL) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: table parameter is NULL\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n \tstruct rte_table_hash *t = table;\n \n+\tif (table == NULL)\n+\t\treturn -EINVAL;\n+\n \trte_hash_free(t->h_table);\n \trte_free(t);\n \n@@ -200,25 +185,18 @@ rte_table_hash_cuckoo_free(void *table) {\n \n static int\n rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,\n-\t\tint *key_found, void **entry_ptr) {\n+\tint *key_found, void **entry_ptr)\n+{\n+\tstruct rte_table_hash *t = table;\n \tint pos = 0;\n \n-\tif (table == NULL) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: table parameter is NULL\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\tif (key == NULL) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: key parameter is NULL\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\tif (entry == NULL) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: entry parameter is NULL\\n\", __func__);\n+\t/* Check input parameters */\n+\tif ((table == NULL) ||\n+\t\t(key == NULL) ||\n+\t\t(entry == NULL) ||\n+\t\t(key_found == NULL) ||\n+\t\t(entry_ptr == NULL))\n \t\treturn -EINVAL;\n-\t}\n-\n-\tstruct rte_table_hash *t = table;\n \n \t/*  Find Existing entries */\n \tpos = rte_hash_lookup(t->h_table, key);\n@@ -231,17 +209,15 @@ rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,\n \t\t*entry_ptr = existing_entry;\n \n \t\treturn 0;\n-} else if (pos == -ENOENT) {\n-\t/* Entry not found. Adding new entry */\n+\t}\n+\n+\tif (pos == -ENOENT) {\n+\t\t/* Entry not found. Adding new entry */\n \t\tuint8_t *new_entry;\n \n \t\tpos = rte_hash_add_key(t->h_table, key);\n-\t\tif (pos < 0) {\n-\t\t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\t\"%s: Entry not added, status : %u\\n\",\n-\t\t\t\t__func__, pos);\n+\t\tif (pos < 0)\n \t\t\treturn pos;\n-\t\t}\n \n \t\tnew_entry = &t->memory[pos * t->entry_size];\n \t\tmemcpy(new_entry, entry, t->entry_size);\n@@ -250,25 +226,22 @@ rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,\n \t\t*entry_ptr = new_entry;\n \t\treturn 0;\n \t}\n+\n \treturn pos;\n }\n \n static int\n rte_table_hash_cuckoo_entry_delete(void *table, void *key,\n-\t\tint *key_found, __rte_unused void *entry) {\n+\tint *key_found, void *entry)\n+{\n+\tstruct rte_table_hash *t = table;\n \tint pos = 0;\n \n-\tif (table == NULL) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: table parameter is NULL\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\tif (key == NULL) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: key parameter is NULL\\n\", __func__);\n+\t/* Check input parameters */\n+\tif ((table == NULL) ||\n+\t\t(key == NULL) ||\n+\t\t(key_found == NULL))\n \t\treturn -EINVAL;\n-\t}\n-\n-\tstruct rte_table_hash *t = table;\n \n \tpos = rte_hash_del_key(t->h_table, key);\n \tif (pos >= 0) {\n@@ -279,20 +252,21 @@ rte_table_hash_cuckoo_entry_delete(void *table, void *key,\n \t\t\tmemcpy(entry, entry_ptr, t->entry_size);\n \n \t\tmemset(&t->memory[pos * t->entry_size], 0, t->entry_size);\n+\t\treturn 0;\n \t}\n \n+\t*key_found = 0;\n \treturn pos;\n }\n \n-\n static int\n-rte_table_hash_cuckoo_lookup_dosig(void *table,\n+rte_table_hash_cuckoo_lookup(void *table,\n \tstruct rte_mbuf **pkts,\n \tuint64_t pkts_mask,\n \tuint64_t *lookup_hit_mask,\n \tvoid **entries)\n {\n-\tstruct rte_table_hash *t = (struct rte_table_hash *)table;\n+\tstruct rte_table_hash *t = table;\n \tuint64_t pkts_mask_out = 0;\n \tuint32_t i;\n \n@@ -301,20 +275,19 @@ rte_table_hash_cuckoo_lookup_dosig(void *table,\n \tRTE_TABLE_HASH_CUCKOO_STATS_PKTS_IN_ADD(t, n_pkts_in);\n \n \tif ((pkts_mask & (pkts_mask + 1)) == 0) {\n-\t\tconst uint8_t *keys[64];\n-\t\tint32_t positions[64], status;\n+\t\tconst uint8_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];\n+\t\tint32_t positions[RTE_PORT_IN_BURST_SIZE_MAX], status;\n \n \t\t/* Keys for bulk lookup */\n \t\tfor (i = 0; i < n_pkts_in; i++)\n \t\t\tkeys[i] = RTE_MBUF_METADATA_UINT8_PTR(pkts[i],\n-\t\t\t\t\tt->key_offset);\n+\t\t\t\tt->key_offset);\n \n \t\t/* Bulk Lookup */\n \t\tstatus = rte_hash_lookup_bulk(t->h_table,\n \t\t\t\t(const void **) keys,\n \t\t\t\tn_pkts_in,\n \t\t\t\tpositions);\n-\n \t\tif (status == 0) {\n \t\t\tfor (i = 0; i < n_pkts_in; i++) {\n \t\t\t\tif (likely(positions[i] >= 0)) {\n@@ -326,7 +299,7 @@ rte_table_hash_cuckoo_lookup_dosig(void *table,\n \t\t\t\t}\n \t\t\t}\n \t\t}\n-\t} else {\n+\t} else\n \t\tfor (i = 0; i < (uint32_t)(RTE_PORT_IN_BURST_SIZE_MAX\n \t\t\t\t\t- __builtin_clzll(pkts_mask)); i++) {\n \t\t\tuint64_t pkt_mask = 1LLU << i;\n@@ -345,7 +318,6 @@ rte_table_hash_cuckoo_lookup_dosig(void *table,\n \t\t\t\t}\n \t\t\t}\n \t\t}\n-\t}\n \n \t*lookup_hit_mask = pkts_mask_out;\n \tRTE_TABLE_HASH_CUCKOO_STATS_PKTS_LOOKUP_MISS(t,\n@@ -370,13 +342,13 @@ rte_table_hash_cuckoo_stats_read(void *table, struct rte_table_stats *stats,\n \treturn 0;\n }\n \n-struct rte_table_ops rte_table_hash_cuckoo_dosig_ops = {\n+struct rte_table_ops rte_table_hash_cuckoo_ops = {\n \t.f_create = rte_table_hash_cuckoo_create,\n \t.f_free = rte_table_hash_cuckoo_free,\n \t.f_add = rte_table_hash_cuckoo_entry_add,\n \t.f_delete = rte_table_hash_cuckoo_entry_delete,\n \t.f_add_bulk = NULL,\n \t.f_delete_bulk = NULL,\n-\t.f_lookup = rte_table_hash_cuckoo_lookup_dosig,\n+\t.f_lookup = rte_table_hash_cuckoo_lookup,\n \t.f_stats = rte_table_hash_cuckoo_stats_read,\n };\ndiff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c\nindex e718102..45e3ce6 100644\n--- a/lib/librte_table/rte_table_hash_ext.c\n+++ b/lib/librte_table/rte_table_hash_ext.c\n@@ -1,34 +1,34 @@\n /*-\n- *\t BSD LICENSE\n+ *   BSD LICENSE\n  *\n- *\t Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n- *\t All rights reserved.\n+ *   Copyright(c) 2010-2017 Intel Corporation. All rights reserved.\n+ *   All rights reserved.\n  *\n- *\t Redistribution and use in source and binary forms, with or without\n- *\t modification, are permitted provided that the following conditions\n- *\t are met:\n+ *   Redistribution and use in source and binary forms, with or without\n+ *   modification, are permitted provided that the following conditions\n+ *   are met:\n  *\n- *\t* Redistributions of source code must retain the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer.\n- *\t* Redistributions in binary form must reproduce the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer in\n- *\t\t the documentation and/or other materials provided with the\n- *\t\t distribution.\n- *\t* Neither the name of Intel Corporation nor the names of its\n- *\t\t contributors may be used to endorse or promote products derived\n- *\t\t from this software without specific prior written permission.\n+ *     * Redistributions of source code must retain the above copyright\n+ *       notice, this list of conditions and the following disclaimer.\n+ *     * Redistributions in binary form must reproduce the above copyright\n+ *       notice, this list of conditions and the following disclaimer in\n+ *       the documentation and/or other materials provided with the\n+ *       distribution.\n+ *     * Neither the name of Intel Corporation nor the names of its\n+ *       contributors may be used to endorse or promote products derived\n+ *       from this software without specific prior written permission.\n  *\n- *\t THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n- *\t \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n- *\t LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n- *\t A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n- *\t OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n- *\t SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n- *\t LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n- *\t DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n- *\t THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n- *\t (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n- *\t OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ *   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n  */\n \n #include <string.h>\n@@ -106,7 +106,6 @@ struct rte_table_hash {\n \tuint32_t n_buckets_ext;\n \trte_table_hash_op_hash f_hash;\n \tuint64_t seed;\n-\tuint32_t signature_offset;\n \tuint32_t key_offset;\n \n \t/* Internal */\n@@ -120,6 +119,7 @@ struct rte_table_hash {\n \tstruct grinder grinders[RTE_PORT_IN_BURST_SIZE_MAX];\n \n \t/* Tables */\n+\tuint64_t *key_mask;\n \tstruct bucket *buckets;\n \tstruct bucket *buckets_ext;\n \tuint8_t *key_mem;\n@@ -132,12 +132,39 @@ struct rte_table_hash {\n };\n \n static int\n-check_params_create(struct rte_table_hash_ext_params *params)\n+keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes)\n {\n-\tuint32_t n_buckets_min;\n+\tuint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;\n+\tuint32_t i;\n+\n+\tfor (i = 0; i < n_bytes / sizeof(uint64_t); i++)\n+\t\tif (a64[i] != (b64[i] & b_mask64[i]))\n+\t\t\treturn 1;\n+\n+\treturn 0;\n+}\n+\n+static void\n+keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes)\n+{\n+\tuint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;\n+\tuint32_t i;\n+\n+\tfor (i = 0; i < n_bytes / sizeof(uint64_t); i++)\n+\t\tdst64[i] = src64[i] & src_mask64[i];\n+}\n+\n+static int\n+check_params_create(struct rte_table_hash_params *params)\n+{\n+\t/* name */\n+\tif (params->name == NULL) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: name invalid value\\n\", __func__);\n+\t\treturn -EINVAL;\n+\t}\n \n \t/* key_size */\n-\tif ((params->key_size == 0) ||\n+\tif ((params->key_size < sizeof(uint64_t)) ||\n \t\t(!rte_is_power_of_2(params->key_size))) {\n \t\tRTE_LOG(ERR, TABLE, \"%s: key_size invalid value\\n\", __func__);\n \t\treturn -EINVAL;\n@@ -151,10 +178,8 @@ check_params_create(struct rte_table_hash_ext_params *params)\n \t}\n \n \t/* n_buckets */\n-\tn_buckets_min = (params->n_keys + KEYS_PER_BUCKET - 1) / params->n_keys;\n \tif ((params->n_buckets == 0) ||\n-\t\t(!rte_is_power_of_2(params->n_keys)) ||\n-\t\t(params->n_buckets < n_buckets_min)) {\n+\t\t(!rte_is_power_of_2(params->n_buckets))) {\n \t\tRTE_LOG(ERR, TABLE, \"%s: n_buckets invalid value\\n\", __func__);\n \t\treturn -EINVAL;\n \t}\n@@ -171,15 +196,14 @@ check_params_create(struct rte_table_hash_ext_params *params)\n static void *\n rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)\n {\n-\tstruct rte_table_hash_ext_params *p =\n-\t\tparams;\n+\tstruct rte_table_hash_params *p = params;\n \tstruct rte_table_hash *t;\n \tuint32_t total_size, table_meta_sz;\n-\tuint32_t bucket_sz, bucket_ext_sz, key_sz;\n+\tuint32_t key_mask_sz, bucket_sz, bucket_ext_sz, key_sz;\n \tuint32_t key_stack_sz, bkt_ext_stack_sz, data_sz;\n-\tuint32_t bucket_offset, bucket_ext_offset, key_offset;\n+\tuint32_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset;\n \tuint32_t key_stack_offset, bkt_ext_stack_offset, data_offset;\n-\tuint32_t i;\n+\tuint32_t n_keys, n_buckets_ext, i;\n \n \t/* Check input parameters */\n \tif ((check_params_create(p) != 0) ||\n@@ -188,38 +212,58 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)\n \t\t(sizeof(struct bucket) != (RTE_CACHE_LINE_SIZE / 2)))\n \t\treturn NULL;\n \n+\t/*\n+\t * Table dimensioning\n+\t *\n+\t * Objective: Pick the number of bucket extensions (n_buckets_ext) so that\n+\t * it is guaranteed that n_keys keys can be stored in the table at any time.\n+\t *\n+\t * The worst case scenario takes place when all the n_keys keys fall into\n+\t * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst\n+\t * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the\n+\t * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall\n+\t * into a different bucket. This case defeats the purpose of the hash table.\n+\t * It indicates unsuitable f_hash or n_keys to n_buckets ratio.\n+\t *\n+\t * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1\n+\t */\n+\tn_keys = p->n_keys;\n+\tif (n_keys < KEYS_PER_BUCKET)\n+\t\tn_keys = KEYS_PER_BUCKET;\n+\tn_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;\n+\n \t/* Memory allocation */\n \ttable_meta_sz = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));\n+\tkey_mask_sz = RTE_CACHE_LINE_ROUNDUP(p->key_size);\n \tbucket_sz = RTE_CACHE_LINE_ROUNDUP(p->n_buckets * sizeof(struct bucket));\n \tbucket_ext_sz =\n-\t\tRTE_CACHE_LINE_ROUNDUP(p->n_buckets_ext * sizeof(struct bucket));\n-\tkey_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * p->key_size);\n-\tkey_stack_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * sizeof(uint32_t));\n+\t\tRTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(struct bucket));\n+\tkey_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * p->key_size);\n+\tkey_stack_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * sizeof(uint32_t));\n \tbkt_ext_stack_sz =\n-\t\tRTE_CACHE_LINE_ROUNDUP(p->n_buckets_ext * sizeof(uint32_t));\n-\tdata_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);\n-\ttotal_size = table_meta_sz + bucket_sz + bucket_ext_sz + key_sz +\n-\t\tkey_stack_sz + bkt_ext_stack_sz + data_sz;\n+\t\tRTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));\n+\tdata_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * entry_size);\n+\ttotal_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz +\n+\t\tkey_sz + key_stack_sz + bkt_ext_stack_sz + data_sz;\n \n-\tt = rte_zmalloc_socket(\"TABLE\", total_size, RTE_CACHE_LINE_SIZE, socket_id);\n+\tt = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);\n \tif (t == NULL) {\n \t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\"%s: Cannot allocate %u bytes for hash table\\n\",\n-\t\t\t__func__, total_size);\n+\t\t\t\"%s: Cannot allocate %u bytes for hash table %s\\n\",\n+\t\t\t__func__, total_size, p->name);\n \t\treturn NULL;\n \t}\n-\tRTE_LOG(INFO, TABLE, \"%s (%u-byte key): Hash table memory footprint is \"\n-\t\t\"%u bytes\\n\", __func__, p->key_size, total_size);\n+\tRTE_LOG(INFO, TABLE, \"%s (%u-byte key): Hash table %s memory footprint is \"\n+\t\t\"%u bytes\\n\", __func__, p->key_size, p->name, total_size);\n \n \t/* Memory initialization */\n \tt->key_size = p->key_size;\n \tt->entry_size = entry_size;\n-\tt->n_keys = p->n_keys;\n+\tt->n_keys = n_keys;\n \tt->n_buckets = p->n_buckets;\n-\tt->n_buckets_ext = p->n_buckets_ext;\n+\tt->n_buckets_ext = n_buckets_ext;\n \tt->f_hash = p->f_hash;\n \tt->seed = p->seed;\n-\tt->signature_offset = p->signature_offset;\n \tt->key_offset = p->key_offset;\n \n \t/* Internal */\n@@ -228,13 +272,15 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)\n \tt->data_size_shl = __builtin_ctzl(entry_size);\n \n \t/* Tables */\n-\tbucket_offset = 0;\n+\tkey_mask_offset = 0;\n+\tbucket_offset = key_mask_offset + key_mask_sz;\n \tbucket_ext_offset = bucket_offset + bucket_sz;\n \tkey_offset = bucket_ext_offset + bucket_ext_sz;\n \tkey_stack_offset = key_offset + key_sz;\n \tbkt_ext_stack_offset = key_stack_offset + key_stack_sz;\n \tdata_offset = bkt_ext_stack_offset + bkt_ext_stack_sz;\n \n+\tt->key_mask = (uint64_t *) &t->memory[key_mask_offset];\n \tt->buckets = (struct bucket *) &t->memory[bucket_offset];\n \tt->buckets_ext = (struct bucket *) &t->memory[bucket_ext_offset];\n \tt->key_mem = &t->memory[key_offset];\n@@ -242,6 +288,12 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)\n \tt->bkt_ext_stack = (uint32_t *) &t->memory[bkt_ext_stack_offset];\n \tt->data_mem = &t->memory[data_offset];\n \n+\t/* Key mask */\n+\tif (p->key_mask == NULL)\n+\t\tmemset(t->key_mask, 0xFF, p->key_size);\n+\telse\n+\t\tmemcpy(t->key_mask, p->key_mask, p->key_size);\n+\n \t/* Key stack */\n \tfor (i = 0; i < t->n_keys; i++)\n \t\tt->key_stack[i] = t->n_keys - 1 - i;\n@@ -277,7 +329,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,\n \tuint64_t sig;\n \tuint32_t bkt_index, i;\n \n-\tsig = t->f_hash(key, t->key_size, t->seed);\n+\tsig = t->f_hash(key, t->key_mask, t->key_size, t->seed);\n \tbkt_index = sig & t->bucket_mask;\n \tbkt0 = &t->buckets[bkt_index];\n \tsig = (sig >> 16) | 1LLU;\n@@ -290,7 +342,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,\n \t\t\tuint8_t *bkt_key =\n \t\t\t\t&t->key_mem[bkt_key_index << t->key_size_shl];\n \n-\t\t\tif ((sig == bkt_sig) && (memcmp(key, bkt_key,\n+\t\t\tif ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,\n \t\t\t\tt->key_size) == 0)) {\n \t\t\t\tuint8_t *data = &t->data_mem[bkt_key_index <<\n \t\t\t\t\tt->data_size_shl];\n@@ -327,7 +379,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,\n \n \t\t\t\tbkt->sig[i] = (uint16_t) sig;\n \t\t\t\tbkt->key_pos[i] = bkt_key_index;\n-\t\t\t\tmemcpy(bkt_key, key, t->key_size);\n+\t\t\t\tkeycpy(bkt_key, key, t->key_mask, t->key_size);\n \t\t\t\tmemcpy(data, entry, t->entry_size);\n \n \t\t\t\t*key_found = 0;\n@@ -358,7 +410,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,\n \t\t/* Install new key into bucket */\n \t\tbkt->sig[0] = (uint16_t) sig;\n \t\tbkt->key_pos[0] = bkt_key_index;\n-\t\tmemcpy(bkt_key, key, t->key_size);\n+\t\tkeycpy(bkt_key, key, t->key_mask, t->key_size);\n \t\tmemcpy(data, entry, t->entry_size);\n \n \t\t*key_found = 0;\n@@ -378,7 +430,7 @@ void *entry)\n \tuint64_t sig;\n \tuint32_t bkt_index, i;\n \n-\tsig = t->f_hash(key, t->key_size, t->seed);\n+\tsig = t->f_hash(key, t->key_mask, t->key_size, t->seed);\n \tbkt_index = sig & t->bucket_mask;\n \tbkt0 = &t->buckets[bkt_index];\n \tsig = (sig >> 16) | 1LLU;\n@@ -392,7 +444,7 @@ void *entry)\n \t\t\tuint8_t *bkt_key = &t->key_mem[bkt_key_index <<\n \t\t\t\tt->key_size_shl];\n \n-\t\t\tif ((sig == bkt_sig) && (memcmp(key, bkt_key,\n+\t\t\tif ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,\n \t\t\t\tt->key_size) == 0)) {\n \t\t\t\tuint8_t *data = &t->data_mem[bkt_key_index <<\n \t\t\t\t\tt->data_size_shl];\n@@ -437,8 +489,7 @@ static int rte_table_hash_ext_lookup_unoptimized(\n \tstruct rte_mbuf **pkts,\n \tuint64_t pkts_mask,\n \tuint64_t *lookup_hit_mask,\n-\tvoid **entries,\n-\tint dosig)\n+\tvoid **entries)\n {\n \tstruct rte_table_hash *t = (struct rte_table_hash *) table;\n \tuint64_t pkts_mask_out = 0;\n@@ -458,11 +509,7 @@ static int rte_table_hash_ext_lookup_unoptimized(\n \n \t\tpkt = pkts[pkt_index];\n \t\tkey = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);\n-\t\tif (dosig)\n-\t\t\tsig = (uint64_t) t->f_hash(key, t->key_size, t->seed);\n-\t\telse\n-\t\t\tsig = RTE_MBUF_METADATA_UINT32(pkt,\n-\t\t\t\tt->signature_offset);\n+\t\tsig = (uint64_t) t->f_hash(key, t->key_mask, t->key_size, t->seed);\n \n \t\tbkt_index = sig & t->bucket_mask;\n \t\tbkt0 = &t->buckets[bkt_index];\n@@ -476,8 +523,8 @@ static int rte_table_hash_ext_lookup_unoptimized(\n \t\t\t\tuint8_t *bkt_key = &t->key_mem[bkt_key_index <<\n \t\t\t\t\tt->key_size_shl];\n \n-\t\t\t\tif ((sig == bkt_sig) && (memcmp(key, bkt_key,\n-\t\t\t\t\tt->key_size) == 0)) {\n+\t\t\t\tif ((sig == bkt_sig) && (keycmp(bkt_key, key,\n+\t\t\t\t\tt->key_mask, t->key_size) == 0)) {\n \t\t\t\t\tuint8_t *data = &t->data_mem[\n \t\t\t\t\tbkt_key_index << t->data_size_shl];\n \n@@ -576,11 +623,12 @@ static int rte_table_hash_ext_lookup_unoptimized(\n {\t\t\t\t\t\t\t\t\t\\\n \tuint64_t *pkt_key = RTE_MBUF_METADATA_UINT64_PTR(mbuf, f->key_offset);\\\n \tuint64_t *bkt_key = (uint64_t *) key;\t\t\t\t\\\n+\tuint64_t *key_mask = f->key_mask;\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\t\\\n \tswitch (f->key_size) {\t\t\t\t\t\t\\\n \tcase 8:\t\t\t\t\t\t\t\t\\\n \t{\t\t\t\t\t\t\t\t\\\n-\t\tuint64_t xor = pkt_key[0] ^ bkt_key[0];\t\t\t\\\n+\t\tuint64_t xor = (pkt_key[0] & key_mask[0]) ^ bkt_key[0];\t\\\n \t\tmatch_key = 0;\t\t\t\t\t\t\\\n \t\tif (xor == 0)\t\t\t\t\t\t\\\n \t\t\tmatch_key = 1;\t\t\t\t\t\\\n@@ -591,8 +639,8 @@ static int rte_table_hash_ext_lookup_unoptimized(\n \t{\t\t\t\t\t\t\t\t\\\n \t\tuint64_t xor[2], or;\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\t\\\n-\t\txor[0] = pkt_key[0] ^ bkt_key[0];\t\t\t\\\n-\t\txor[1] = pkt_key[1] ^ bkt_key[1];\t\t\t\\\n+\t\txor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0];\t\t\\\n+\t\txor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1];\t\t\\\n \t\tor = xor[0] | xor[1];\t\t\t\t\t\\\n \t\tmatch_key = 0;\t\t\t\t\t\t\\\n \t\tif (or == 0)\t\t\t\t\t\t\\\n@@ -604,10 +652,10 @@ static int rte_table_hash_ext_lookup_unoptimized(\n \t{\t\t\t\t\t\t\t\t\\\n \t\tuint64_t xor[4], or;\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\t\\\n-\t\txor[0] = pkt_key[0] ^ bkt_key[0];\t\t\t\\\n-\t\txor[1] = pkt_key[1] ^ bkt_key[1];\t\t\t\\\n-\t\txor[2] = pkt_key[2] ^ bkt_key[2];\t\t\t\\\n-\t\txor[3] = pkt_key[3] ^ bkt_key[3];\t\t\t\\\n+\t\txor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0];\t\t\\\n+\t\txor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1];\t\t\\\n+\t\txor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2];\t\t\\\n+\t\txor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3];\t\t\\\n \t\tor = xor[0] | xor[1] | xor[2] | xor[3];\t\t\t\\\n \t\tmatch_key = 0;\t\t\t\t\t\t\\\n \t\tif (or == 0)\t\t\t\t\t\t\\\n@@ -619,14 +667,14 @@ static int rte_table_hash_ext_lookup_unoptimized(\n \t{\t\t\t\t\t\t\t\t\\\n \t\tuint64_t xor[8], or;\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\t\\\n-\t\txor[0] = pkt_key[0] ^ bkt_key[0];\t\t\t\\\n-\t\txor[1] = pkt_key[1] ^ bkt_key[1];\t\t\t\\\n-\t\txor[2] = pkt_key[2] ^ bkt_key[2];\t\t\t\\\n-\t\txor[3] = pkt_key[3] ^ bkt_key[3];\t\t\t\\\n-\t\txor[4] = pkt_key[4] ^ bkt_key[4];\t\t\t\\\n-\t\txor[5] = pkt_key[5] ^ bkt_key[5];\t\t\t\\\n-\t\txor[6] = pkt_key[6] ^ bkt_key[6];\t\t\t\\\n-\t\txor[7] = pkt_key[7] ^ bkt_key[7];\t\t\t\\\n+\t\txor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0];\t\t\\\n+\t\txor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1];\t\t\\\n+\t\txor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2];\t\t\\\n+\t\txor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3];\t\t\\\n+\t\txor[4] = (pkt_key[4] & key_mask[4]) ^ bkt_key[4];\t\t\\\n+\t\txor[5] = (pkt_key[5] & key_mask[5]) ^ bkt_key[5];\t\t\\\n+\t\txor[6] = (pkt_key[6] & key_mask[6]) ^ bkt_key[6];\t\t\\\n+\t\txor[7] = (pkt_key[7] & key_mask[7]) ^ bkt_key[7];\t\t\\\n \t\tor = xor[0] | xor[1] | xor[2] | xor[3] |\t\t\\\n \t\t\txor[4] | xor[5] | xor[6] | xor[7];\t\t\\\n \t\tmatch_key = 0;\t\t\t\t\t\t\\\n@@ -637,7 +685,7 @@ static int rte_table_hash_ext_lookup_unoptimized(\n \t\t\t\t\t\t\t\t\t\\\n \tdefault:\t\t\t\t\t\t\t\\\n \t\tmatch_key = 0;\t\t\t\t\t\t\\\n-\t\tif (memcmp(pkt_key, bkt_key, f->key_size) == 0)\t\t\\\n+\t\tif (keycmp(bkt_key, pkt_key, key_mask, f->key_size) == 0)\t\\\n \t\t\tmatch_key = 1;\t\t\t\t\t\\\n \t}\t\t\t\t\t\t\t\t\\\n }\n@@ -685,38 +733,7 @@ static int rte_table_hash_ext_lookup_unoptimized(\n \trte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\\\n }\n \n-#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index)\t\t\\\n-{\t\t\t\t\t\t\t\t\t\\\n-\tstruct grinder *g10, *g11;\t\t\t\t\t\\\n-\tuint64_t sig10, sig11, bkt10_index, bkt11_index;\t\t\\\n-\tstruct rte_mbuf *mbuf10, *mbuf11;\t\t\t\t\\\n-\tstruct bucket *bkt10, *bkt11, *buckets = t->buckets;\t\t\\\n-\tuint64_t bucket_mask = t->bucket_mask;\t\t\t\t\\\n-\tuint32_t signature_offset = t->signature_offset;\t\t\\\n-\t\t\t\t\t\t\t\t\t\\\n-\tmbuf10 = pkts[pkt10_index];\t\t\t\t\t\\\n-\tsig10 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf10, signature_offset);\\\n-\tbkt10_index = sig10 & bucket_mask;\t\t\t\t\\\n-\tbkt10 = &buckets[bkt10_index];\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\t\\\n-\tmbuf11 = pkts[pkt11_index];\t\t\t\t\t\\\n-\tsig11 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf11, signature_offset);\\\n-\tbkt11_index = sig11 & bucket_mask;\t\t\t\t\\\n-\tbkt11 = &buckets[bkt11_index];\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\t\\\n-\trte_prefetch0(bkt10);\t\t\t\t\t\t\\\n-\trte_prefetch0(bkt11);\t\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\t\\\n-\tg10 = &g[pkt10_index];\t\t\t\t\t\t\\\n-\tg10->sig = sig10;\t\t\t\t\t\t\\\n-\tg10->bkt = bkt10;\t\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\t\\\n-\tg11 = &g[pkt11_index];\t\t\t\t\t\t\\\n-\tg11->sig = sig11;\t\t\t\t\t\t\\\n-\tg11->bkt = bkt11;\t\t\t\t\t\t\\\n-}\n-\n-#define lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index)\t\\\n+#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index)\t\\\n {\t\t\t\t\t\t\t\t\t\\\n \tstruct grinder *g10, *g11;\t\t\t\t\t\\\n \tuint64_t sig10, sig11, bkt10_index, bkt11_index;\t\t\\\n@@ -731,13 +748,13 @@ static int rte_table_hash_ext_lookup_unoptimized(\n \t\t\t\t\t\t\t\t\t\\\n \tmbuf10 = pkts[pkt10_index];\t\t\t\t\t\\\n \tkey10 = RTE_MBUF_METADATA_UINT8_PTR(mbuf10, key_offset);\t\\\n-\tsig10 = (uint64_t) f_hash(key10, key_size, seed);\t\t\\\n+\tsig10 = (uint64_t) f_hash(key10, t->key_mask, key_size, seed);\t\\\n \tbkt10_index = sig10 & bucket_mask;\t\t\t\t\\\n \tbkt10 = &buckets[bkt10_index];\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\t\\\n \tmbuf11 = pkts[pkt11_index];\t\t\t\t\t\\\n \tkey11 = RTE_MBUF_METADATA_UINT8_PTR(mbuf11, key_offset);\t\\\n-\tsig11 = (uint64_t) f_hash(key11, key_size, seed);\t\t\\\n+\tsig11 = (uint64_t) f_hash(key11, t->key_mask, key_size, seed);\t\\\n \tbkt11_index = sig11 & bucket_mask;\t\t\t\t\\\n \tbkt11 = &buckets[bkt11_index];\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\t\\\n@@ -874,7 +891,7 @@ static int rte_table_hash_ext_lookup(\n \t/* Cannot run the pipeline with less than 7 packets */\n \tif (__builtin_popcountll(pkts_mask) < 7) {\n \t\tstatus = rte_table_hash_ext_lookup_unoptimized(table, pkts,\n-\t\t\tpkts_mask, lookup_hit_mask, entries, 0);\n+\t\t\tpkts_mask, lookup_hit_mask, entries);\n \t\tRTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in -\n \t\t\t\t__builtin_popcountll(*lookup_hit_mask));\n \t\treturn status;\n@@ -982,144 +999,7 @@ static int rte_table_hash_ext_lookup(\n \t\tuint64_t pkts_mask_out_slow = 0;\n \n \t\tstatus = rte_table_hash_ext_lookup_unoptimized(table, pkts,\n-\t\t\tpkts_mask_match_many, &pkts_mask_out_slow, entries, 0);\n-\t\tpkts_mask_out |= pkts_mask_out_slow;\n-\t}\n-\n-\t*lookup_hit_mask = pkts_mask_out;\n-\tRTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in - __builtin_popcountll(pkts_mask_out));\n-\treturn status;\n-}\n-\n-static int rte_table_hash_ext_lookup_dosig(\n-\tvoid *table,\n-\tstruct rte_mbuf **pkts,\n-\tuint64_t pkts_mask,\n-\tuint64_t *lookup_hit_mask,\n-\tvoid **entries)\n-{\n-\tstruct rte_table_hash *t = (struct rte_table_hash *) table;\n-\tstruct grinder *g = t->grinders;\n-\tuint64_t pkt00_index, pkt01_index, pkt10_index, pkt11_index;\n-\tuint64_t pkt20_index, pkt21_index, pkt30_index, pkt31_index;\n-\tuint64_t pkts_mask_out = 0, pkts_mask_match_many = 0;\n-\tint status = 0;\n-\n-\t__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);\n-\tRTE_TABLE_HASH_EXT_STATS_PKTS_IN_ADD(t, n_pkts_in);\n-\n-\t/* Cannot run the pipeline with less than 7 packets */\n-\tif (__builtin_popcountll(pkts_mask) < 7) {\n-\t\tstatus = rte_table_hash_ext_lookup_unoptimized(table, pkts,\n-\t\t\tpkts_mask, lookup_hit_mask, entries, 1);\n-\t\tRTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in -\n-\t\t\t\t__builtin_popcountll(*lookup_hit_mask));\n-\t\treturn status;\n-\t}\n-\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);\n-\n-\t/* Pipeline feed */\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);\n-\n-\t/* Pipeline feed */\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);\n-\n-\t/*\n-\t* Pipeline run\n-\t*\n-\t*/\n-\tfor ( ; pkts_mask; ) {\n-\t\t/* Pipeline feed */\n-\t\tpkt30_index = pkt20_index;\n-\t\tpkt31_index = pkt21_index;\n-\t\tpkt20_index = pkt10_index;\n-\t\tpkt21_index = pkt11_index;\n-\t\tpkt10_index = pkt00_index;\n-\t\tpkt11_index = pkt01_index;\n-\n-\t\t/* Pipeline stage 0 */\n-\t\tlookup2_stage0_with_odd_support(t, g, pkts, pkts_mask,\n-\t\t\tpkt00_index, pkt01_index);\n-\n-\t\t/* Pipeline stage 1 */\n-\t\tlookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);\n-\n-\t\t/* Pipeline stage 2 */\n-\t\tlookup2_stage2(t, g, pkt20_index, pkt21_index,\n-\t\t\tpkts_mask_match_many);\n-\n-\t\t/* Pipeline stage 3 */\n-\t\tlookup2_stage3(t, g, pkts, pkt30_index, pkt31_index,\n-\t\t\tpkts_mask_out, entries);\n-\t}\n-\n-\t/* Pipeline feed */\n-\tpkt30_index = pkt20_index;\n-\tpkt31_index = pkt21_index;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);\n-\n-\t/* Pipeline stage 3 */\n-\tlookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,\n-\t\tentries);\n-\n-\t/* Pipeline feed */\n-\tpkt30_index = pkt20_index;\n-\tpkt31_index = pkt21_index;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);\n-\n-\t/* Pipeline stage 3 */\n-\tlookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,\n-\t\tentries);\n-\n-\t/* Pipeline feed */\n-\tpkt30_index = pkt20_index;\n-\tpkt31_index = pkt21_index;\n-\n-\t/* Pipeline stage 3 */\n-\tlookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,\n-\t\tentries);\n-\n-\t/* Slow path */\n-\tpkts_mask_match_many &= ~pkts_mask_out;\n-\tif (pkts_mask_match_many) {\n-\t\tuint64_t pkts_mask_out_slow = 0;\n-\n-\t\tstatus = rte_table_hash_ext_lookup_unoptimized(table, pkts,\n-\t\t\tpkts_mask_match_many, &pkts_mask_out_slow, entries, 1);\n+\t\t\tpkts_mask_match_many, &pkts_mask_out_slow, entries);\n \t\tpkts_mask_out |= pkts_mask_out_slow;\n \t}\n \n@@ -1152,14 +1032,3 @@ struct rte_table_ops rte_table_hash_ext_ops\t = {\n \t.f_lookup = rte_table_hash_ext_lookup,\n \t.f_stats = rte_table_hash_ext_stats_read,\n };\n-\n-struct rte_table_ops rte_table_hash_ext_dosig_ops  = {\n-\t.f_create = rte_table_hash_ext_create,\n-\t.f_free = rte_table_hash_ext_free,\n-\t.f_add = rte_table_hash_ext_entry_add,\n-\t.f_delete = rte_table_hash_ext_entry_delete,\n-\t.f_add_bulk = NULL,\n-\t.f_delete_bulk = NULL,\n-\t.f_lookup = rte_table_hash_ext_lookup_dosig,\n-\t.f_stats = rte_table_hash_ext_stats_read,\n-};\ndiff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c\nindex ce057b7..f6cf6c6 100644\n--- a/lib/librte_table/rte_table_hash_key16.c\n+++ b/lib/librte_table/rte_table_hash_key16.c\n@@ -1,34 +1,34 @@\n /*-\n- *\t BSD LICENSE\n+ *   BSD LICENSE\n  *\n- *\t Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n- *\t All rights reserved.\n+ *   Copyright(c) 2010-2017 Intel Corporation. All rights reserved.\n+ *   All rights reserved.\n  *\n- *\t Redistribution and use in source and binary forms, with or without\n- *\t modification, are permitted provided that the following conditions\n- *\t are met:\n+ *   Redistribution and use in source and binary forms, with or without\n+ *   modification, are permitted provided that the following conditions\n+ *   are met:\n  *\n- *\t* Redistributions of source code must retain the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer.\n- *\t* Redistributions in binary form must reproduce the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer in\n- *\t\t the documentation and/or other materials provided with the\n- *\t\t distribution.\n- *\t* Neither the name of Intel Corporation nor the names of its\n- *\t\t contributors may be used to endorse or promote products derived\n- *\t\t from this software without specific prior written permission.\n+ *     * Redistributions of source code must retain the above copyright\n+ *       notice, this list of conditions and the following disclaimer.\n+ *     * Redistributions in binary form must reproduce the above copyright\n+ *       notice, this list of conditions and the following disclaimer in\n+ *       the documentation and/or other materials provided with the\n+ *       distribution.\n+ *     * Neither the name of Intel Corporation nor the names of its\n+ *       contributors may be used to endorse or promote products derived\n+ *       from this software without specific prior written permission.\n  *\n- *\t THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n- *\t \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n- *\t LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n- *\t A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n- *\t OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n- *\t SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n- *\t LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n- *\t DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n- *\t THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n- *\t (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n- *\t OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ *   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n  */\n #include <string.h>\n #include <stdio.h>\n@@ -42,7 +42,9 @@\n #include \"rte_table_hash.h\"\n #include \"rte_lru.h\"\n \n-#define RTE_TABLE_HASH_KEY_SIZE\t\t\t\t\t\t16\n+#define KEY_SIZE\t\t\t\t\t\t16\n+\n+#define KEYS_PER_BUCKET\t\t\t\t\t4\n \n #define RTE_BUCKET_ENTRY_VALID\t\t\t\t\t\t0x1LLU\n \n@@ -79,11 +81,9 @@ struct rte_table_hash {\n \n \t/* Input parameters */\n \tuint32_t n_buckets;\n-\tuint32_t n_entries_per_bucket;\n \tuint32_t key_size;\n \tuint32_t entry_size;\n \tuint32_t bucket_size;\n-\tuint32_t signature_offset;\n \tuint32_t key_offset;\n \tuint64_t key_mask[2];\n \trte_table_hash_op_hash f_hash;\n@@ -99,17 +99,47 @@ struct rte_table_hash {\n };\n \n static int\n-check_params_create_lru(struct rte_table_hash_key16_lru_params *params) {\n-\t/* n_entries */\n-\tif (params->n_entries == 0) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: n_entries is zero\\n\", __func__);\n+keycmp(void *a, void *b, void *b_mask)\n+{\n+\tuint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;\n+\n+\treturn (a64[0] != (b64[0] & b_mask64[0])) ||\n+\t\t(a64[1] != (b64[1] & b_mask64[1]));\n+}\n+\n+static void\n+keycpy(void *dst, void *src, void *src_mask)\n+{\n+\tuint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;\n+\n+\tdst64[0] = src64[0] & src_mask64[0];\n+\tdst64[1] = src64[1] & src_mask64[1];\n+}\n+\n+static int\n+check_params_create(struct rte_table_hash_params *params) {\n+\t/* name */\n+\tif (params->name == NULL) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: name invalid value\\n\", __func__);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* key_size */\n+\tif (params->key_size != KEY_SIZE) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: key_size invalid value\\n\", __func__);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* n_keys */\n+\tif (params->n_keys == 0) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: n_keys is zero\\n\", __func__);\n \t\treturn -EINVAL;\n \t}\n \n \t/* f_hash */\n \tif (params->f_hash == NULL) {\n-\t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\"%s: f_hash function pointer is NULL\\n\", __func__);\n+\t\tRTE_LOG(ERR, TABLE, \"%s: f_hash function pointer is NULL\\n\",\n+\t\t\t__func__);\n \t\treturn -EINVAL;\n \t}\n \n@@ -121,46 +151,45 @@ rte_table_hash_create_key16_lru(void *params,\n \t\tint socket_id,\n \t\tuint32_t entry_size)\n {\n-\tstruct rte_table_hash_key16_lru_params *p =\n-\t\t\t(struct rte_table_hash_key16_lru_params *) params;\n+\tstruct rte_table_hash_params *p = params;\n \tstruct rte_table_hash *f;\n-\tuint32_t n_buckets, n_entries_per_bucket,\n-\t\t\tkey_size, bucket_size_cl, total_size, i;\n+\tuint32_t n_buckets, bucket_size_cl;\n+\tuint32_t total_size, n_keys, i;\n \n \t/* Check input parameters */\n-\tif ((check_params_create_lru(p) != 0) ||\n+\tif ((check_params_create(p) != 0) ||\n \t\t((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||\n \t\t((sizeof(struct rte_bucket_4_16) % 64) != 0))\n \t\treturn NULL;\n-\tn_entries_per_bucket = 4;\n-\tkey_size = 16;\n+\n+\tn_keys = p->n_keys;\n+\tif (n_keys < KEYS_PER_BUCKET)\n+\t\tn_keys = KEYS_PER_BUCKET;\n \n \t/* Memory allocation */\n-\tn_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /\n-\t\tn_entries_per_bucket);\n-\tbucket_size_cl = (sizeof(struct rte_bucket_4_16) + n_entries_per_bucket\n-\t\t* entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;\n+\tn_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /\n+\t\tKEYS_PER_BUCKET);\n+\tbucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_16) +\n+\t\tKEYS_PER_BUCKET * entry_size);\n \ttotal_size = sizeof(struct rte_table_hash) + n_buckets *\n \t\tbucket_size_cl * RTE_CACHE_LINE_SIZE;\n \n-\tf = rte_zmalloc_socket(\"TABLE\", total_size, RTE_CACHE_LINE_SIZE, socket_id);\n+\tf = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);\n \tif (f == NULL) {\n \t\tRTE_LOG(ERR, TABLE,\n-\t\t\"%s: Cannot allocate %u bytes for hash table\\n\",\n-\t\t__func__, total_size);\n+\t\t\"%s: Cannot allocate %u bytes for hash table %s\\n\",\n+\t\t__func__, total_size, p->name);\n \t\treturn NULL;\n \t}\n \tRTE_LOG(INFO, TABLE,\n-\t\t\"%s: Hash table memory footprint is %u bytes\\n\",\n-\t\t__func__, total_size);\n+\t\t\"%s: Hash table %s memory footprint is %u bytes\\n\",\n+\t\t__func__, p->name, total_size);\n \n \t/* Memory initialization */\n \tf->n_buckets = n_buckets;\n-\tf->n_entries_per_bucket = n_entries_per_bucket;\n-\tf->key_size = key_size;\n+\tf->key_size = KEY_SIZE;\n \tf->entry_size = entry_size;\n \tf->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;\n-\tf->signature_offset = p->signature_offset;\n \tf->key_offset = p->key_offset;\n \tf->f_hash = p->f_hash;\n \tf->seed = p->seed;\n@@ -212,19 +241,19 @@ rte_table_hash_entry_add_key16_lru(\n \tuint64_t signature, pos;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket = (struct rte_bucket_4_16 *)\n-\t\t\t&f->memory[bucket_index * f->bucket_size];\n+\t\t&f->memory[bucket_index * f->bucket_size];\n \tsignature |= RTE_BUCKET_ENTRY_VALID;\n \n \t/* Key is present in the bucket */\n \tfor (i = 0; i < 4; i++) {\n \t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\tif ((bucket_signature == signature) &&\n-\t\t\t\t(memcmp(key, bucket_key, f->key_size) == 0)) {\n+\t\t\t(keycmp(bucket_key, key, f->key_mask) == 0)) {\n \t\t\tuint8_t *bucket_data = &bucket->data[i * f->entry_size];\n \n \t\t\tmemcpy(bucket_data, entry, f->entry_size);\n@@ -238,13 +267,13 @@ rte_table_hash_entry_add_key16_lru(\n \t/* Key is not present in the bucket */\n \tfor (i = 0; i < 4; i++) {\n \t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\tif (bucket_signature == 0) {\n \t\t\tuint8_t *bucket_data = &bucket->data[i * f->entry_size];\n \n \t\t\tbucket->signature[i] = signature;\n-\t\t\tmemcpy(bucket_key, key, f->key_size);\n+\t\t\tkeycpy(bucket_key, key, f->key_mask);\n \t\t\tmemcpy(bucket_data, entry, f->entry_size);\n \t\t\tlru_update(bucket, i);\n \t\t\t*key_found = 0;\n@@ -257,7 +286,7 @@ rte_table_hash_entry_add_key16_lru(\n \t/* Bucket full: replace LRU entry */\n \tpos = lru_pos(bucket);\n \tbucket->signature[pos] = signature;\n-\tmemcpy(bucket->key[pos], key, f->key_size);\n+\tkeycpy(&bucket->key[pos], key, f->key_mask);\n \tmemcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);\n \tlru_update(bucket, pos);\n \t*key_found = 0;\n@@ -278,19 +307,19 @@ rte_table_hash_entry_delete_key16_lru(\n \tuint64_t signature;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket = (struct rte_bucket_4_16 *)\n-\t\t\t&f->memory[bucket_index * f->bucket_size];\n+\t\t&f->memory[bucket_index * f->bucket_size];\n \tsignature |= RTE_BUCKET_ENTRY_VALID;\n \n \t/* Key is present in the bucket */\n \tfor (i = 0; i < 4; i++) {\n \t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\tif ((bucket_signature == signature) &&\n-\t\t\t\t(memcmp(key, bucket_key, f->key_size) == 0)) {\n+\t\t\t(keycmp(bucket_key, key, f->key_mask) == 0)) {\n \t\t\tuint8_t *bucket_data = &bucket->data[i * f->entry_size];\n \n \t\t\tbucket->signature[i] = 0;\n@@ -306,81 +335,68 @@ rte_table_hash_entry_delete_key16_lru(\n \treturn 0;\n }\n \n-static int\n-check_params_create_ext(struct rte_table_hash_key16_ext_params *params) {\n-\t/* n_entries */\n-\tif (params->n_entries == 0) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: n_entries is zero\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\t/* n_entries_ext */\n-\tif (params->n_entries_ext == 0) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: n_entries_ext is zero\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\t/* f_hash */\n-\tif (params->f_hash == NULL) {\n-\t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\"%s: f_hash function pointer is NULL\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\treturn 0;\n-}\n-\n static void *\n rte_table_hash_create_key16_ext(void *params,\n \t\tint socket_id,\n \t\tuint32_t entry_size)\n {\n-\tstruct rte_table_hash_key16_ext_params *p =\n-\t\t\t(struct rte_table_hash_key16_ext_params *) params;\n+\tstruct rte_table_hash_params *p = params;\n \tstruct rte_table_hash *f;\n-\tuint32_t n_buckets, n_buckets_ext, n_entries_per_bucket, key_size,\n-\t\t\tbucket_size_cl, stack_size_cl, total_size, i;\n+\tuint32_t n_keys, n_buckets, n_buckets_ext;\n+\tuint32_t bucket_size_cl, stack_size_cl, total_size, i;\n \n \t/* Check input parameters */\n-\tif ((check_params_create_ext(p) != 0) ||\n+\tif ((check_params_create(p) != 0) ||\n \t\t((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||\n \t\t((sizeof(struct rte_bucket_4_16) % 64) != 0))\n \t\treturn NULL;\n \n-\tn_entries_per_bucket = 4;\n-\tkey_size = 16;\n+\t/*\n+\t * Table dimensioning\n+\t *\n+\t * Objective: Pick the number of bucket extensions (n_buckets_ext) so that\n+\t * it is guaranteed that n_keys keys can be stored in the table at any time.\n+\t *\n+\t * The worst case scenario takes place when all the n_keys keys fall into\n+\t * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst\n+\t * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the\n+\t * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall\n+\t * into a different bucket. This case defeats the purpose of the hash table.\n+\t * It indicates unsuitable f_hash or n_keys to n_buckets ratio.\n+\t *\n+\t * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1\n+\t */\n+\tn_keys = p->n_keys;\n+\tif (n_keys < KEYS_PER_BUCKET)\n+\t\tn_keys = KEYS_PER_BUCKET;\n+\tn_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;\n \n \t/* Memory allocation */\n-\tn_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /\n-\t\tn_entries_per_bucket);\n-\tn_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /\n-\t\tn_entries_per_bucket;\n-\tbucket_size_cl = (sizeof(struct rte_bucket_4_16) + n_entries_per_bucket\n-\t\t* entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;\n-\tstack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)\n-\t\t/ RTE_CACHE_LINE_SIZE;\n+\tn_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /\n+\t\tKEYS_PER_BUCKET);\n+\tbucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_16) +\n+\t\tKEYS_PER_BUCKET * entry_size);\n+\tstack_size_cl = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));\n \ttotal_size = sizeof(struct rte_table_hash) +\n \t\t((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *\n \t\tRTE_CACHE_LINE_SIZE;\n \n-\tf = rte_zmalloc_socket(\"TABLE\", total_size, RTE_CACHE_LINE_SIZE, socket_id);\n+\tf = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);\n \tif (f == NULL) {\n \t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\"%s: Cannot allocate %u bytes for hash table\\n\",\n-\t\t\t__func__, total_size);\n+\t\t\t\"%s: Cannot allocate %u bytes for hash table %s\\n\",\n+\t\t\t__func__, total_size, p->name);\n \t\treturn NULL;\n \t}\n \tRTE_LOG(INFO, TABLE,\n-\t\t\"%s: Hash table memory footprint is %u bytes\\n\",\n-\t\t__func__, total_size);\n+\t\t\"%s: Hash table %s memory footprint is %u bytes\\n\",\n+\t\t__func__, p->name, total_size);\n \n \t/* Memory initialization */\n \tf->n_buckets = n_buckets;\n-\tf->n_entries_per_bucket = n_entries_per_bucket;\n-\tf->key_size = key_size;\n+\tf->key_size = KEY_SIZE;\n \tf->entry_size = entry_size;\n \tf->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;\n-\tf->signature_offset = p->signature_offset;\n \tf->key_offset = p->key_offset;\n \tf->f_hash = p->f_hash;\n \tf->seed = p->seed;\n@@ -390,9 +406,6 @@ rte_table_hash_create_key16_ext(void *params,\n \tf->stack = (uint32_t *)\n \t\t&f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];\n \n-\tfor (i = 0; i < n_buckets_ext; i++)\n-\t\tf->stack[i] = i;\n-\n \tif (p->key_mask != NULL) {\n \t\tf->key_mask[0] = (((uint64_t *)p->key_mask)[0]);\n \t\tf->key_mask[1] = (((uint64_t *)p->key_mask)[1]);\n@@ -401,6 +414,9 @@ rte_table_hash_create_key16_ext(void *params,\n \t\tf->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;\n \t}\n \n+\tfor (i = 0; i < n_buckets_ext; i++)\n+\t\tf->stack[i] = i;\n+\n \treturn f;\n }\n \n@@ -432,20 +448,20 @@ rte_table_hash_entry_add_key16_ext(\n \tuint64_t signature;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket0 = (struct rte_bucket_4_16 *)\n-\t\t\t&f->memory[bucket_index * f->bucket_size];\n+\t\t&f->memory[bucket_index * f->bucket_size];\n \tsignature |= RTE_BUCKET_ENTRY_VALID;\n \n \t/* Key is present in the bucket */\n \tfor (bucket = bucket0; bucket != NULL; bucket = bucket->next)\n \t\tfor (i = 0; i < 4; i++) {\n \t\t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\t\tif ((bucket_signature == signature) &&\n-\t\t\t\t(memcmp(key, bucket_key, f->key_size) == 0)) {\n+\t\t\t\t(keycmp(bucket_key, key, f->key_mask) == 0)) {\n \t\t\t\tuint8_t *bucket_data = &bucket->data[i *\n \t\t\t\t\tf->entry_size];\n \n@@ -458,17 +474,17 @@ rte_table_hash_entry_add_key16_ext(\n \n \t/* Key is not present in the bucket */\n \tfor (bucket_prev = NULL, bucket = bucket0; bucket != NULL;\n-\t\t\t bucket_prev = bucket, bucket = bucket->next)\n+\t\tbucket_prev = bucket, bucket = bucket->next)\n \t\tfor (i = 0; i < 4; i++) {\n \t\t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\t\tif (bucket_signature == 0) {\n \t\t\t\tuint8_t *bucket_data = &bucket->data[i *\n \t\t\t\t\tf->entry_size];\n \n \t\t\t\tbucket->signature[i] = signature;\n-\t\t\t\tmemcpy(bucket_key, key, f->key_size);\n+\t\t\t\tkeycpy(bucket_key, key, f->key_mask);\n \t\t\t\tmemcpy(bucket_data, entry, f->entry_size);\n \t\t\t\t*key_found = 0;\n \t\t\t\t*entry_ptr = (void *) bucket_data;\n@@ -487,7 +503,7 @@ rte_table_hash_entry_add_key16_ext(\n \t\tbucket_prev->next_valid = 1;\n \n \t\tbucket->signature[0] = signature;\n-\t\tmemcpy(bucket->key[0], key, f->key_size);\n+\t\tkeycpy(&bucket->key[0], key, f->key_mask);\n \t\tmemcpy(&bucket->data[0], entry, f->entry_size);\n \t\t*key_found = 0;\n \t\t*entry_ptr = (void *) &bucket->data[0];\n@@ -509,7 +525,7 @@ rte_table_hash_entry_delete_key16_ext(\n \tuint64_t signature;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket0 = (struct rte_bucket_4_16 *)\n \t\t&f->memory[bucket_index * f->bucket_size];\n@@ -520,18 +536,17 @@ rte_table_hash_entry_delete_key16_ext(\n \t\tbucket_prev = bucket, bucket = bucket->next)\n \t\tfor (i = 0; i < 4; i++) {\n \t\t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\t\tif ((bucket_signature == signature) &&\n-\t\t\t\t(memcmp(key, bucket_key, f->key_size) == 0)) {\n+\t\t\t\t(keycmp(bucket_key, key, f->key_mask) == 0)) {\n \t\t\t\tuint8_t *bucket_data = &bucket->data[i *\n \t\t\t\t\tf->entry_size];\n \n \t\t\t\tbucket->signature[i] = 0;\n \t\t\t\t*key_found = 1;\n \t\t\t\tif (entry)\n-\t\t\t\t\tmemcpy(entry, bucket_data,\n-\t\t\t\t\tf->entry_size);\n+\t\t\t\t\tmemcpy(entry, bucket_data, f->entry_size);\n \n \t\t\t\tif ((bucket->signature[0] == 0) &&\n \t\t\t\t\t(bucket->signature[1] == 0) &&\n@@ -558,26 +573,28 @@ rte_table_hash_entry_delete_key16_ext(\n \treturn 0;\n }\n \n-#define lookup_key16_cmp(key_in, bucket, pos)\t\t\t\\\n+#define lookup_key16_cmp(key_in, bucket, pos, f)\t\t\t\\\n {\t\t\t\t\t\t\t\t\\\n-\tuint64_t xor[4][2], or[4], signature[4];\t\t\\\n+\tuint64_t xor[4][2], or[4], signature[4], k[2];\t\t\\\n \t\t\t\t\t\t\t\t\\\n+\tk[0] = key_in[0] & f->key_mask[0];\t\t\t\t\\\n+\tk[1] = key_in[1] & f->key_mask[1];\t\t\t\t\\\n \tsignature[0] = (~bucket->signature[0]) & 1;\t\t\\\n \tsignature[1] = (~bucket->signature[1]) & 1;\t\t\\\n \tsignature[2] = (~bucket->signature[2]) & 1;\t\t\\\n \tsignature[3] = (~bucket->signature[3]) & 1;\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\txor[0][0] = key_in[0] ^\t bucket->key[0][0];\t\t\\\n-\txor[0][1] = key_in[1] ^\t bucket->key[0][1];\t\t\\\n+\txor[0][0] = k[0] ^ bucket->key[0][0];\t\t\t\\\n+\txor[0][1] = k[1] ^ bucket->key[0][1];\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\txor[1][0] = key_in[0] ^\t bucket->key[1][0];\t\t\\\n-\txor[1][1] = key_in[1] ^\t bucket->key[1][1];\t\t\\\n+\txor[1][0] = k[0] ^ bucket->key[1][0];\t\t\t\\\n+\txor[1][1] = k[1] ^ bucket->key[1][1];\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\txor[2][0] = key_in[0] ^\t bucket->key[2][0];\t\t\\\n-\txor[2][1] = key_in[1] ^\t bucket->key[2][1];\t\t\\\n+\txor[2][0] = k[0] ^ bucket->key[2][0];\t\t\t\\\n+\txor[2][1] = k[1] ^ bucket->key[2][1];\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\txor[3][0] = key_in[0] ^\t bucket->key[3][0];\t\t\\\n-\txor[3][1] = key_in[1] ^\t bucket->key[3][1];\t\t\\\n+\txor[3][0] = k[0] ^ bucket->key[3][0];\t\t\t\\\n+\txor[3][1] = k[1] ^ bucket->key[3][1];\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tor[0] = xor[0][0] | xor[0][1] | signature[0];\t\t\\\n \tor[1] = xor[1][0] | xor[1][1] | signature[1];\t\t\\\n@@ -610,30 +627,12 @@ rte_table_hash_entry_delete_key16_ext(\n \n #define lookup1_stage1(mbuf1, bucket1, f)\t\t\t\\\n {\t\t\t\t\t\t\t\t\\\n-\tuint64_t signature;\t\t\t\t\t\\\n-\tuint32_t bucket_index;\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tsignature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\\\n-\tbucket_index = signature & (f->n_buckets - 1);\t\t\\\n-\tbucket1 = (struct rte_bucket_4_16 *)\t\t\t\\\n-\t\t&f->memory[bucket_index * f->bucket_size];\t\\\n-\trte_prefetch0(bucket1);\t\t\t\t\t\\\n-\trte_prefetch0((void *)(((uintptr_t) bucket1) + RTE_CACHE_LINE_SIZE));\\\n-}\n-\n-#define lookup1_stage1_dosig(mbuf1, bucket1, f)\t\t\t\\\n-{\t\t\t\t\t\t\t\t\\\n \tuint64_t *key;\t\t\t\t\t\t\\\n \tuint64_t signature = 0;\t\t\t\t\\\n \tuint32_t bucket_index;\t\t\t\t\\\n-\tuint64_t hash_key_buffer[2];\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\\\n-\t\t\t\t\t\t\t\t\\\n-\thash_key_buffer[0] = key[0] & f->key_mask[0];\t\\\n-\thash_key_buffer[1] = key[1] & f->key_mask[1];\t\\\n-\tsignature = f->f_hash(hash_key_buffer,\t\t\t\\\n-\t\t\tRTE_TABLE_HASH_KEY_SIZE, f->seed);\t\t\\\n+\tsignature = f->f_hash(key, f->key_mask, KEY_SIZE, f->seed);\t\\\n \t\t\t\t\t\t\t\t\\\n \tbucket_index = signature & (f->n_buckets - 1);\t\t\\\n \tbucket1 = (struct rte_bucket_4_16 *)\t\t\t\\\n@@ -648,14 +647,10 @@ rte_table_hash_entry_delete_key16_ext(\n \tvoid *a;\t\t\t\t\t\t\\\n \tuint64_t pkt_mask;\t\t\t\t\t\\\n \tuint64_t *key;\t\t\t\t\t\t\\\n-\tuint64_t hash_key_buffer[2];\t\t\\\n \tuint32_t pos;\t\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\\\n-\thash_key_buffer[0] = key[0] & f->key_mask[0];\t\\\n-\thash_key_buffer[1] = key[1] & f->key_mask[1];\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tlookup_key16_cmp(hash_key_buffer, bucket2, pos);\t\\\n+\tlookup_key16_cmp(key, bucket2, pos, f);\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\\\n \tpkts_mask_out |= pkt_mask;\t\t\t\t\\\n@@ -673,14 +668,10 @@ rte_table_hash_entry_delete_key16_ext(\n \tvoid *a;\t\t\t\t\t\t\\\n \tuint64_t pkt_mask, bucket_mask;\t\t\t\t\\\n \tuint64_t *key;\t\t\t\t\t\t\\\n-\tuint64_t hash_key_buffer[2];\t\t\\\n \tuint32_t pos;\t\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\\\n-\thash_key_buffer[0] = key[0] & f->key_mask[0];\t\\\n-\thash_key_buffer[1] = key[1] & f->key_mask[1];\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tlookup_key16_cmp(hash_key_buffer, bucket2, pos);\t\\\n+\tlookup_key16_cmp(key, bucket2, pos, f);\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\\\n \tpkts_mask_out |= pkt_mask;\t\t\t\t\\\n@@ -703,15 +694,11 @@ rte_table_hash_entry_delete_key16_ext(\n \tvoid *a;\t\t\t\t\t\t\\\n \tuint64_t pkt_mask, bucket_mask;\t\t\t\t\\\n \tuint64_t *key;\t\t\t\t\t\t\\\n-\tuint64_t hash_key_buffer[2];\t\t\\\n \tuint32_t pos;\t\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tbucket = buckets[pkt_index];\t\t\t\t\\\n \tkey = keys[pkt_index];\t\t\t\t\t\\\n-\thash_key_buffer[0] = key[0] & f->key_mask[0];\t\\\n-\thash_key_buffer[1] = key[1] & f->key_mask[1];\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tlookup_key16_cmp(hash_key_buffer, bucket, pos);\t\\\n+\tlookup_key16_cmp(key, bucket, pos, f);\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt_mask = (bucket->signature[pos] & 1LLU) << pkt_index;\\\n \tpkts_mask_out |= pkt_mask;\t\t\t\t\\\n@@ -775,36 +762,12 @@ rte_table_hash_entry_delete_key16_ext(\n \n #define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f)\t\\\n {\t\t\t\t\t\t\t\t\\\n-\tuint64_t signature10, signature11;\t\t\t\\\n-\tuint32_t bucket10_index, bucket11_index;\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tsignature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\\\n-\tbucket10_index = signature10 & (f->n_buckets - 1);\t\\\n-\tbucket10 = (struct rte_bucket_4_16 *)\t\t\t\\\n-\t\t&f->memory[bucket10_index * f->bucket_size];\t\\\n-\trte_prefetch0(bucket10);\t\t\t\t\\\n-\trte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\\\n-\t\t\t\t\t\t\t\t\\\n-\tsignature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\\\n-\tbucket11_index = signature11 & (f->n_buckets - 1);\t\\\n-\tbucket11 = (struct rte_bucket_4_16 *)\t\t\t\\\n-\t\t&f->memory[bucket11_index * f->bucket_size];\t\\\n-\trte_prefetch0(bucket11);\t\t\t\t\\\n-\trte_prefetch0((void *)(((uintptr_t) bucket11) + RTE_CACHE_LINE_SIZE));\\\n-}\n-\n-#define lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f)\t\\\n-{\t\t\t\t\t\t\t\t\\\n \tuint64_t *key10, *key11;\t\t\t\t\t\\\n-\tuint64_t hash_offset_buffer[2];\t\t\t\t\\\n \tuint64_t signature10, signature11;\t\t\t\\\n \tuint32_t bucket10_index, bucket11_index;\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, f->key_offset);\\\n-\thash_offset_buffer[0] = key10[0] & f->key_mask[0];\t\\\n-\thash_offset_buffer[1] = key10[1] & f->key_mask[1];\t\\\n-\tsignature10 = f->f_hash(hash_offset_buffer,\t\t\t\\\n-\t\t\tRTE_TABLE_HASH_KEY_SIZE, f->seed);\\\n+\tsignature10 = f->f_hash(key10, f->key_mask,\t KEY_SIZE, f->seed);\\\n \tbucket10_index = signature10 & (f->n_buckets - 1);\t\\\n \tbucket10 = (struct rte_bucket_4_16 *)\t\t\t\t\\\n \t\t&f->memory[bucket10_index * f->bucket_size];\t\\\n@@ -812,10 +775,7 @@ rte_table_hash_entry_delete_key16_ext(\n \trte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\\\n \t\t\t\t\t\t\t\t\\\n \tkey11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, f->key_offset);\\\n-\thash_offset_buffer[0] = key11[0] & f->key_mask[0];\t\\\n-\thash_offset_buffer[1] = key11[1] & f->key_mask[1];\t\\\n-\tsignature11 = f->f_hash(hash_offset_buffer,\t\t\t\\\n-\t\t\tRTE_TABLE_HASH_KEY_SIZE, f->seed);\\\n+\tsignature11 = f->f_hash(key11, f->key_mask,\t KEY_SIZE, f->seed);\\\n \tbucket11_index = signature11 & (f->n_buckets - 1);\t\\\n \tbucket11 = (struct rte_bucket_4_16 *)\t\t\t\\\n \t\t&f->memory[bucket11_index * f->bucket_size];\t\\\n@@ -829,19 +789,13 @@ rte_table_hash_entry_delete_key16_ext(\n \tvoid *a20, *a21;\t\t\t\t\t\\\n \tuint64_t pkt20_mask, pkt21_mask;\t\t\t\\\n \tuint64_t *key20, *key21;\t\t\t\t\\\n-\tuint64_t hash_key_buffer20[2];\t\t\t\\\n-\tuint64_t hash_key_buffer21[2];\t\t\t\\\n \tuint32_t pos20, pos21;\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\\\n \tkey21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\\\n-\thash_key_buffer20[0] = key20[0] & f->key_mask[0];\t\\\n-\thash_key_buffer20[1] = key20[1] & f->key_mask[1];\t\\\n-\thash_key_buffer21[0] = key21[0] & f->key_mask[0];\t\\\n-\thash_key_buffer21[1] = key21[1] & f->key_mask[1];\t\\\n \t\t\t\t\t\t\t\t\\\n-\tlookup_key16_cmp(hash_key_buffer20, bucket20, pos20);\t\\\n-\tlookup_key16_cmp(hash_key_buffer21, bucket21, pos21);\t\\\n+\tlookup_key16_cmp(key20, bucket20, pos20, f);\t\t\\\n+\tlookup_key16_cmp(key21, bucket21, pos21, f);\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\\\n \tpkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\\\n@@ -864,19 +818,13 @@ rte_table_hash_entry_delete_key16_ext(\n \tvoid *a20, *a21;\t\t\t\t\t\\\n \tuint64_t pkt20_mask, pkt21_mask, bucket20_mask, bucket21_mask;\\\n \tuint64_t *key20, *key21;\t\t\t\t\\\n-\tuint64_t hash_key_buffer20[2];\t\t\t\\\n-\tuint64_t hash_key_buffer21[2];\t\t\t\\\n \tuint32_t pos20, pos21;\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\\\n \tkey21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\\\n-\thash_key_buffer20[0] = key20[0] & f->key_mask[0];\t\\\n-\thash_key_buffer20[1] = key20[1] & f->key_mask[1];\t\\\n-\thash_key_buffer21[0] = key21[0] & f->key_mask[0];\t\\\n-\thash_key_buffer21[1] = key21[1] & f->key_mask[1];\t\\\n \t\t\t\t\t\t\t\t\\\n-\tlookup_key16_cmp(hash_key_buffer20, bucket20, pos20);\t\\\n-\tlookup_key16_cmp(hash_key_buffer21, bucket21, pos21);\t\\\n+\tlookup_key16_cmp(key20, bucket20, pos20, f);\t\\\n+\tlookup_key16_cmp(key21, bucket21, pos21, f);\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\\\n \tpkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\\\n@@ -916,6 +864,7 @@ rte_table_hash_lookup_key16_lru(\n \tuint64_t pkts_mask_out = 0;\n \n \t__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);\n+\n \tRTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);\n \n \t/* Cannot run the pipeline with less than 5 packets */\n@@ -932,8 +881,8 @@ rte_table_hash_lookup_key16_lru(\n \t\t}\n \n \t\t*lookup_hit_mask = pkts_mask_out;\n-\t\tRTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f,\n-\t\t\tn_pkts_in - __builtin_popcountll(pkts_mask_out));\n+\t\tRTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -\n+\t\t\t__builtin_popcountll(pkts_mask_out));\n \t\treturn 0;\n \t}\n \n@@ -1026,136 +975,7 @@ rte_table_hash_lookup_key16_lru(\n \tRTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -\n \t\t__builtin_popcountll(pkts_mask_out));\n \treturn 0;\n-} /* rte_table_hash_lookup_key16_lru() */\n-\n-static int\n-rte_table_hash_lookup_key16_lru_dosig(\n-\tvoid *table,\n-\tstruct rte_mbuf **pkts,\n-\tuint64_t pkts_mask,\n-\tuint64_t *lookup_hit_mask,\n-\tvoid **entries)\n-{\n-\tstruct rte_table_hash *f = (struct rte_table_hash *) table;\n-\tstruct rte_bucket_4_16 *bucket10, *bucket11, *bucket20, *bucket21;\n-\tstruct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;\n-\tuint32_t pkt00_index, pkt01_index, pkt10_index;\n-\tuint32_t pkt11_index, pkt20_index, pkt21_index;\n-\tuint64_t pkts_mask_out = 0;\n-\n-\t__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);\n-\n-\tRTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);\n-\n-\t/* Cannot run the pipeline with less than 5 packets */\n-\tif (__builtin_popcountll(pkts_mask) < 5) {\n-\t\tfor ( ; pkts_mask; ) {\n-\t\t\tstruct rte_bucket_4_16 *bucket;\n-\t\t\tstruct rte_mbuf *mbuf;\n-\t\t\tuint32_t pkt_index;\n-\n-\t\t\tlookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);\n-\t\t\tlookup1_stage1_dosig(mbuf, bucket, f);\n-\t\t\tlookup1_stage2_lru(pkt_index, mbuf, bucket,\n-\t\t\t\tpkts_mask_out, entries, f);\n-\t\t}\n-\n-\t\t*lookup_hit_mask = pkts_mask_out;\n-\t\tRTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -\n-\t\t\t__builtin_popcountll(pkts_mask_out));\n-\t\treturn 0;\n-\t}\n-\n-\t/*\n-\t * Pipeline fill\n-\t *\n-\t */\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,\n-\t\tpkts_mask, f);\n-\n-\t/* Pipeline feed */\n-\tmbuf10 = mbuf00;\n-\tmbuf11 = mbuf01;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,\n-\t\tpkts_mask, f);\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t/*\n-\t * Pipeline run\n-\t *\n-\t */\n-\tfor ( ; pkts_mask; ) {\n-\t\t/* Pipeline feed */\n-\t\tbucket20 = bucket10;\n-\t\tbucket21 = bucket11;\n-\t\tmbuf20 = mbuf10;\n-\t\tmbuf21 = mbuf11;\n-\t\tmbuf10 = mbuf00;\n-\t\tmbuf11 = mbuf01;\n-\t\tpkt20_index = pkt10_index;\n-\t\tpkt21_index = pkt11_index;\n-\t\tpkt10_index = pkt00_index;\n-\t\tpkt11_index = pkt01_index;\n-\n-\t\t/* Pipeline stage 0 */\n-\t\tlookup2_stage0_with_odd_support(pkt00_index, pkt01_index,\n-\t\t\tmbuf00, mbuf01, pkts, pkts_mask, f);\n-\n-\t\t/* Pipeline stage 1 */\n-\t\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t\t/* Pipeline stage 2 */\n-\t\tlookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\t\tbucket20, bucket21, pkts_mask_out, entries, f);\n-\t}\n-\n-\t/*\n-\t * Pipeline flush\n-\t *\n-\t */\n-\t/* Pipeline feed */\n-\tbucket20 = bucket10;\n-\tbucket21 = bucket11;\n-\tmbuf20 = mbuf10;\n-\tmbuf21 = mbuf11;\n-\tmbuf10 = mbuf00;\n-\tmbuf11 = mbuf01;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\tbucket20, bucket21, pkts_mask_out, entries, f);\n-\n-\t/* Pipeline feed */\n-\tbucket20 = bucket10;\n-\tbucket21 = bucket11;\n-\tmbuf20 = mbuf10;\n-\tmbuf21 = mbuf11;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\tbucket20, bucket21, pkts_mask_out, entries, f);\n-\n-\t*lookup_hit_mask = pkts_mask_out;\n-\tRTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -\n-\t\t__builtin_popcountll(pkts_mask_out));\n-\treturn 0;\n-} /* rte_table_hash_lookup_key16_lru_dosig() */\n+} /* lookup LRU */\n \n static int\n rte_table_hash_lookup_key16_ext(\n@@ -1175,6 +995,7 @@ rte_table_hash_lookup_key16_ext(\n \tuint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];\n \n \t__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);\n+\n \tRTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);\n \n \t/* Cannot run the pipeline with less than 5 packets */\n@@ -1306,159 +1127,7 @@ rte_table_hash_lookup_key16_ext(\n \tRTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -\n \t\t__builtin_popcountll(pkts_mask_out));\n \treturn 0;\n-} /* rte_table_hash_lookup_key16_ext() */\n-\n-static int\n-rte_table_hash_lookup_key16_ext_dosig(\n-\tvoid *table,\n-\tstruct rte_mbuf **pkts,\n-\tuint64_t pkts_mask,\n-\tuint64_t *lookup_hit_mask,\n-\tvoid **entries)\n-{\n-\tstruct rte_table_hash *f = (struct rte_table_hash *) table;\n-\tstruct rte_bucket_4_16 *bucket10, *bucket11, *bucket20, *bucket21;\n-\tstruct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;\n-\tuint32_t pkt00_index, pkt01_index, pkt10_index;\n-\tuint32_t pkt11_index, pkt20_index, pkt21_index;\n-\tuint64_t pkts_mask_out = 0, buckets_mask = 0;\n-\tstruct rte_bucket_4_16 *buckets[RTE_PORT_IN_BURST_SIZE_MAX];\n-\tuint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];\n-\n-\t__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);\n-\n-\tRTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);\n-\n-\t/* Cannot run the pipeline with less than 5 packets */\n-\tif (__builtin_popcountll(pkts_mask) < 5) {\n-\t\tfor ( ; pkts_mask; ) {\n-\t\t\tstruct rte_bucket_4_16 *bucket;\n-\t\t\tstruct rte_mbuf *mbuf;\n-\t\t\tuint32_t pkt_index;\n-\n-\t\t\tlookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);\n-\t\t\tlookup1_stage1_dosig(mbuf, bucket, f);\n-\t\t\tlookup1_stage2_ext(pkt_index, mbuf, bucket,\n-\t\t\t\tpkts_mask_out, entries, buckets_mask,\n-\t\t\t\tbuckets, keys, f);\n-\t\t}\n-\n-\t\tgoto grind_next_buckets;\n-\t}\n-\n-\t/*\n-\t * Pipeline fill\n-\t *\n-\t */\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,\n-\t\tpkts_mask, f);\n-\n-\t/* Pipeline feed */\n-\tmbuf10 = mbuf00;\n-\tmbuf11 = mbuf01;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,\n-\t\tpkts_mask, f);\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t/*\n-\t * Pipeline run\n-\t *\n-\t */\n-\tfor ( ; pkts_mask; ) {\n-\t\t/* Pipeline feed */\n-\t\tbucket20 = bucket10;\n-\t\tbucket21 = bucket11;\n-\t\tmbuf20 = mbuf10;\n-\t\tmbuf21 = mbuf11;\n-\t\tmbuf10 = mbuf00;\n-\t\tmbuf11 = mbuf01;\n-\t\tpkt20_index = pkt10_index;\n-\t\tpkt21_index = pkt11_index;\n-\t\tpkt10_index = pkt00_index;\n-\t\tpkt11_index = pkt01_index;\n-\n-\t\t/* Pipeline stage 0 */\n-\t\tlookup2_stage0_with_odd_support(pkt00_index, pkt01_index,\n-\t\t\tmbuf00, mbuf01, pkts, pkts_mask, f);\n-\n-\t\t/* Pipeline stage 1 */\n-\t\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t\t/* Pipeline stage 2 */\n-\t\tlookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\t\tbucket20, bucket21, pkts_mask_out, entries,\n-\t\t\tbuckets_mask, buckets, keys, f);\n-\t}\n-\n-\t/*\n-\t * Pipeline flush\n-\t *\n-\t */\n-\t/* Pipeline feed */\n-\tbucket20 = bucket10;\n-\tbucket21 = bucket11;\n-\tmbuf20 = mbuf10;\n-\tmbuf21 = mbuf11;\n-\tmbuf10 = mbuf00;\n-\tmbuf11 = mbuf01;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\tbucket20, bucket21, pkts_mask_out, entries,\n-\t\tbuckets_mask, buckets, keys, f);\n-\n-\t/* Pipeline feed */\n-\tbucket20 = bucket10;\n-\tbucket21 = bucket11;\n-\tmbuf20 = mbuf10;\n-\tmbuf21 = mbuf11;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\tbucket20, bucket21, pkts_mask_out, entries,\n-\t\tbuckets_mask, buckets, keys, f);\n-\n-grind_next_buckets:\n-\t/* Grind next buckets */\n-\tfor ( ; buckets_mask; ) {\n-\t\tuint64_t buckets_mask_next = 0;\n-\n-\t\tfor ( ; buckets_mask; ) {\n-\t\t\tuint64_t pkt_mask;\n-\t\t\tuint32_t pkt_index;\n-\n-\t\t\tpkt_index = __builtin_ctzll(buckets_mask);\n-\t\t\tpkt_mask = 1LLU << pkt_index;\n-\t\t\tbuckets_mask &= ~pkt_mask;\n-\n-\t\t\tlookup_grinder(pkt_index, buckets, keys, pkts_mask_out,\n-\t\t\t\tentries, buckets_mask_next, f);\n-\t\t}\n-\n-\t\tbuckets_mask = buckets_mask_next;\n-\t}\n-\n-\t*lookup_hit_mask = pkts_mask_out;\n-\tRTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -\n-\t\t__builtin_popcountll(pkts_mask_out));\n-\treturn 0;\n-} /* rte_table_hash_lookup_key16_ext_dosig() */\n+} /* lookup EXT */\n \n static int\n rte_table_hash_key16_stats_read(void *table, struct rte_table_stats *stats, int clear)\n@@ -1485,15 +1154,6 @@ struct rte_table_ops rte_table_hash_key16_lru_ops = {\n \t.f_stats = rte_table_hash_key16_stats_read,\n };\n \n-struct rte_table_ops rte_table_hash_key16_lru_dosig_ops = {\n-\t.f_create = rte_table_hash_create_key16_lru,\n-\t.f_free = rte_table_hash_free_key16_lru,\n-\t.f_add = rte_table_hash_entry_add_key16_lru,\n-\t.f_delete = rte_table_hash_entry_delete_key16_lru,\n-\t.f_lookup = rte_table_hash_lookup_key16_lru_dosig,\n-\t.f_stats = rte_table_hash_key16_stats_read,\n-};\n-\n struct rte_table_ops rte_table_hash_key16_ext_ops = {\n \t.f_create = rte_table_hash_create_key16_ext,\n \t.f_free = rte_table_hash_free_key16_ext,\n@@ -1504,12 +1164,3 @@ struct rte_table_ops rte_table_hash_key16_ext_ops = {\n \t.f_lookup = rte_table_hash_lookup_key16_ext,\n \t.f_stats = rte_table_hash_key16_stats_read,\n };\n-\n-struct rte_table_ops rte_table_hash_key16_ext_dosig_ops = {\n-\t.f_create = rte_table_hash_create_key16_ext,\n-\t.f_free = rte_table_hash_free_key16_ext,\n-\t.f_add = rte_table_hash_entry_add_key16_ext,\n-\t.f_delete = rte_table_hash_entry_delete_key16_ext,\n-\t.f_lookup = rte_table_hash_lookup_key16_ext_dosig,\n-\t.f_stats = rte_table_hash_key16_stats_read,\n-};\ndiff --git a/lib/librte_table/rte_table_hash_key32.c b/lib/librte_table/rte_table_hash_key32.c\nindex 31fe6fd..5038669 100644\n--- a/lib/librte_table/rte_table_hash_key32.c\n+++ b/lib/librte_table/rte_table_hash_key32.c\n@@ -1,34 +1,34 @@\n /*-\n- *\t BSD LICENSE\n+ *   BSD LICENSE\n  *\n- *\t Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n- *\t All rights reserved.\n+ *   Copyright(c) 2010-2017 Intel Corporation. All rights reserved.\n+ *   All rights reserved.\n  *\n- *\t Redistribution and use in source and binary forms, with or without\n- *\t modification, are permitted provided that the following conditions\n- *\t are met:\n+ *   Redistribution and use in source and binary forms, with or without\n+ *   modification, are permitted provided that the following conditions\n+ *   are met:\n  *\n- *\t* Redistributions of source code must retain the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer.\n- *\t* Redistributions in binary form must reproduce the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer in\n- *\t\t the documentation and/or other materials provided with the\n- *\t\t distribution.\n- *\t* Neither the name of Intel Corporation nor the names of its\n- *\t\t contributors may be used to endorse or promote products derived\n- *\t\t from this software without specific prior written permission.\n+ *     * Redistributions of source code must retain the above copyright\n+ *       notice, this list of conditions and the following disclaimer.\n+ *     * Redistributions in binary form must reproduce the above copyright\n+ *       notice, this list of conditions and the following disclaimer in\n+ *       the documentation and/or other materials provided with the\n+ *       distribution.\n+ *     * Neither the name of Intel Corporation nor the names of its\n+ *       contributors may be used to endorse or promote products derived\n+ *       from this software without specific prior written permission.\n  *\n- *\t THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n- *\t \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n- *\t LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n- *\t A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n- *\t OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n- *\t SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n- *\t LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n- *\t DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n- *\t THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n- *\t (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n- *\t OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ *   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n  */\n #include <string.h>\n #include <stdio.h>\n@@ -42,7 +42,9 @@\n #include \"rte_table_hash.h\"\n #include \"rte_lru.h\"\n \n-#define RTE_TABLE_HASH_KEY_SIZE\t\t\t\t\t\t32\n+#define KEY_SIZE\t\t\t\t\t\t32\n+\n+#define KEYS_PER_BUCKET\t\t\t\t\t4\n \n #define RTE_BUCKET_ENTRY_VALID\t\t\t\t\t\t0x1LLU\n \n@@ -79,12 +81,11 @@ struct rte_table_hash {\n \n \t/* Input parameters */\n \tuint32_t n_buckets;\n-\tuint32_t n_entries_per_bucket;\n \tuint32_t key_size;\n \tuint32_t entry_size;\n \tuint32_t bucket_size;\n-\tuint32_t signature_offset;\n \tuint32_t key_offset;\n+\tuint64_t key_mask[4];\n \trte_table_hash_op_hash f_hash;\n \tuint64_t seed;\n \n@@ -98,10 +99,44 @@ struct rte_table_hash {\n };\n \n static int\n-check_params_create_lru(struct rte_table_hash_key32_lru_params *params) {\n-\t/* n_entries */\n-\tif (params->n_entries == 0) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: n_entries is zero\\n\", __func__);\n+keycmp(void *a, void *b, void *b_mask)\n+{\n+\tuint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;\n+\n+\treturn (a64[0] != (b64[0] & b_mask64[0])) ||\n+\t\t(a64[1] != (b64[1] & b_mask64[1])) ||\n+\t\t(a64[2] != (b64[2] & b_mask64[2])) ||\n+\t\t(a64[3] != (b64[3] & b_mask64[3]));\n+}\n+\n+static void\n+keycpy(void *dst, void *src, void *src_mask)\n+{\n+\tuint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;\n+\n+\tdst64[0] = src64[0] & src_mask64[0];\n+\tdst64[1] = src64[1] & src_mask64[1];\n+\tdst64[2] = src64[2] & src_mask64[2];\n+\tdst64[3] = src64[3] & src_mask64[3];\n+}\n+\n+static int\n+check_params_create(struct rte_table_hash_params *params) {\n+\t/* name */\n+\tif (params->name == NULL) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: name invalid value\\n\", __func__);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* key_size */\n+\tif (params->key_size != KEY_SIZE) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: key_size invalid value\\n\", __func__);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* n_keys */\n+\tif (params->n_keys == 0) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: n_keys is zero\\n\", __func__);\n \t\treturn -EINVAL;\n \t}\n \n@@ -120,51 +155,61 @@ rte_table_hash_create_key32_lru(void *params,\n \t\tint socket_id,\n \t\tuint32_t entry_size)\n {\n-\tstruct rte_table_hash_key32_lru_params *p =\n-\t\t(struct rte_table_hash_key32_lru_params *) params;\n+\tstruct rte_table_hash_params *p = params;\n \tstruct rte_table_hash *f;\n-\tuint32_t n_buckets, n_entries_per_bucket, key_size, bucket_size_cl;\n-\tuint32_t total_size, i;\n+\tuint32_t n_buckets, bucket_size_cl;\n+\tuint32_t total_size, n_keys, i;\n \n \t/* Check input parameters */\n-\tif ((check_params_create_lru(p) != 0) ||\n+\tif ((check_params_create(p) != 0) ||\n \t\t((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||\n-\t\t((sizeof(struct rte_bucket_4_32) % 64) != 0)) {\n+\t\t((sizeof(struct rte_bucket_4_32) % 64) != 0))\n \t\treturn NULL;\n-\t}\n-\tn_entries_per_bucket = 4;\n-\tkey_size = 32;\n+\n+\tn_keys = p->n_keys;\n+\tif (n_keys < KEYS_PER_BUCKET)\n+\t\tn_keys = KEYS_PER_BUCKET;\n \n \t/* Memory allocation */\n-\tn_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /\n-\t\tn_entries_per_bucket);\n-\tbucket_size_cl = (sizeof(struct rte_bucket_4_32) + n_entries_per_bucket\n-\t\t* entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;\n+\tn_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /\n+\t\tKEYS_PER_BUCKET);\n+\tbucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_32) +\n+\t\tKEYS_PER_BUCKET * entry_size);\n \ttotal_size = sizeof(struct rte_table_hash) + n_buckets *\n \t\tbucket_size_cl * RTE_CACHE_LINE_SIZE;\n \n-\tf = rte_zmalloc_socket(\"TABLE\", total_size, RTE_CACHE_LINE_SIZE, socket_id);\n+\tf = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);\n \tif (f == NULL) {\n \t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\"%s: Cannot allocate %u bytes for hash table\\n\",\n-\t\t\t__func__, total_size);\n+\t\t\t\"%s: Cannot allocate %u bytes for hash table %s\\n\",\n+\t\t\t__func__, total_size, p->name);\n \t\treturn NULL;\n \t}\n \tRTE_LOG(INFO, TABLE,\n-\t\t\"%s: Hash table memory footprint is %u bytes\\n\", __func__,\n-\t\ttotal_size);\n+\t\t\"%s: Hash table %s memory footprint is %u bytes\\n\",\n+\t\t__func__, p->name, total_size);\n \n \t/* Memory initialization */\n \tf->n_buckets = n_buckets;\n-\tf->n_entries_per_bucket = n_entries_per_bucket;\n-\tf->key_size = key_size;\n+\tf->key_size = KEY_SIZE;\n \tf->entry_size = entry_size;\n \tf->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;\n-\tf->signature_offset = p->signature_offset;\n \tf->key_offset = p->key_offset;\n \tf->f_hash = p->f_hash;\n \tf->seed = p->seed;\n \n+\tif (p->key_mask != NULL) {\n+\t\tf->key_mask[0] = ((uint64_t *)p->key_mask)[0];\n+\t\tf->key_mask[1] = ((uint64_t *)p->key_mask)[1];\n+\t\tf->key_mask[2] = ((uint64_t *)p->key_mask)[2];\n+\t\tf->key_mask[3] = ((uint64_t *)p->key_mask)[3];\n+\t} else {\n+\t\tf->key_mask[0] = 0xFFFFFFFFFFFFFFFFLLU;\n+\t\tf->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;\n+\t\tf->key_mask[2] = 0xFFFFFFFFFFFFFFFFLLU;\n+\t\tf->key_mask[3] = 0xFFFFFFFFFFFFFFFFLLU;\n+\t}\n+\n \tfor (i = 0; i < n_buckets; i++) {\n \t\tstruct rte_bucket_4_32 *bucket;\n \n@@ -204,7 +249,7 @@ rte_table_hash_entry_add_key32_lru(\n \tuint64_t signature, pos;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket = (struct rte_bucket_4_32 *)\n \t\t&f->memory[bucket_index * f->bucket_size];\n@@ -213,10 +258,10 @@ rte_table_hash_entry_add_key32_lru(\n \t/* Key is present in the bucket */\n \tfor (i = 0; i < 4; i++) {\n \t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\tif ((bucket_signature == signature) &&\n-\t\t\t(memcmp(key, bucket_key, f->key_size) == 0)) {\n+\t\t\t(keycmp(bucket_key, key, f->key_mask) == 0)) {\n \t\t\tuint8_t *bucket_data = &bucket->data[i * f->entry_size];\n \n \t\t\tmemcpy(bucket_data, entry, f->entry_size);\n@@ -230,13 +275,13 @@ rte_table_hash_entry_add_key32_lru(\n \t/* Key is not present in the bucket */\n \tfor (i = 0; i < 4; i++) {\n \t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\tif (bucket_signature == 0) {\n \t\t\tuint8_t *bucket_data = &bucket->data[i * f->entry_size];\n \n \t\t\tbucket->signature[i] = signature;\n-\t\t\tmemcpy(bucket_key, key, f->key_size);\n+\t\t\tkeycpy(bucket_key, key, f->key_mask);\n \t\t\tmemcpy(bucket_data, entry, f->entry_size);\n \t\t\tlru_update(bucket, i);\n \t\t\t*key_found = 0;\n@@ -249,10 +294,10 @@ rte_table_hash_entry_add_key32_lru(\n \t/* Bucket full: replace LRU entry */\n \tpos = lru_pos(bucket);\n \tbucket->signature[pos] = signature;\n-\tmemcpy(bucket->key[pos], key, f->key_size);\n+\tkeycpy(&bucket->key[pos], key, f->key_mask);\n \tmemcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);\n \tlru_update(bucket, pos);\n-\t*key_found\t= 0;\n+\t*key_found = 0;\n \t*entry_ptr = (void *) &bucket->data[pos * f->entry_size];\n \n \treturn 0;\n@@ -270,7 +315,7 @@ rte_table_hash_entry_delete_key32_lru(\n \tuint64_t signature;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket = (struct rte_bucket_4_32 *)\n \t\t&f->memory[bucket_index * f->bucket_size];\n@@ -279,10 +324,10 @@ rte_table_hash_entry_delete_key32_lru(\n \t/* Key is present in the bucket */\n \tfor (i = 0; i < 4; i++) {\n \t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\tif ((bucket_signature == signature) &&\n-\t\t\t(memcmp(key, bucket_key, f->key_size) == 0)) {\n+\t\t\t(keycmp(bucket_key, key, f->key_mask) == 0)) {\n \t\t\tuint8_t *bucket_data = &bucket->data[i * f->entry_size];\n \n \t\t\tbucket->signature[i] = 0;\n@@ -299,81 +344,69 @@ rte_table_hash_entry_delete_key32_lru(\n \treturn 0;\n }\n \n-static int\n-check_params_create_ext(struct rte_table_hash_key32_ext_params *params) {\n-\t/* n_entries */\n-\tif (params->n_entries == 0) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: n_entries is zero\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\t/* n_entries_ext */\n-\tif (params->n_entries_ext == 0) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: n_entries_ext is zero\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\t/* f_hash */\n-\tif (params->f_hash == NULL) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: f_hash function pointer is NULL\\n\",\n-\t\t\t__func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\treturn 0;\n-}\n-\n static void *\n rte_table_hash_create_key32_ext(void *params,\n \tint socket_id,\n \tuint32_t entry_size)\n {\n-\tstruct rte_table_hash_key32_ext_params *p =\n-\t\t\tparams;\n+\tstruct rte_table_hash_params *p = params;\n \tstruct rte_table_hash *f;\n-\tuint32_t n_buckets, n_buckets_ext, n_entries_per_bucket;\n-\tuint32_t key_size, bucket_size_cl, stack_size_cl, total_size, i;\n+\tuint32_t n_keys, n_buckets, n_buckets_ext;\n+\tuint32_t bucket_size_cl, stack_size_cl, total_size, i;\n \n \t/* Check input parameters */\n-\tif ((check_params_create_ext(p) != 0) ||\n+\tif ((check_params_create(p) != 0) ||\n \t\t((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||\n \t\t((sizeof(struct rte_bucket_4_32) % 64) != 0))\n \t\treturn NULL;\n \n-\tn_entries_per_bucket = 4;\n-\tkey_size = 32;\n+\t/*\n+\t * Table dimensioning\n+\t *\n+\t * Objective: Pick the number of bucket extensions (n_buckets_ext) so that\n+\t * it is guaranteed that n_keys keys can be stored in the table at any time.\n+\t *\n+\t * The worst case scenario takes place when all the n_keys keys fall into\n+\t * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst\n+\t * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the\n+\t * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall\n+\t * into a different bucket. This case defeats the purpose of the hash table.\n+\t * It indicates unsuitable f_hash or n_keys to n_buckets ratio.\n+\t *\n+\t * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1\n+\t */\n+\tn_keys = p->n_keys;\n+\tif (n_keys < KEYS_PER_BUCKET)\n+\t\tn_keys = KEYS_PER_BUCKET;\n+\tn_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;\n \n \t/* Memory allocation */\n-\tn_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /\n-\t\tn_entries_per_bucket);\n-\tn_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /\n-\t\tn_entries_per_bucket;\n-\tbucket_size_cl = (sizeof(struct rte_bucket_4_32) + n_entries_per_bucket\n-\t\t* entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;\n-\tstack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)\n-\t\t/ RTE_CACHE_LINE_SIZE;\n+\tn_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /\n+\t\tKEYS_PER_BUCKET);\n+\tbucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_32) +\n+\t\tKEYS_PER_BUCKET * entry_size);\n+\tstack_size_cl = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));\n+\n \ttotal_size = sizeof(struct rte_table_hash) +\n \t\t((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *\n \t\tRTE_CACHE_LINE_SIZE;\n \n-\tf = rte_zmalloc_socket(\"TABLE\", total_size, RTE_CACHE_LINE_SIZE, socket_id);\n+\tf = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);\n \tif (f == NULL) {\n \t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\"%s: Cannot allocate %u bytes for hash table\\n\",\n-\t\t\t__func__, total_size);\n+\t\t\t\"%s: Cannot allocate %u bytes for hash table %s\\n\",\n+\t\t\t__func__, total_size, p->name);\n \t\treturn NULL;\n \t}\n \tRTE_LOG(INFO, TABLE,\n-\t\t\"%s: Hash table memory footprint is %u bytes\\n\", __func__,\n-\t\ttotal_size);\n+\t\t\"%s: Hash table %s memory footprint is %u bytes\\n\",\n+\t\t__func__, p->name, total_size);\n \n \t/* Memory initialization */\n \tf->n_buckets = n_buckets;\n-\tf->n_entries_per_bucket = n_entries_per_bucket;\n-\tf->key_size = key_size;\n+\tf->key_size = KEY_SIZE;\n \tf->entry_size = entry_size;\n \tf->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;\n-\tf->signature_offset = p->signature_offset;\n \tf->key_offset = p->key_offset;\n \tf->f_hash = p->f_hash;\n \tf->seed = p->seed;\n@@ -383,6 +416,18 @@ rte_table_hash_create_key32_ext(void *params,\n \tf->stack = (uint32_t *)\n \t\t&f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];\n \n+\tif (p->key_mask != NULL) {\n+\t\tf->key_mask[0] = (((uint64_t *)p->key_mask)[0]);\n+\t\tf->key_mask[1] = (((uint64_t *)p->key_mask)[1]);\n+\t\tf->key_mask[2] = (((uint64_t *)p->key_mask)[2]);\n+\t\tf->key_mask[3] = (((uint64_t *)p->key_mask)[3]);\n+\t} else {\n+\t\tf->key_mask[0] = 0xFFFFFFFFFFFFFFFFLLU;\n+\t\tf->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;\n+\t\tf->key_mask[2] = 0xFFFFFFFFFFFFFFFFLLU;\n+\t\tf->key_mask[3] = 0xFFFFFFFFFFFFFFFFLLU;\n+\t}\n+\n \tfor (i = 0; i < n_buckets_ext; i++)\n \t\tf->stack[i] = i;\n \n@@ -417,7 +462,7 @@ rte_table_hash_entry_add_key32_ext(\n \tuint64_t signature;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket0 = (struct rte_bucket_4_32 *)\n \t\t\t&f->memory[bucket_index * f->bucket_size];\n@@ -427,10 +472,10 @@ rte_table_hash_entry_add_key32_ext(\n \tfor (bucket = bucket0; bucket != NULL; bucket = bucket->next) {\n \t\tfor (i = 0; i < 4; i++) {\n \t\t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\t\tif ((bucket_signature == signature) &&\n-\t\t\t\t(memcmp(key, bucket_key, f->key_size) == 0)) {\n+\t\t\t\t(keycmp(bucket_key, key, f->key_mask) == 0)) {\n \t\t\t\tuint8_t *bucket_data = &bucket->data[i *\n \t\t\t\t\tf->entry_size];\n \n@@ -448,14 +493,14 @@ rte_table_hash_entry_add_key32_ext(\n \t\tbucket_prev = bucket, bucket = bucket->next)\n \t\tfor (i = 0; i < 4; i++) {\n \t\t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\t\tif (bucket_signature == 0) {\n \t\t\t\tuint8_t *bucket_data = &bucket->data[i *\n \t\t\t\t\tf->entry_size];\n \n \t\t\t\tbucket->signature[i] = signature;\n-\t\t\t\tmemcpy(bucket_key, key, f->key_size);\n+\t\t\t\tkeycpy(bucket_key, key, f->key_mask);\n \t\t\t\tmemcpy(bucket_data, entry, f->entry_size);\n \t\t\t\t*key_found = 0;\n \t\t\t\t*entry_ptr = (void *) bucket_data;\n@@ -475,7 +520,7 @@ rte_table_hash_entry_add_key32_ext(\n \t\tbucket_prev->next_valid = 1;\n \n \t\tbucket->signature[0] = signature;\n-\t\tmemcpy(bucket->key[0], key, f->key_size);\n+\t\tkeycpy(&bucket->key[0], key, f->key_mask);\n \t\tmemcpy(&bucket->data[0], entry, f->entry_size);\n \t\t*key_found = 0;\n \t\t*entry_ptr = (void *) &bucket->data[0];\n@@ -497,7 +542,7 @@ rte_table_hash_entry_delete_key32_ext(\n \tuint64_t signature;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket0 = (struct rte_bucket_4_32 *)\n \t\t&f->memory[bucket_index * f->bucket_size];\n@@ -508,24 +553,23 @@ rte_table_hash_entry_delete_key32_ext(\n \t\tbucket_prev = bucket, bucket = bucket->next)\n \t\tfor (i = 0; i < 4; i++) {\n \t\t\tuint64_t bucket_signature = bucket->signature[i];\n-\t\t\tuint8_t *bucket_key = (uint8_t *) bucket->key[i];\n+\t\t\tuint8_t *bucket_key = (uint8_t *) &bucket->key[i];\n \n \t\t\tif ((bucket_signature == signature) &&\n-\t\t\t\t(memcmp(key, bucket_key, f->key_size) == 0)) {\n+\t\t\t\t(keycmp(bucket_key, key, f->key_mask) == 0)) {\n \t\t\t\tuint8_t *bucket_data = &bucket->data[i *\n \t\t\t\t\tf->entry_size];\n \n \t\t\t\tbucket->signature[i] = 0;\n \t\t\t\t*key_found = 1;\n \t\t\t\tif (entry)\n-\t\t\t\t\tmemcpy(entry, bucket_data,\n-\t\t\t\t\t\tf->entry_size);\n+\t\t\t\t\tmemcpy(entry, bucket_data, f->entry_size);\n \n \t\t\t\tif ((bucket->signature[0] == 0) &&\n-\t\t\t\t\t\t(bucket->signature[1] == 0) &&\n-\t\t\t\t\t\t(bucket->signature[2] == 0) &&\n-\t\t\t\t\t\t(bucket->signature[3] == 0) &&\n-\t\t\t\t\t\t(bucket_prev != NULL)) {\n+\t\t\t\t\t(bucket->signature[1] == 0) &&\n+\t\t\t\t\t(bucket->signature[2] == 0) &&\n+\t\t\t\t\t(bucket->signature[3] == 0) &&\n+\t\t\t\t\t(bucket_prev != NULL)) {\n \t\t\t\t\tbucket_prev->next = bucket->next;\n \t\t\t\t\tbucket_prev->next_valid =\n \t\t\t\t\t\tbucket->next_valid;\n@@ -546,34 +590,39 @@ rte_table_hash_entry_delete_key32_ext(\n \treturn 0;\n }\n \n-#define lookup_key32_cmp(key_in, bucket, pos)\t\t\t\\\n+#define lookup_key32_cmp(key_in, bucket, pos, f)\t\t\t\\\n {\t\t\t\t\t\t\t\t\\\n-\tuint64_t xor[4][4], or[4], signature[4];\t\t\\\n+\tuint64_t xor[4][4], or[4], signature[4], k[4];\t\t\\\n+\t\t\t\t\t\t\t\t\\\n+\tk[0] = key_in[0] & f->key_mask[0];\t\t\t\t\\\n+\tk[1] = key_in[1] & f->key_mask[1];\t\t\t\t\\\n+\tk[2] = key_in[2] & f->key_mask[2];\t\t\t\t\\\n+\tk[3] = key_in[3] & f->key_mask[3];\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tsignature[0] = ((~bucket->signature[0]) & 1);\t\t\\\n \tsignature[1] = ((~bucket->signature[1]) & 1);\t\t\\\n \tsignature[2] = ((~bucket->signature[2]) & 1);\t\t\\\n \tsignature[3] = ((~bucket->signature[3]) & 1);\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\txor[0][0] = key_in[0] ^\t bucket->key[0][0];\t\t\\\n-\txor[0][1] = key_in[1] ^\t bucket->key[0][1];\t\t\\\n-\txor[0][2] = key_in[2] ^\t bucket->key[0][2];\t\t\\\n-\txor[0][3] = key_in[3] ^\t bucket->key[0][3];\t\t\\\n+\txor[0][0] = k[0] ^ bucket->key[0][0];\t\t\t\\\n+\txor[0][1] = k[1] ^ bucket->key[0][1];\t\t\t\\\n+\txor[0][2] = k[2] ^ bucket->key[0][2];\t\t\t\\\n+\txor[0][3] = k[3] ^ bucket->key[0][3];\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\txor[1][0] = key_in[0] ^\t bucket->key[1][0];\t\t\\\n-\txor[1][1] = key_in[1] ^\t bucket->key[1][1];\t\t\\\n-\txor[1][2] = key_in[2] ^\t bucket->key[1][2];\t\t\\\n-\txor[1][3] = key_in[3] ^\t bucket->key[1][3];\t\t\\\n+\txor[1][0] = k[0] ^ bucket->key[1][0];\t\t\t\\\n+\txor[1][1] = k[1] ^ bucket->key[1][1];\t\t\t\\\n+\txor[1][2] = k[2] ^ bucket->key[1][2];\t\t\t\\\n+\txor[1][3] = k[3] ^ bucket->key[1][3];\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\txor[2][0] = key_in[0] ^\t bucket->key[2][0];\t\t\\\n-\txor[2][1] = key_in[1] ^\t bucket->key[2][1];\t\t\\\n-\txor[2][2] = key_in[2] ^\t bucket->key[2][2];\t\t\\\n-\txor[2][3] = key_in[3] ^\t bucket->key[2][3];\t\t\\\n+\txor[2][0] = k[0] ^ bucket->key[2][0];\t\t\t\\\n+\txor[2][1] = k[1] ^ bucket->key[2][1];\t\t\t\\\n+\txor[2][2] = k[2] ^ bucket->key[2][2];\t\t\t\\\n+\txor[2][3] = k[3] ^ bucket->key[2][3];\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\txor[3][0] = key_in[0] ^\t bucket->key[3][0];\t\t\\\n-\txor[3][1] = key_in[1] ^\t bucket->key[3][1];\t\t\\\n-\txor[3][2] = key_in[2] ^\t bucket->key[3][2];\t\t\\\n-\txor[3][3] = key_in[3] ^\t bucket->key[3][3];\t\t\\\n+\txor[3][0] = k[0] ^ bucket->key[3][0];\t\t\t\\\n+\txor[3][1] = k[1] ^ bucket->key[3][1];\t\t\t\\\n+\txor[3][2] = k[2] ^ bucket->key[3][2];\t\t\t\\\n+\txor[3][3] = k[3] ^ bucket->key[3][3];\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tor[0] = xor[0][0] | xor[0][1] | xor[0][2] | xor[0][3] | signature[0];\\\n \tor[1] = xor[1][0] | xor[1][1] | xor[1][2] | xor[1][3] | signature[1];\\\n@@ -604,12 +653,15 @@ rte_table_hash_entry_delete_key32_ext(\n \trte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf0, key_offset));\\\n }\n \n-#define lookup1_stage1(mbuf1, bucket1, f)\t\t\t\\\n+#define lookup1_stage1(mbuf1, bucket1, f)\t\t\t\t\\\n {\t\t\t\t\t\t\t\t\\\n+\tuint64_t *key;\t\t\t\t\t\t\\\n \tuint64_t signature;\t\t\t\t\t\\\n \tuint32_t bucket_index;\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\tsignature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\\\n+\tkey = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\t\\\n+\tsignature = f->f_hash(key, f->key_mask, KEY_SIZE, f->seed);\t\\\n+\t\t\t\t\t\t\t\t\\\n \tbucket_index = signature & (f->n_buckets - 1);\t\t\\\n \tbucket1 = (struct rte_bucket_4_32 *)\t\t\t\\\n \t\t&f->memory[bucket_index * f->bucket_size];\t\\\n@@ -627,8 +679,7 @@ rte_table_hash_entry_delete_key32_ext(\n \tuint32_t pos;\t\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\\\n-\t\t\t\t\t\t\t\t\\\n-\tlookup_key32_cmp(key, bucket2, pos);\t\t\t\\\n+\tlookup_key32_cmp(key, bucket2, pos, f);\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\\\n \tpkts_mask_out |= pkt_mask;\t\t\t\t\\\n@@ -649,8 +700,7 @@ rte_table_hash_entry_delete_key32_ext(\n \tuint32_t pos;\t\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\\\n-\t\t\t\t\t\t\t\t\\\n-\tlookup_key32_cmp(key, bucket2, pos);\t\t\t\\\n+\tlookup_key32_cmp(key, bucket2, pos, f);\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\\\n \tpkts_mask_out |= pkt_mask;\t\t\t\t\\\n@@ -678,7 +728,7 @@ rte_table_hash_entry_delete_key32_ext(\n \tbucket = buckets[pkt_index];\t\t\t\t\\\n \tkey = keys[pkt_index];\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\tlookup_key32_cmp(key, bucket, pos);\t\t\t\\\n+\tlookup_key32_cmp(key, bucket, pos, f);\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt_mask = (bucket->signature[pos] & 1LLU) << pkt_index;\\\n \tpkts_mask_out |= pkt_mask;\t\t\t\t\\\n@@ -745,22 +795,27 @@ rte_table_hash_entry_delete_key32_ext(\n \n #define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f)\t\\\n {\t\t\t\t\t\t\t\t\\\n-\tuint64_t signature10, signature11;\t\t\t\\\n-\tuint32_t bucket10_index, bucket11_index;\t\t\\\n+\tuint64_t *key10, *key11;\t\t\t\t\t\\\n+\tuint64_t signature10, signature11;\t\t\t\t\\\n+\tuint32_t bucket10_index, bucket11_index;\t\t\t\\\n+\t\t\t\t\t\t\t\t\\\n+\tkey10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, f->key_offset);\t\\\n+\tsignature10 = f->f_hash(key10, f->key_mask,\t KEY_SIZE, f->seed); \\\n \t\t\t\t\t\t\t\t\\\n-\tsignature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\\\n-\tbucket10_index = signature10 & (f->n_buckets - 1);\t\\\n+\tbucket10_index = signature10 & (f->n_buckets - 1);\t\t\\\n \tbucket10 = (struct rte_bucket_4_32 *)\t\t\t\\\n \t\t&f->memory[bucket10_index * f->bucket_size];\t\\\n-\trte_prefetch0(bucket10);\t\t\t\t\\\n+\trte_prefetch0(bucket10);\t\t\t\t\t\\\n \trte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\\\n \trte_prefetch0((void *)(((uintptr_t) bucket10) + 2 * RTE_CACHE_LINE_SIZE));\\\n \t\t\t\t\t\t\t\t\\\n-\tsignature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\\\n-\tbucket11_index = signature11 & (f->n_buckets - 1);\t\\\n+\tkey11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, f->key_offset);\t\\\n+\tsignature11 = f->f_hash(key11, f->key_mask, KEY_SIZE, f->seed);\\\n+\t\t\t\t\t\t\t\t\\\n+\tbucket11_index = signature11 & (f->n_buckets - 1);\t\t\\\n \tbucket11 = (struct rte_bucket_4_32 *)\t\t\t\\\n \t\t&f->memory[bucket11_index * f->bucket_size];\t\\\n-\trte_prefetch0(bucket11);\t\t\t\t\\\n+\trte_prefetch0(bucket11);\t\t\t\t\t\\\n \trte_prefetch0((void *)(((uintptr_t) bucket11) + RTE_CACHE_LINE_SIZE));\\\n \trte_prefetch0((void *)(((uintptr_t) bucket11) + 2 * RTE_CACHE_LINE_SIZE));\\\n }\n@@ -776,8 +831,8 @@ rte_table_hash_entry_delete_key32_ext(\n \tkey20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\\\n \tkey21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\\\n \t\t\t\t\t\t\t\t\\\n-\tlookup_key32_cmp(key20, bucket20, pos20);\t\t\\\n-\tlookup_key32_cmp(key21, bucket21, pos21);\t\t\\\n+\tlookup_key32_cmp(key20, bucket20, pos20, f);\t\t\\\n+\tlookup_key32_cmp(key21, bucket21, pos21, f);\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\\\n \tpkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\\\n@@ -805,8 +860,8 @@ rte_table_hash_entry_delete_key32_ext(\n \tkey20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\\\n \tkey21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\\\n \t\t\t\t\t\t\t\t\\\n-\tlookup_key32_cmp(key20, bucket20, pos20);\t\t\\\n-\tlookup_key32_cmp(key21, bucket21, pos21);\t\t\\\n+\tlookup_key32_cmp(key20, bucket20, pos20, f);\t\t\\\n+\tlookup_key32_cmp(key21, bucket21, pos21, f);\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\\\n \tpkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\\\ndiff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c\nindex 5f0c656..39d0458 100644\n--- a/lib/librte_table/rte_table_hash_key8.c\n+++ b/lib/librte_table/rte_table_hash_key8.c\n@@ -1,34 +1,34 @@\n /*-\n- *\t BSD LICENSE\n+ *   BSD LICENSE\n  *\n- *\t Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n- *\t All rights reserved.\n+ *   Copyright(c) 2010-2017 Intel Corporation. All rights reserved.\n+ *   All rights reserved.\n  *\n- *\t Redistribution and use in source and binary forms, with or without\n- *\t modification, are permitted provided that the following conditions\n- *\t are met:\n+ *   Redistribution and use in source and binary forms, with or without\n+ *   modification, are permitted provided that the following conditions\n+ *   are met:\n  *\n- *\t* Redistributions of source code must retain the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer.\n- *\t* Redistributions in binary form must reproduce the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer in\n- *\t\t the documentation and/or other materials provided with the\n- *\t\t distribution.\n- *\t* Neither the name of Intel Corporation nor the names of its\n- *\t\t contributors may be used to endorse or promote products derived\n- *\t\t from this software without specific prior written permission.\n+ *     * Redistributions of source code must retain the above copyright\n+ *       notice, this list of conditions and the following disclaimer.\n+ *     * Redistributions in binary form must reproduce the above copyright\n+ *       notice, this list of conditions and the following disclaimer in\n+ *       the documentation and/or other materials provided with the\n+ *       distribution.\n+ *     * Neither the name of Intel Corporation nor the names of its\n+ *       contributors may be used to endorse or promote products derived\n+ *       from this software without specific prior written permission.\n  *\n- *\t THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n- *\t \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n- *\t LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n- *\t A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n- *\t OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n- *\t SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n- *\t LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n- *\t DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n- *\t THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n- *\t (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n- *\t OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ *   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n  */\n #include <string.h>\n #include <stdio.h>\n@@ -42,7 +42,9 @@\n #include \"rte_table_hash.h\"\n #include \"rte_lru.h\"\n \n-#define RTE_TABLE_HASH_KEY_SIZE\t\t\t\t\t\t8\n+#define KEY_SIZE\t\t\t\t\t\t8\n+\n+#define KEYS_PER_BUCKET\t\t\t\t\t4\n \n #ifdef RTE_TABLE_STATS_COLLECT\n \n@@ -76,11 +78,9 @@ struct rte_table_hash {\n \n \t/* Input parameters */\n \tuint32_t n_buckets;\n-\tuint32_t n_entries_per_bucket;\n \tuint32_t key_size;\n \tuint32_t entry_size;\n \tuint32_t bucket_size;\n-\tuint32_t signature_offset;\n \tuint32_t key_offset;\n \tuint64_t key_mask;\n \trte_table_hash_op_hash f_hash;\n@@ -96,10 +96,38 @@ struct rte_table_hash {\n };\n \n static int\n-check_params_create_lru(struct rte_table_hash_key8_lru_params *params) {\n-\t/* n_entries */\n-\tif (params->n_entries == 0) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: n_entries is zero\\n\", __func__);\n+keycmp(void *a, void *b, void *b_mask)\n+{\n+\tuint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;\n+\n+\treturn a64[0] != (b64[0] & b_mask64[0]);\n+}\n+\n+static void\n+keycpy(void *dst, void *src, void *src_mask)\n+{\n+\tuint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;\n+\n+\tdst64[0] = src64[0] & src_mask64[0];\n+}\n+\n+static int\n+check_params_create(struct rte_table_hash_params *params) {\n+\t/* name */\n+\tif (params->name == NULL) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: name invalid value\\n\", __func__);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* key_size */\n+\tif (params->key_size != KEY_SIZE) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: key_size invalid value\\n\", __func__);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* n_keys */\n+\tif (params->n_keys == 0) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: n_keys is zero\\n\", __func__);\n \t\treturn -EINVAL;\n \t}\n \n@@ -116,47 +144,45 @@ check_params_create_lru(struct rte_table_hash_key8_lru_params *params) {\n static void *\n rte_table_hash_create_key8_lru(void *params, int socket_id, uint32_t entry_size)\n {\n-\tstruct rte_table_hash_key8_lru_params *p =\n-\t\t(struct rte_table_hash_key8_lru_params *) params;\n+\tstruct rte_table_hash_params *p = params;\n \tstruct rte_table_hash *f;\n-\tuint32_t n_buckets, n_entries_per_bucket, key_size, bucket_size_cl;\n-\tuint32_t total_size, i;\n+\tuint32_t n_buckets, bucket_size_cl;\n+\tuint32_t total_size, n_keys, i;\n \n \t/* Check input parameters */\n-\tif ((check_params_create_lru(p) != 0) ||\n+\tif ((check_params_create(p) != 0) ||\n \t\t((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||\n-\t\t((sizeof(struct rte_bucket_4_8) % 64) != 0)) {\n+\t\t((sizeof(struct rte_bucket_4_8) % 64) != 0))\n \t\treturn NULL;\n-\t}\n-\tn_entries_per_bucket = 4;\n-\tkey_size = 8;\n+\n+\tn_keys = p->n_keys;\n+\tif (n_keys < KEYS_PER_BUCKET)\n+\t\tn_keys = KEYS_PER_BUCKET;\n \n \t/* Memory allocation */\n-\tn_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /\n-\t\tn_entries_per_bucket);\n-\tbucket_size_cl = (sizeof(struct rte_bucket_4_8) + n_entries_per_bucket *\n-\t\tentry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;\n+\tn_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /\n+\t\tKEYS_PER_BUCKET);\n+\tbucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_8) +\n+\t\tKEYS_PER_BUCKET * entry_size);\n \ttotal_size = sizeof(struct rte_table_hash) + n_buckets *\n \t\tbucket_size_cl * RTE_CACHE_LINE_SIZE;\n \n-\tf = rte_zmalloc_socket(\"TABLE\", total_size, RTE_CACHE_LINE_SIZE, socket_id);\n+\tf = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);\n \tif (f == NULL) {\n \t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\"%s: Cannot allocate %u bytes for hash table\\n\",\n-\t\t\t__func__, total_size);\n+\t\t\t\"%s: Cannot allocate %u bytes for hash table %s\\n\",\n+\t\t\t__func__, total_size, p->name);\n \t\treturn NULL;\n \t}\n \tRTE_LOG(INFO, TABLE,\n-\t\t\"%s: Hash table memory footprint is %u bytes\\n\",\n-\t\t__func__, total_size);\n+\t\t\"%s: Hash table %s memory footprint is %u bytes\\n\",\n+\t\t__func__, p->name, total_size);\n \n \t/* Memory initialization */\n \tf->n_buckets = n_buckets;\n-\tf->n_entries_per_bucket = n_entries_per_bucket;\n-\tf->key_size = key_size;\n+\tf->key_size = KEY_SIZE;\n \tf->entry_size = entry_size;\n \tf->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;\n-\tf->signature_offset = p->signature_offset;\n \tf->key_offset = p->key_offset;\n \tf->f_hash = p->f_hash;\n \tf->seed = p->seed;\n@@ -205,7 +231,7 @@ rte_table_hash_entry_add_key8_lru(\n \tuint64_t signature, mask, pos;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket = (struct rte_bucket_4_8 *)\n \t\t&f->memory[bucket_index * f->bucket_size];\n@@ -213,10 +239,10 @@ rte_table_hash_entry_add_key8_lru(\n \t/* Key is present in the bucket */\n \tfor (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {\n \t\tuint64_t bucket_signature = bucket->signature;\n-\t\tuint64_t bucket_key = bucket->key[i];\n+\t\tuint64_t *bucket_key = &bucket->key[i];\n \n \t\tif ((bucket_signature & mask) &&\n-\t\t    (*((uint64_t *) key) == bucket_key)) {\n+\t\t\t(keycmp(bucket_key, key, &f->key_mask) == 0)) {\n \t\t\tuint8_t *bucket_data = &bucket->data[i * f->entry_size];\n \n \t\t\tmemcpy(bucket_data, entry, f->entry_size);\n@@ -235,7 +261,7 @@ rte_table_hash_entry_add_key8_lru(\n \t\t\tuint8_t *bucket_data = &bucket->data[i * f->entry_size];\n \n \t\t\tbucket->signature |= mask;\n-\t\t\tbucket->key[i] = *((uint64_t *) key);\n+\t\t\tkeycpy(&bucket->key[i], key, &f->key_mask);\n \t\t\tmemcpy(bucket_data, entry, f->entry_size);\n \t\t\tlru_update(bucket, i);\n \t\t\t*key_found = 0;\n@@ -247,10 +273,10 @@ rte_table_hash_entry_add_key8_lru(\n \n \t/* Bucket full: replace LRU entry */\n \tpos = lru_pos(bucket);\n-\tbucket->key[pos] = *((uint64_t *) key);\n+\tkeycpy(&bucket->key[pos], key, &f->key_mask);\n \tmemcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);\n \tlru_update(bucket, pos);\n-\t*key_found\t= 0;\n+\t*key_found = 0;\n \t*entry_ptr = (void *) &bucket->data[pos * f->entry_size];\n \n \treturn 0;\n@@ -268,7 +294,7 @@ rte_table_hash_entry_delete_key8_lru(\n \tuint64_t signature, mask;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket = (struct rte_bucket_4_8 *)\n \t\t&f->memory[bucket_index * f->bucket_size];\n@@ -276,10 +302,10 @@ rte_table_hash_entry_delete_key8_lru(\n \t/* Key is present in the bucket */\n \tfor (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {\n \t\tuint64_t bucket_signature = bucket->signature;\n-\t\tuint64_t bucket_key = bucket->key[i];\n+\t\tuint64_t *bucket_key = &bucket->key[i];\n \n \t\tif ((bucket_signature & mask) &&\n-\t\t    (*((uint64_t *) key) == bucket_key)) {\n+\t\t\t(keycmp(bucket_key, key, &f->key_mask) == 0)) {\n \t\t\tuint8_t *bucket_data = &bucket->data[i * f->entry_size];\n \n \t\t\tbucket->signature &= ~mask;\n@@ -296,79 +322,66 @@ rte_table_hash_entry_delete_key8_lru(\n \treturn 0;\n }\n \n-static int\n-check_params_create_ext(struct rte_table_hash_key8_ext_params *params) {\n-\t/* n_entries */\n-\tif (params->n_entries == 0) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: n_entries is zero\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\t/* n_entries_ext */\n-\tif (params->n_entries_ext == 0) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: n_entries_ext is zero\\n\", __func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\t/* f_hash */\n-\tif (params->f_hash == NULL) {\n-\t\tRTE_LOG(ERR, TABLE, \"%s: f_hash function pointer is NULL\\n\",\n-\t\t\t__func__);\n-\t\treturn -EINVAL;\n-\t}\n-\n-\treturn 0;\n-}\n-\n static void *\n rte_table_hash_create_key8_ext(void *params, int socket_id, uint32_t entry_size)\n {\n-\tstruct rte_table_hash_key8_ext_params *p =\n-\t\t(struct rte_table_hash_key8_ext_params *) params;\n+\tstruct rte_table_hash_params *p = params;\n \tstruct rte_table_hash *f;\n-\tuint32_t n_buckets, n_buckets_ext, n_entries_per_bucket, key_size;\n+\tuint32_t n_keys, n_buckets, n_buckets_ext;\n \tuint32_t bucket_size_cl, stack_size_cl, total_size, i;\n \n \t/* Check input parameters */\n-\tif ((check_params_create_ext(p) != 0) ||\n+\tif ((check_params_create(p) != 0) ||\n \t\t((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||\n \t\t((sizeof(struct rte_bucket_4_8) % 64) != 0))\n \t\treturn NULL;\n \n-\tn_entries_per_bucket = 4;\n-\tkey_size = 8;\n+\t/*\n+\t * Table dimensioning\n+\t *\n+\t * Objective: Pick the number of bucket extensions (n_buckets_ext) so that\n+\t * it is guaranteed that n_keys keys can be stored in the table at any time.\n+\t *\n+\t * The worst case scenario takes place when all the n_keys keys fall into\n+\t * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst\n+\t * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the\n+\t * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall\n+\t * into a different bucket. This case defeats the purpose of the hash table.\n+\t * It indicates unsuitable f_hash or n_keys to n_buckets ratio.\n+\t *\n+\t * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1\n+\t */\n+\tn_keys = p->n_keys;\n+\tif (n_keys < KEYS_PER_BUCKET)\n+\t\tn_keys = KEYS_PER_BUCKET;\n+\tn_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;\n \n \t/* Memory allocation */\n-\tn_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /\n-\t\tn_entries_per_bucket);\n-\tn_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /\n-\t\tn_entries_per_bucket;\n-\tbucket_size_cl = (sizeof(struct rte_bucket_4_8) + n_entries_per_bucket *\n-\t\tentry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;\n-\tstack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)\n-\t\t/ RTE_CACHE_LINE_SIZE;\n-\ttotal_size = sizeof(struct rte_table_hash) + ((n_buckets +\n-\t\tn_buckets_ext) * bucket_size_cl + stack_size_cl) *\n+\tn_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /\n+\t\tKEYS_PER_BUCKET);\n+\tbucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_8) +\n+\t\tKEYS_PER_BUCKET * entry_size);\n+\tstack_size_cl = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));\n+\ttotal_size = sizeof(struct rte_table_hash) +\n+\t\t((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *\n \t\tRTE_CACHE_LINE_SIZE;\n \n-\tf = rte_zmalloc_socket(\"TABLE\", total_size, RTE_CACHE_LINE_SIZE, socket_id);\n+\tf = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);\n \tif (f == NULL) {\n \t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\"%s: Cannot allocate %u bytes for hash table\\n\",\n-\t\t\t__func__, total_size);\n+\t\t\t\"%s: Cannot allocate %u bytes for hash table %s\\n\",\n+\t\t\t__func__, total_size, p->name);\n \t\treturn NULL;\n \t}\n \tRTE_LOG(INFO, TABLE,\n-\t\t\"%s: Hash table memory footprint is %u bytes\\n\",\n-\t\t__func__, total_size);\n+\t\t\"%s: Hash table %s memory footprint is %u bytes\\n\",\n+\t\t__func__, p->name, total_size);\n \n \t/* Memory initialization */\n \tf->n_buckets = n_buckets;\n-\tf->n_entries_per_bucket = n_entries_per_bucket;\n-\tf->key_size = key_size;\n+\tf->key_size = KEY_SIZE;\n \tf->entry_size = entry_size;\n \tf->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;\n-\tf->signature_offset = p->signature_offset;\n \tf->key_offset = p->key_offset;\n \tf->f_hash = p->f_hash;\n \tf->seed = p->seed;\n@@ -417,7 +430,7 @@ rte_table_hash_entry_add_key8_ext(\n \tuint64_t signature;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket0 = (struct rte_bucket_4_8 *)\n \t\t&f->memory[bucket_index * f->bucket_size];\n@@ -428,10 +441,10 @@ rte_table_hash_entry_add_key8_ext(\n \n \t\tfor (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {\n \t\t\tuint64_t bucket_signature = bucket->signature;\n-\t\t\tuint64_t bucket_key = bucket->key[i];\n+\t\t\tuint64_t *bucket_key = &bucket->key[i];\n \n \t\t\tif ((bucket_signature & mask) &&\n-\t\t\t\t\t(*((uint64_t *) key) == bucket_key)) {\n+\t\t\t\t(keycmp(bucket_key, key, &f->key_mask) == 0)) {\n \t\t\t\tuint8_t *bucket_data = &bucket->data[i *\n \t\t\t\t\tf->entry_size];\n \n@@ -456,7 +469,7 @@ rte_table_hash_entry_add_key8_ext(\n \t\t\t\t\tf->entry_size];\n \n \t\t\t\tbucket->signature |= mask;\n-\t\t\t\tbucket->key[i] = *((uint64_t *) key);\n+\t\t\t\tkeycpy(&bucket->key[i], key, &f->key_mask);\n \t\t\t\tmemcpy(bucket_data, entry, f->entry_size);\n \t\t\t\t*key_found = 0;\n \t\t\t\t*entry_ptr = (void *) bucket_data;\n@@ -476,7 +489,7 @@ rte_table_hash_entry_add_key8_ext(\n \t\tbucket_prev->next_valid = 1;\n \n \t\tbucket->signature = 1;\n-\t\tbucket->key[0] = *((uint64_t *) key);\n+\t\tkeycpy(&bucket->key[0], key, &f->key_mask);\n \t\tmemcpy(&bucket->data[0], entry, f->entry_size);\n \t\t*key_found = 0;\n \t\t*entry_ptr = (void *) &bucket->data[0];\n@@ -498,7 +511,7 @@ rte_table_hash_entry_delete_key8_ext(\n \tuint64_t signature;\n \tuint32_t bucket_index, i;\n \n-\tsignature = f->f_hash(key, f->key_size, f->seed);\n+\tsignature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);\n \tbucket_index = signature & (f->n_buckets - 1);\n \tbucket0 = (struct rte_bucket_4_8 *)\n \t\t&f->memory[bucket_index * f->bucket_size];\n@@ -510,10 +523,10 @@ rte_table_hash_entry_delete_key8_ext(\n \n \t\tfor (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {\n \t\t\tuint64_t bucket_signature = bucket->signature;\n-\t\t\tuint64_t bucket_key = bucket->key[i];\n+\t\t\tuint64_t *bucket_key = &bucket->key[i];\n \n \t\t\tif ((bucket_signature & mask) &&\n-\t\t\t\t(*((uint64_t *) key) == bucket_key)) {\n+\t\t\t\t(keycmp(bucket_key, key, &f->key_mask) == 0)) {\n \t\t\t\tuint8_t *bucket_data = &bucket->data[i *\n \t\t\t\t\tf->entry_size];\n \n@@ -546,16 +559,17 @@ rte_table_hash_entry_delete_key8_ext(\n \treturn 0;\n }\n \n-#define lookup_key8_cmp(key_in, bucket, pos)\t\t\t\\\n+#define lookup_key8_cmp(key_in, bucket, pos, f)\t\t\t\\\n {\t\t\t\t\t\t\t\t\\\n-\tuint64_t xor[4], signature;\t\t\t\t\\\n+\tuint64_t xor[4], signature, k;\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tsignature = ~bucket->signature;\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\txor[0] = (key_in[0] ^\t bucket->key[0]) | (signature & 1);\\\n-\txor[1] = (key_in[0] ^\t bucket->key[1]) | (signature & 2);\\\n-\txor[2] = (key_in[0] ^\t bucket->key[2]) | (signature & 4);\\\n-\txor[3] = (key_in[0] ^\t bucket->key[3]) | (signature & 8);\\\n+\tk = key_in[0] & f->key_mask;\t\t\t\t\\\n+\txor[0] = (k ^ bucket->key[0]) | (signature & 1);\t\t\\\n+\txor[1] = (k ^ bucket->key[1]) | (signature & 2);\t\t\\\n+\txor[2] = (k ^ bucket->key[2]) | (signature & 4);\t\t\\\n+\txor[3] = (k ^ bucket->key[3]) | (signature & 8);\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpos = 4;\t\t\t\t\t\t\\\n \tif (xor[0] == 0)\t\t\t\t\t\\\n@@ -583,27 +597,12 @@ rte_table_hash_entry_delete_key8_ext(\n \n #define lookup1_stage1(mbuf1, bucket1, f)\t\t\t\\\n {\t\t\t\t\t\t\t\t\\\n-\tuint64_t signature;\t\t\t\t\t\\\n-\tuint32_t bucket_index;\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tsignature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\\\n-\tbucket_index = signature & (f->n_buckets - 1);\t\t\\\n-\tbucket1 = (struct rte_bucket_4_8 *)\t\t\t\\\n-\t\t&f->memory[bucket_index * f->bucket_size];\t\\\n-\trte_prefetch0(bucket1);\t\t\t\t\t\\\n-}\n-\n-#define lookup1_stage1_dosig(mbuf1, bucket1, f)\t\t\t\\\n-{\t\t\t\t\t\t\t\t\\\n \tuint64_t *key;\t\t\t\t\t\t\\\n \tuint64_t signature;\t\t\t\t\t\\\n \tuint32_t bucket_index;\t\t\t\t\t\\\n-\tuint64_t hash_key_buffer;\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\\\n-\thash_key_buffer = *key & f->key_mask;\t\t\t\\\n-\tsignature = f->f_hash(&hash_key_buffer,\t\t\t\\\n-\t\tRTE_TABLE_HASH_KEY_SIZE, f->seed);\t\t\\\n+\tsignature = f->f_hash(key, &f->key_mask, KEY_SIZE, f->seed);\t\\\n \tbucket_index = signature & (f->n_buckets - 1);\t\t\\\n \tbucket1 = (struct rte_bucket_4_8 *)\t\t\t\\\n \t\t&f->memory[bucket_index * f->bucket_size];\t\\\n@@ -617,12 +616,9 @@ rte_table_hash_entry_delete_key8_ext(\n \tuint64_t pkt_mask;\t\t\t\t\t\\\n \tuint64_t *key;\t\t\t\t\t\t\\\n \tuint32_t pos;\t\t\t\t\t\t\\\n-\tuint64_t hash_key_buffer;\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\\\n-\thash_key_buffer = key[0] & f->key_mask;\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tlookup_key8_cmp((&hash_key_buffer), bucket2, pos);\t\\\n+\tlookup_key8_cmp(key, bucket2, pos, f);\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt_mask = ((bucket2->signature >> pos) & 1LLU) << pkt2_index;\\\n \tpkts_mask_out |= pkt_mask;\t\t\t\t\\\n@@ -641,12 +637,9 @@ rte_table_hash_entry_delete_key8_ext(\n \tuint64_t pkt_mask, bucket_mask;\t\t\t\t\\\n \tuint64_t *key;\t\t\t\t\t\t\\\n \tuint32_t pos;\t\t\t\t\t\t\\\n-\tuint64_t hash_key_buffer;\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\\\n-\thash_key_buffer = *key & f->key_mask;\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tlookup_key8_cmp((&hash_key_buffer), bucket2, pos);\t\\\n+\tlookup_key8_cmp(key, bucket2, pos, f);\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt_mask = ((bucket2->signature >> pos) & 1LLU) << pkt2_index;\\\n \tpkts_mask_out |= pkt_mask;\t\t\t\t\\\n@@ -670,13 +663,10 @@ rte_table_hash_entry_delete_key8_ext(\n \tuint64_t pkt_mask, bucket_mask;\t\t\t\t\\\n \tuint64_t *key;\t\t\t\t\t\t\\\n \tuint32_t pos;\t\t\t\t\t\t\\\n-\tuint64_t hash_key_buffer;\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tbucket = buckets[pkt_index];\t\t\t\t\\\n \tkey = keys[pkt_index];\t\t\t\t\t\\\n-\thash_key_buffer = (*key) & f->key_mask;\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tlookup_key8_cmp((&hash_key_buffer), bucket, pos);\t\\\n+\tlookup_key8_cmp(key, bucket, pos, f);\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt_mask = ((bucket->signature >> pos) & 1LLU) << pkt_index;\\\n \tpkts_mask_out |= pkt_mask;\t\t\t\t\\\n@@ -738,29 +728,9 @@ rte_table_hash_entry_delete_key8_ext(\n \trte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\\\n }\n \n-#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f)\t\\\n-{\t\t\t\t\t\t\t\t\\\n-\tuint64_t signature10, signature11;\t\t\t\\\n-\tuint32_t bucket10_index, bucket11_index;\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tsignature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\\\n-\tbucket10_index = signature10 & (f->n_buckets - 1);\t\\\n-\tbucket10 = (struct rte_bucket_4_8 *)\t\t\t\\\n-\t\t&f->memory[bucket10_index * f->bucket_size];\t\\\n-\trte_prefetch0(bucket10);\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tsignature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\\\n-\tbucket11_index = signature11 & (f->n_buckets - 1);\t\\\n-\tbucket11 = (struct rte_bucket_4_8 *)\t\t\t\\\n-\t\t&f->memory[bucket11_index * f->bucket_size];\t\\\n-\trte_prefetch0(bucket11);\t\t\t\t\\\n-}\n-\n-#define lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f)\\\n+#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f)\\\n {\t\t\t\t\t\t\t\t\\\n \tuint64_t *key10, *key11;\t\t\t\t\\\n-\tuint64_t hash_offset_buffer10;\t\t\t\t\\\n-\tuint64_t hash_offset_buffer11;\t\t\t\t\\\n \tuint64_t signature10, signature11;\t\t\t\\\n \tuint32_t bucket10_index, bucket11_index;\t\t\\\n \trte_table_hash_op_hash f_hash = f->f_hash;\t\t\\\n@@ -769,18 +739,14 @@ rte_table_hash_entry_delete_key8_ext(\n \t\t\t\t\t\t\t\t\\\n \tkey10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, key_offset);\\\n \tkey11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, key_offset);\\\n-\thash_offset_buffer10 = *key10 & f->key_mask;\t\t\\\n-\thash_offset_buffer11 = *key11 & f->key_mask;\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\tsignature10 = f_hash(&hash_offset_buffer10,\t\t\\\n-\t\tRTE_TABLE_HASH_KEY_SIZE, seed);\t\t\t\\\n+\tsignature10 = f_hash(key10, &f->key_mask, KEY_SIZE, seed);\t\\\n \tbucket10_index = signature10 & (f->n_buckets - 1);\t\\\n \tbucket10 = (struct rte_bucket_4_8 *)\t\t\t\\\n \t\t&f->memory[bucket10_index * f->bucket_size];\t\\\n \trte_prefetch0(bucket10);\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\tsignature11 = f_hash(&hash_offset_buffer11,\t\t\\\n-\t\tRTE_TABLE_HASH_KEY_SIZE, seed);\t\t\t\\\n+\tsignature11 = f_hash(key11, &f->key_mask,KEY_SIZE, seed);\t\\\n \tbucket11_index = signature11 & (f->n_buckets - 1);\t\\\n \tbucket11 = (struct rte_bucket_4_8 *)\t\t\t\\\n \t\t&f->memory[bucket11_index * f->bucket_size];\t\\\n@@ -793,17 +759,13 @@ rte_table_hash_entry_delete_key8_ext(\n \tvoid *a20, *a21;\t\t\t\t\t\\\n \tuint64_t pkt20_mask, pkt21_mask;\t\t\t\\\n \tuint64_t *key20, *key21;\t\t\t\t\\\n-\tuint64_t hash_offset_buffer20;\t\t\t\t\\\n-\tuint64_t hash_offset_buffer21;\t\t\t\t\\\n \tuint32_t pos20, pos21;\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\\\n \tkey21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\\\n-\thash_offset_buffer20 = *key20 & f->key_mask;\t\t\\\n-\thash_offset_buffer21 = *key21 & f->key_mask;\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\tlookup_key8_cmp((&hash_offset_buffer20), bucket20, pos20);\\\n-\tlookup_key8_cmp((&hash_offset_buffer21), bucket21, pos21);\\\n+\tlookup_key8_cmp(key20, bucket20, pos20, f);\t\t\t\\\n+\tlookup_key8_cmp(key21, bucket21, pos21, f);\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt20_mask = ((bucket20->signature >> pos20) & 1LLU) << pkt20_index;\\\n \tpkt21_mask = ((bucket21->signature >> pos21) & 1LLU) << pkt21_index;\\\n@@ -826,17 +788,13 @@ rte_table_hash_entry_delete_key8_ext(\n \tvoid *a20, *a21;\t\t\t\t\t\\\n \tuint64_t pkt20_mask, pkt21_mask, bucket20_mask, bucket21_mask;\\\n \tuint64_t *key20, *key21;\t\t\t\t\\\n-\tuint64_t hash_offset_buffer20;\t\t\t\t\\\n-\tuint64_t hash_offset_buffer21;\t\t\t\t\\\n \tuint32_t pos20, pos21;\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tkey20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\\\n \tkey21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\\\n-\thash_offset_buffer20 = *key20 & f->key_mask;\t\t\\\n-\thash_offset_buffer21 = *key21 & f->key_mask;\t\t\\\n \t\t\t\t\t\t\t\t\\\n-\tlookup_key8_cmp((&hash_offset_buffer20), bucket20, pos20);\\\n-\tlookup_key8_cmp((&hash_offset_buffer21), bucket21, pos21);\\\n+\tlookup_key8_cmp(key20, bucket20, pos20, f);\t\t\t\\\n+\tlookup_key8_cmp(key21, bucket21, pos21, f);\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tpkt20_mask = ((bucket20->signature >> pos20) & 1LLU) << pkt20_index;\\\n \tpkt21_mask = ((bucket21->signature >> pos21) & 1LLU) << pkt21_index;\\\n@@ -871,8 +829,8 @@ rte_table_hash_lookup_key8_lru(\n \tstruct rte_table_hash *f = (struct rte_table_hash *) table;\n \tstruct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;\n \tstruct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;\n-\tuint32_t pkt00_index, pkt01_index, pkt10_index,\n-\t\t\tpkt11_index, pkt20_index, pkt21_index;\n+\tuint32_t pkt00_index, pkt01_index, pkt10_index;\n+\tuint32_t pkt11_index, pkt20_index, pkt21_index;\n \tuint64_t pkts_mask_out = 0;\n \n \t__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);\n@@ -888,7 +846,7 @@ rte_table_hash_lookup_key8_lru(\n \t\t\tlookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);\n \t\t\tlookup1_stage1(mbuf, bucket, f);\n \t\t\tlookup1_stage2_lru(pkt_index, mbuf, bucket,\n-\t\t\t\t\tpkts_mask_out, entries, f);\n+\t\t\t\tpkts_mask_out, entries, f);\n \t\t}\n \n \t\t*lookup_hit_mask = pkts_mask_out;\n@@ -984,133 +942,7 @@ rte_table_hash_lookup_key8_lru(\n \t*lookup_hit_mask = pkts_mask_out;\n \tRTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));\n \treturn 0;\n-} /* rte_table_hash_lookup_key8_lru() */\n-\n-static int\n-rte_table_hash_lookup_key8_lru_dosig(\n-\tvoid *table,\n-\tstruct rte_mbuf **pkts,\n-\tuint64_t pkts_mask,\n-\tuint64_t *lookup_hit_mask,\n-\tvoid **entries)\n-{\n-\tstruct rte_table_hash *f = (struct rte_table_hash *) table;\n-\tstruct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;\n-\tstruct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;\n-\tuint32_t pkt00_index, pkt01_index, pkt10_index;\n-\tuint32_t pkt11_index, pkt20_index, pkt21_index;\n-\tuint64_t pkts_mask_out = 0;\n-\n-\t__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);\n-\tRTE_TABLE_HASH_KEY8_STATS_PKTS_IN_ADD(f, n_pkts_in);\n-\n-\t/* Cannot run the pipeline with less than 5 packets */\n-\tif (__builtin_popcountll(pkts_mask) < 5) {\n-\t\tfor ( ; pkts_mask; ) {\n-\t\t\tstruct rte_bucket_4_8 *bucket;\n-\t\t\tstruct rte_mbuf *mbuf;\n-\t\t\tuint32_t pkt_index;\n-\n-\t\t\tlookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);\n-\t\t\tlookup1_stage1_dosig(mbuf, bucket, f);\n-\t\t\tlookup1_stage2_lru(pkt_index, mbuf, bucket,\n-\t\t\t\tpkts_mask_out, entries, f);\n-\t\t}\n-\n-\t\t*lookup_hit_mask = pkts_mask_out;\n-\t\tRTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));\n-\t\treturn 0;\n-\t}\n-\n-\t/*\n-\t * Pipeline fill\n-\t *\n-\t */\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,\n-\t\tpkts_mask, f);\n-\n-\t/* Pipeline feed */\n-\tmbuf10 = mbuf00;\n-\tmbuf11 = mbuf01;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,\n-\t\tpkts_mask, f);\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t/*\n-\t * Pipeline run\n-\t *\n-\t */\n-\tfor ( ; pkts_mask; ) {\n-\t\t/* Pipeline feed */\n-\t\tbucket20 = bucket10;\n-\t\tbucket21 = bucket11;\n-\t\tmbuf20 = mbuf10;\n-\t\tmbuf21 = mbuf11;\n-\t\tmbuf10 = mbuf00;\n-\t\tmbuf11 = mbuf01;\n-\t\tpkt20_index = pkt10_index;\n-\t\tpkt21_index = pkt11_index;\n-\t\tpkt10_index = pkt00_index;\n-\t\tpkt11_index = pkt01_index;\n-\n-\t\t/* Pipeline stage 0 */\n-\t\tlookup2_stage0_with_odd_support(pkt00_index, pkt01_index,\n-\t\t\tmbuf00, mbuf01, pkts, pkts_mask, f);\n-\n-\t\t/* Pipeline stage 1 */\n-\t\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t\t/* Pipeline stage 2 */\n-\t\tlookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\t\tbucket20, bucket21, pkts_mask_out, entries, f);\n-\t}\n-\n-\t/*\n-\t * Pipeline flush\n-\t *\n-\t */\n-\t/* Pipeline feed */\n-\tbucket20 = bucket10;\n-\tbucket21 = bucket11;\n-\tmbuf20 = mbuf10;\n-\tmbuf21 = mbuf11;\n-\tmbuf10 = mbuf00;\n-\tmbuf11 = mbuf01;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\tbucket20, bucket21, pkts_mask_out, entries, f);\n-\n-\t/* Pipeline feed */\n-\tbucket20 = bucket10;\n-\tbucket21 = bucket11;\n-\tmbuf20 = mbuf10;\n-\tmbuf21 = mbuf11;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\tbucket20, bucket21, pkts_mask_out, entries, f);\n-\n-\t*lookup_hit_mask = pkts_mask_out;\n-\tRTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));\n-\treturn 0;\n-} /* rte_table_hash_lookup_key8_lru_dosig() */\n+} /* lookup LRU */\n \n static int\n rte_table_hash_lookup_key8_ext(\n@@ -1142,8 +974,8 @@ rte_table_hash_lookup_key8_ext(\n \t\t\tlookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);\n \t\t\tlookup1_stage1(mbuf, bucket, f);\n \t\t\tlookup1_stage2_ext(pkt_index, mbuf, bucket,\n-\t\t\t\tpkts_mask_out, entries, buckets_mask, buckets,\n-\t\t\t\tkeys, f);\n+\t\t\t\tpkts_mask_out, entries, buckets_mask,\n+\t\t\t\tbuckets, keys, f);\n \t\t}\n \n \t\tgoto grind_next_buckets;\n@@ -1260,157 +1092,7 @@ rte_table_hash_lookup_key8_ext(\n \t*lookup_hit_mask = pkts_mask_out;\n \tRTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));\n \treturn 0;\n-} /* rte_table_hash_lookup_key8_ext() */\n-\n-static int\n-rte_table_hash_lookup_key8_ext_dosig(\n-\tvoid *table,\n-\tstruct rte_mbuf **pkts,\n-\tuint64_t pkts_mask,\n-\tuint64_t *lookup_hit_mask,\n-\tvoid **entries)\n-{\n-\tstruct rte_table_hash *f = (struct rte_table_hash *) table;\n-\tstruct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;\n-\tstruct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;\n-\tuint32_t pkt00_index, pkt01_index, pkt10_index;\n-\tuint32_t pkt11_index, pkt20_index, pkt21_index;\n-\tuint64_t pkts_mask_out = 0, buckets_mask = 0;\n-\tstruct rte_bucket_4_8 *buckets[RTE_PORT_IN_BURST_SIZE_MAX];\n-\tuint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];\n-\n-\t__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);\n-\tRTE_TABLE_HASH_KEY8_STATS_PKTS_IN_ADD(f, n_pkts_in);\n-\n-\t/* Cannot run the pipeline with less than 5 packets */\n-\tif (__builtin_popcountll(pkts_mask) < 5) {\n-\t\tfor ( ; pkts_mask; ) {\n-\t\t\tstruct rte_bucket_4_8 *bucket;\n-\t\t\tstruct rte_mbuf *mbuf;\n-\t\t\tuint32_t pkt_index;\n-\n-\t\t\tlookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);\n-\t\t\tlookup1_stage1_dosig(mbuf, bucket, f);\n-\t\t\tlookup1_stage2_ext(pkt_index, mbuf, bucket,\n-\t\t\t\tpkts_mask_out, entries, buckets_mask,\n-\t\t\t\tbuckets, keys, f);\n-\t\t}\n-\n-\t\tgoto grind_next_buckets;\n-\t}\n-\n-\t/*\n-\t * Pipeline fill\n-\t *\n-\t */\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,\n-\t\tpkts_mask, f);\n-\n-\t/* Pipeline feed */\n-\tmbuf10 = mbuf00;\n-\tmbuf11 = mbuf01;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,\n-\t\tpkts_mask, f);\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t/*\n-\t * Pipeline run\n-\t *\n-\t */\n-\tfor ( ; pkts_mask; ) {\n-\t\t/* Pipeline feed */\n-\t\tbucket20 = bucket10;\n-\t\tbucket21 = bucket11;\n-\t\tmbuf20 = mbuf10;\n-\t\tmbuf21 = mbuf11;\n-\t\tmbuf10 = mbuf00;\n-\t\tmbuf11 = mbuf01;\n-\t\tpkt20_index = pkt10_index;\n-\t\tpkt21_index = pkt11_index;\n-\t\tpkt10_index = pkt00_index;\n-\t\tpkt11_index = pkt01_index;\n-\n-\t\t/* Pipeline stage 0 */\n-\t\tlookup2_stage0_with_odd_support(pkt00_index, pkt01_index,\n-\t\t\tmbuf00, mbuf01, pkts, pkts_mask, f);\n-\n-\t\t/* Pipeline stage 1 */\n-\t\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t\t/* Pipeline stage 2 */\n-\t\tlookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\t\tbucket20, bucket21, pkts_mask_out, entries,\n-\t\t\tbuckets_mask, buckets, keys, f);\n-\t}\n-\n-\t/*\n-\t * Pipeline flush\n-\t *\n-\t */\n-\t/* Pipeline feed */\n-\tbucket20 = bucket10;\n-\tbucket21 = bucket11;\n-\tmbuf20 = mbuf10;\n-\tmbuf21 = mbuf11;\n-\tmbuf10 = mbuf00;\n-\tmbuf11 = mbuf01;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\tbucket20, bucket21, pkts_mask_out, entries,\n-\t\tbuckets_mask, buckets, keys, f);\n-\n-\t/* Pipeline feed */\n-\tbucket20 = bucket10;\n-\tbucket21 = bucket11;\n-\tmbuf20 = mbuf10;\n-\tmbuf21 = mbuf11;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,\n-\t\tbucket20, bucket21, pkts_mask_out, entries,\n-\t\tbuckets_mask, buckets, keys, f);\n-\n-grind_next_buckets:\n-\t/* Grind next buckets */\n-\tfor ( ; buckets_mask; ) {\n-\t\tuint64_t buckets_mask_next = 0;\n-\n-\t\tfor ( ; buckets_mask; ) {\n-\t\t\tuint64_t pkt_mask;\n-\t\t\tuint32_t pkt_index;\n-\n-\t\t\tpkt_index = __builtin_ctzll(buckets_mask);\n-\t\t\tpkt_mask = 1LLU << pkt_index;\n-\t\t\tbuckets_mask &= ~pkt_mask;\n-\n-\t\t\tlookup_grinder(pkt_index, buckets, keys, pkts_mask_out,\n-\t\t\t\tentries, buckets_mask_next, f);\n-\t\t}\n-\n-\t\tbuckets_mask = buckets_mask_next;\n-\t}\n-\n-\t*lookup_hit_mask = pkts_mask_out;\n-\tRTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));\n-\treturn 0;\n-} /* rte_table_hash_lookup_key8_dosig_ext() */\n+} /* lookup EXT */\n \n static int\n rte_table_hash_key8_stats_read(void *table, struct rte_table_stats *stats, int clear)\n@@ -1437,17 +1119,6 @@ struct rte_table_ops rte_table_hash_key8_lru_ops = {\n \t.f_stats = rte_table_hash_key8_stats_read,\n };\n \n-struct rte_table_ops rte_table_hash_key8_lru_dosig_ops = {\n-\t.f_create = rte_table_hash_create_key8_lru,\n-\t.f_free = rte_table_hash_free_key8_lru,\n-\t.f_add = rte_table_hash_entry_add_key8_lru,\n-\t.f_delete = rte_table_hash_entry_delete_key8_lru,\n-\t.f_add_bulk = NULL,\n-\t.f_delete_bulk = NULL,\n-\t.f_lookup = rte_table_hash_lookup_key8_lru_dosig,\n-\t.f_stats = rte_table_hash_key8_stats_read,\n-};\n-\n struct rte_table_ops rte_table_hash_key8_ext_ops = {\n \t.f_create = rte_table_hash_create_key8_ext,\n \t.f_free = rte_table_hash_free_key8_ext,\n@@ -1458,14 +1129,3 @@ struct rte_table_ops rte_table_hash_key8_ext_ops = {\n \t.f_lookup = rte_table_hash_lookup_key8_ext,\n \t.f_stats = rte_table_hash_key8_stats_read,\n };\n-\n-struct rte_table_ops rte_table_hash_key8_ext_dosig_ops = {\n-\t.f_create = rte_table_hash_create_key8_ext,\n-\t.f_free = rte_table_hash_free_key8_ext,\n-\t.f_add = rte_table_hash_entry_add_key8_ext,\n-\t.f_delete = rte_table_hash_entry_delete_key8_ext,\n-\t.f_add_bulk = NULL,\n-\t.f_delete_bulk = NULL,\n-\t.f_lookup = rte_table_hash_lookup_key8_ext_dosig,\n-\t.f_stats = rte_table_hash_key8_stats_read,\n-};\ndiff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c\nindex 5a4864e..9fbc2c2 100644\n--- a/lib/librte_table/rte_table_hash_lru.c\n+++ b/lib/librte_table/rte_table_hash_lru.c\n@@ -1,34 +1,34 @@\n /*-\n- *\t BSD LICENSE\n+ *   BSD LICENSE\n  *\n- *\t Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n- *\t All rights reserved.\n+ *   Copyright(c) 2010-2017 Intel Corporation. All rights reserved.\n+ *   All rights reserved.\n  *\n- *\t Redistribution and use in source and binary forms, with or without\n- *\t modification, are permitted provided that the following conditions\n- *\t are met:\n+ *   Redistribution and use in source and binary forms, with or without\n+ *   modification, are permitted provided that the following conditions\n+ *   are met:\n  *\n- *\t* Redistributions of source code must retain the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer.\n- *\t* Redistributions in binary form must reproduce the above copyright\n- *\t\t notice, this list of conditions and the following disclaimer in\n- *\t\t the documentation and/or other materials provided with the\n- *\t\t distribution.\n- *\t* Neither the name of Intel Corporation nor the names of its\n- *\t\t contributors may be used to endorse or promote products derived\n- *\t\t from this software without specific prior written permission.\n+ *     * Redistributions of source code must retain the above copyright\n+ *       notice, this list of conditions and the following disclaimer.\n+ *     * Redistributions in binary form must reproduce the above copyright\n+ *       notice, this list of conditions and the following disclaimer in\n+ *       the documentation and/or other materials provided with the\n+ *       distribution.\n+ *     * Neither the name of Intel Corporation nor the names of its\n+ *       contributors may be used to endorse or promote products derived\n+ *       from this software without specific prior written permission.\n  *\n- *\t THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n- *\t \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n- *\t LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n- *\t A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n- *\t OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n- *\t SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n- *\t LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n- *\t DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n- *\t THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n- *\t (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n- *\t OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+ *   \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n  */\n \n #include <string.h>\n@@ -86,7 +86,6 @@ struct rte_table_hash {\n \tuint32_t n_buckets;\n \trte_table_hash_op_hash f_hash;\n \tuint64_t seed;\n-\tuint32_t signature_offset;\n \tuint32_t key_offset;\n \n \t/* Internal */\n@@ -99,6 +98,7 @@ struct rte_table_hash {\n \tstruct grinder grinders[RTE_PORT_IN_BURST_SIZE_MAX];\n \n \t/* Tables */\n+\tuint64_t *key_mask;\n \tstruct bucket *buckets;\n \tuint8_t *key_mem;\n \tuint8_t *data_mem;\n@@ -109,12 +109,39 @@ struct rte_table_hash {\n };\n \n static int\n-check_params_create(struct rte_table_hash_lru_params *params)\n+keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes)\n {\n-\tuint32_t n_buckets_min;\n+\tuint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;\n+\tuint32_t i;\n+\n+\tfor (i = 0; i < n_bytes / sizeof(uint64_t); i++)\n+\t\tif (a64[i] != (b64[i] & b_mask64[i]))\n+\t\t\treturn 1;\n+\n+\treturn 0;\n+}\n+\n+static void\n+keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes)\n+{\n+\tuint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;\n+\tuint32_t i;\n+\n+\tfor (i = 0; i < n_bytes / sizeof(uint64_t); i++)\n+\t\tdst64[i] = src64[i] & src_mask64[i];\n+}\n+\n+static int\n+check_params_create(struct rte_table_hash_params *params)\n+{\n+\t/* name */\n+\tif (params->name == NULL) {\n+\t\tRTE_LOG(ERR, TABLE, \"%s: name invalid value\\n\", __func__);\n+\t\treturn -EINVAL;\n+\t}\n \n \t/* key_size */\n-\tif ((params->key_size == 0) ||\n+\tif ((params->key_size < sizeof(uint64_t)) ||\n \t\t(!rte_is_power_of_2(params->key_size))) {\n \t\tRTE_LOG(ERR, TABLE, \"%s: key_size invalid value\\n\", __func__);\n \t\treturn -EINVAL;\n@@ -128,10 +155,8 @@ check_params_create(struct rte_table_hash_lru_params *params)\n \t}\n \n \t/* n_buckets */\n-\tn_buckets_min = (params->n_keys + KEYS_PER_BUCKET - 1) / params->n_keys;\n \tif ((params->n_buckets == 0) ||\n-\t\t(!rte_is_power_of_2(params->n_keys)) ||\n-\t\t(params->n_buckets < n_buckets_min)) {\n+\t\t(!rte_is_power_of_2(params->n_keys))) {\n \t\tRTE_LOG(ERR, TABLE, \"%s: n_buckets invalid value\\n\", __func__);\n \t\treturn -EINVAL;\n \t}\n@@ -148,13 +173,13 @@ check_params_create(struct rte_table_hash_lru_params *params)\n static void *\n rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)\n {\n-\tstruct rte_table_hash_lru_params *p =\n-\t\tparams;\n+\tstruct rte_table_hash_params *p = params;\n \tstruct rte_table_hash *t;\n \tuint32_t total_size, table_meta_sz;\n-\tuint32_t bucket_sz, key_sz, key_stack_sz, data_sz;\n-\tuint32_t bucket_offset, key_offset, key_stack_offset, data_offset;\n-\tuint32_t i;\n+\tuint32_t key_mask_sz, bucket_sz, key_sz, key_stack_sz, data_sz;\n+\tuint32_t key_mask_offset, bucket_offset, key_offset;\n+\tuint32_t key_stack_offset, data_offset;\n+\tuint32_t n_keys, n_buckets, i;\n \n \t/* Check input parameters */\n \tif ((check_params_create(p) != 0) ||\n@@ -164,33 +189,53 @@ rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)\n \t\treturn NULL;\n \t}\n \n+\t/*\n+\t * Table dimensioning\n+\t *\n+\t * Objective: Pick the number of buckets (n_buckets) so that there a chance\n+\t * to store n_keys keys in the table.\n+\t * \n+\t * Note: Since the buckets do not get extended, it is not possible to\n+\t * guarantee that n_keys keys can be stored in the table at any time. In the\n+\t * worst case scenario when all the n_keys fall into the same bucket, only\n+\t * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case\n+\t * defeats the purpose of the hash table. It indicates unsuitable f_hash or\n+\t * n_keys to n_buckets ratio.\n+\t *\n+\t * MIN(n_buckets) = n_keys / KEYS_PER_BUCKET\n+\t */\n+\tn_keys = p->n_keys;\n+\tif (n_keys < KEYS_PER_BUCKET)\n+\t\tn_keys = KEYS_PER_BUCKET;\n+\tn_buckets = RTE_MAX(p->n_buckets, n_keys / KEYS_PER_BUCKET);\n+\n \t/* Memory allocation */\n \ttable_meta_sz = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));\n-\tbucket_sz = RTE_CACHE_LINE_ROUNDUP(p->n_buckets * sizeof(struct bucket));\n-\tkey_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * p->key_size);\n-\tkey_stack_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * sizeof(uint32_t));\n-\tdata_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);\n-\ttotal_size = table_meta_sz + bucket_sz + key_sz + key_stack_sz +\n-\t\tdata_sz;\n-\n-\tt = rte_zmalloc_socket(\"TABLE\", total_size, RTE_CACHE_LINE_SIZE, socket_id);\n+\tkey_mask_sz = RTE_CACHE_LINE_ROUNDUP(p->key_size);\n+\tbucket_sz = RTE_CACHE_LINE_ROUNDUP(n_buckets * sizeof(struct bucket));\n+\tkey_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * p->key_size);\n+\tkey_stack_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * sizeof(uint32_t));\n+\tdata_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * entry_size);\n+\ttotal_size = table_meta_sz + key_mask_sz + bucket_sz + key_sz +\n+\t\tkey_stack_sz + data_sz;\n+\n+\tt = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);\n \tif (t == NULL) {\n \t\tRTE_LOG(ERR, TABLE,\n-\t\t\t\"%s: Cannot allocate %u bytes for hash table\\n\",\n-\t\t\t__func__, total_size);\n+\t\t\t\"%s: Cannot allocate %u bytes for hash table %s\\n\",\n+\t\t\t__func__, total_size, p->name);\n \t\treturn NULL;\n \t}\n-\tRTE_LOG(INFO, TABLE, \"%s (%u-byte key): Hash table memory footprint is \"\n-\t\t\"%u bytes\\n\", __func__, p->key_size, total_size);\n+\tRTE_LOG(INFO, TABLE, \"%s (%u-byte key): Hash table %s memory footprint is \"\n+\t\t\"%u bytes\\n\", __func__, p->key_size, p->name, total_size);\n \n \t/* Memory initialization */\n \tt->key_size = p->key_size;\n \tt->entry_size = entry_size;\n-\tt->n_keys = p->n_keys;\n-\tt->n_buckets = p->n_buckets;\n+\tt->n_keys = n_keys;\n+\tt->n_buckets = n_buckets;\n \tt->f_hash = p->f_hash;\n \tt->seed = p->seed;\n-\tt->signature_offset = p->signature_offset;\n \tt->key_offset = p->key_offset;\n \n \t/* Internal */\n@@ -199,16 +244,24 @@ rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)\n \tt->data_size_shl = __builtin_ctzl(entry_size);\n \n \t/* Tables */\n-\tbucket_offset = 0;\n+\tkey_mask_offset = 0;\n+\tbucket_offset = key_mask_offset + key_mask_sz;\n \tkey_offset = bucket_offset + bucket_sz;\n \tkey_stack_offset = key_offset + key_sz;\n \tdata_offset = key_stack_offset + key_stack_sz;\n \n+\tt->key_mask = (uint64_t *) &t->memory[key_mask_offset];\n \tt->buckets = (struct bucket *) &t->memory[bucket_offset];\n \tt->key_mem = &t->memory[key_offset];\n \tt->key_stack = (uint32_t *) &t->memory[key_stack_offset];\n \tt->data_mem = &t->memory[data_offset];\n \n+\t/* Key mask */\n+\tif (p->key_mask == NULL)\n+\t\tmemset(t->key_mask, 0xFF, p->key_size);\n+\telse\n+\t\tmemcpy(t->key_mask, p->key_mask, p->key_size);\n+\n \t/* Key stack */\n \tfor (i = 0; i < t->n_keys; i++)\n \t\tt->key_stack[i] = t->n_keys - 1 - i;\n@@ -246,7 +299,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,\n \tuint64_t sig;\n \tuint32_t bkt_index, i;\n \n-\tsig = t->f_hash(key, t->key_size, t->seed);\n+\tsig = t->f_hash(key, t->key_mask, t->key_size, t->seed);\n \tbkt_index = sig & t->bucket_mask;\n \tbkt = &t->buckets[bkt_index];\n \tsig = (sig >> 16) | 1LLU;\n@@ -258,8 +311,8 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,\n \t\tuint8_t *bkt_key = &t->key_mem[bkt_key_index <<\n \t\t\tt->key_size_shl];\n \n-\t\tif ((sig == bkt_sig) && (memcmp(key, bkt_key, t->key_size)\n-\t\t\t== 0)) {\n+\t\tif ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,\n+\t\t\tt->key_size) == 0)) {\n \t\t\tuint8_t *data = &t->data_mem[bkt_key_index <<\n \t\t\t\tt->data_size_shl];\n \n@@ -292,7 +345,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,\n \n \t\t\tbkt->sig[i] = (uint16_t) sig;\n \t\t\tbkt->key_pos[i] = bkt_key_index;\n-\t\t\tmemcpy(bkt_key, key, t->key_size);\n+\t\t\tkeycpy(bkt_key, key, t->key_mask, t->key_size);\n \t\t\tmemcpy(data, entry, t->entry_size);\n \t\t\tlru_update(bkt, i);\n \n@@ -311,7 +364,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,\n \t\tuint8_t *data = &t->data_mem[bkt_key_index << t->data_size_shl];\n \n \t\tbkt->sig[pos] = (uint16_t) sig;\n-\t\tmemcpy(bkt_key, key, t->key_size);\n+\t\tkeycpy(bkt_key, key, t->key_mask, t->key_size);\n \t\tmemcpy(data, entry, t->entry_size);\n \t\tlru_update(bkt, pos);\n \n@@ -330,7 +383,7 @@ rte_table_hash_lru_entry_delete(void *table, void *key, int *key_found,\n \tuint64_t sig;\n \tuint32_t bkt_index, i;\n \n-\tsig = t->f_hash(key, t->key_size, t->seed);\n+\tsig = t->f_hash(key, t->key_mask, t->key_size, t->seed);\n \tbkt_index = sig & t->bucket_mask;\n \tbkt = &t->buckets[bkt_index];\n \tsig = (sig >> 16) | 1LLU;\n@@ -343,7 +396,7 @@ rte_table_hash_lru_entry_delete(void *table, void *key, int *key_found,\n \t\t\tt->key_size_shl];\n \n \t\tif ((sig == bkt_sig) &&\n-\t\t\t(memcmp(key, bkt_key, t->key_size) == 0)) {\n+\t\t\t(keycmp(bkt_key, key, t->key_mask, t->key_size) == 0)) {\n \t\t\tuint8_t *data = &t->data_mem[bkt_key_index <<\n \t\t\t\tt->data_size_shl];\n \n@@ -365,8 +418,7 @@ static int rte_table_hash_lru_lookup_unoptimized(\n \tstruct rte_mbuf **pkts,\n \tuint64_t pkts_mask,\n \tuint64_t *lookup_hit_mask,\n-\tvoid **entries,\n-\tint dosig)\n+\tvoid **entries)\n {\n \tstruct rte_table_hash *t = (struct rte_table_hash *) table;\n \tuint64_t pkts_mask_out = 0;\n@@ -387,11 +439,7 @@ static int rte_table_hash_lru_lookup_unoptimized(\n \n \t\tpkt = pkts[pkt_index];\n \t\tkey = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);\n-\t\tif (dosig)\n-\t\t\tsig = (uint64_t) t->f_hash(key, t->key_size, t->seed);\n-\t\telse\n-\t\t\tsig = RTE_MBUF_METADATA_UINT32(pkt,\n-\t\t\t\tt->signature_offset);\n+\t\tsig = (uint64_t) t->f_hash(key, t->key_mask, t->key_size, t->seed);\n \n \t\tbkt_index = sig & t->bucket_mask;\n \t\tbkt = &t->buckets[bkt_index];\n@@ -404,7 +452,7 @@ static int rte_table_hash_lru_lookup_unoptimized(\n \t\t\tuint8_t *bkt_key = &t->key_mem[bkt_key_index <<\n \t\t\t\tt->key_size_shl];\n \n-\t\t\tif ((sig == bkt_sig) && (memcmp(key, bkt_key,\n+\t\t\tif ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,\n \t\t\t\tt->key_size) == 0)) {\n \t\t\t\tuint8_t *data = &t->data_mem[bkt_key_index <<\n \t\t\t\t\tt->data_size_shl];\n@@ -502,74 +550,75 @@ static int rte_table_hash_lru_lookup_unoptimized(\n \tmatch_pos = (LUT_MATCH_POS >> (mask_all << 1)) & 3;\t\\\n }\n \n-#define lookup_cmp_key(mbuf, key, match_key, f)\t\t\t\\\n-{\t\t\t\t\t\t\t\t\\\n+#define lookup_cmp_key(mbuf, key, match_key, f)\t\t\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n \tuint64_t *pkt_key = RTE_MBUF_METADATA_UINT64_PTR(mbuf, f->key_offset);\\\n-\tuint64_t *bkt_key = (uint64_t *) key;\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tswitch (f->key_size) {\t\t\t\t\t\\\n-\tcase 8:\t\t\t\t\t\t\t\\\n-\t{\t\t\t\t\t\t\t\\\n-\t\tuint64_t xor = pkt_key[0] ^ bkt_key[0];\t\t\\\n-\t\tmatch_key = 0;\t\t\t\t\t\\\n-\t\tif (xor == 0)\t\t\t\t\t\\\n-\t\t\tmatch_key = 1;\t\t\t\t\\\n-\t}\t\t\t\t\t\t\t\\\n-\tbreak;\t\t\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tcase 16:\t\t\t\t\t\t\\\n-\t{\t\t\t\t\t\t\t\\\n-\t\tuint64_t xor[2], or;\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\t\txor[0] = pkt_key[0] ^ bkt_key[0];\t\t\\\n-\t\txor[1] = pkt_key[1] ^ bkt_key[1];\t\t\\\n-\t\tor = xor[0] | xor[1];\t\t\t\t\\\n-\t\tmatch_key = 0;\t\t\t\t\t\\\n-\t\tif (or == 0)\t\t\t\t\t\\\n-\t\t\tmatch_key = 1;\t\t\t\t\\\n-\t}\t\t\t\t\t\t\t\\\n-\tbreak;\t\t\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tcase 32:\t\t\t\t\t\t\\\n-\t{\t\t\t\t\t\t\t\\\n-\t\tuint64_t xor[4], or;\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\t\txor[0] = pkt_key[0] ^ bkt_key[0];\t\t\\\n-\t\txor[1] = pkt_key[1] ^ bkt_key[1];\t\t\\\n-\t\txor[2] = pkt_key[2] ^ bkt_key[2];\t\t\\\n-\t\txor[3] = pkt_key[3] ^ bkt_key[3];\t\t\\\n-\t\tor = xor[0] | xor[1] | xor[2] | xor[3];\t\t\\\n-\t\tmatch_key = 0;\t\t\t\t\t\\\n-\t\tif (or == 0)\t\t\t\t\t\\\n-\t\t\tmatch_key = 1;\t\t\t\t\\\n-\t}\t\t\t\t\t\t\t\\\n-\tbreak;\t\t\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tcase 64:\t\t\t\t\t\t\\\n-\t{\t\t\t\t\t\t\t\\\n-\t\tuint64_t xor[8], or;\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\t\txor[0] = pkt_key[0] ^ bkt_key[0];\t\t\\\n-\t\txor[1] = pkt_key[1] ^ bkt_key[1];\t\t\\\n-\t\txor[2] = pkt_key[2] ^ bkt_key[2];\t\t\\\n-\t\txor[3] = pkt_key[3] ^ bkt_key[3];\t\t\\\n-\t\txor[4] = pkt_key[4] ^ bkt_key[4];\t\t\\\n-\t\txor[5] = pkt_key[5] ^ bkt_key[5];\t\t\\\n-\t\txor[6] = pkt_key[6] ^ bkt_key[6];\t\t\\\n-\t\txor[7] = pkt_key[7] ^ bkt_key[7];\t\t\\\n-\t\tor = xor[0] | xor[1] | xor[2] | xor[3] |\t\\\n-\t\t\txor[4] | xor[5] | xor[6] | xor[7];\t\\\n-\t\tmatch_key = 0;\t\t\t\t\t\\\n-\t\tif (or == 0)\t\t\t\t\t\\\n-\t\t\tmatch_key = 1;\t\t\t\t\\\n-\t}\t\t\t\t\t\t\t\\\n-\tbreak;\t\t\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tdefault:\t\t\t\t\t\t\\\n-\t\tmatch_key = 0;\t\t\t\t\t\\\n-\t\tif (memcmp(pkt_key, bkt_key, f->key_size) == 0)\t\\\n-\t\t\tmatch_key = 1;\t\t\t\t\\\n-\t}\t\t\t\t\t\t\t\\\n+\tuint64_t *bkt_key = (uint64_t *) key;\t\t\t\t\\\n+\tuint64_t *key_mask = f->key_mask;\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\tswitch (f->key_size) {\t\t\t\t\t\t\\\n+\tcase 8:\t\t\t\t\t\t\t\t\\\n+\t{\t\t\t\t\t\t\t\t\\\n+\t\tuint64_t xor = (pkt_key[0] & key_mask[0]) ^ bkt_key[0];\t\\\n+\t\tmatch_key = 0;\t\t\t\t\t\t\\\n+\t\tif (xor == 0)\t\t\t\t\t\t\\\n+\t\t\tmatch_key = 1;\t\t\t\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n+\tbreak;\t\t\t\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\tcase 16:\t\t\t\t\t\t\t\\\n+\t{\t\t\t\t\t\t\t\t\\\n+\t\tuint64_t xor[2], or;\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\t\txor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0];\t\t\\\n+\t\txor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1];\t\t\\\n+\t\tor = xor[0] | xor[1];\t\t\t\t\t\\\n+\t\tmatch_key = 0;\t\t\t\t\t\t\\\n+\t\tif (or == 0)\t\t\t\t\t\t\\\n+\t\t\tmatch_key = 1;\t\t\t\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n+\tbreak;\t\t\t\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\tcase 32:\t\t\t\t\t\t\t\\\n+\t{\t\t\t\t\t\t\t\t\\\n+\t\tuint64_t xor[4], or;\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\t\txor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0];\t\t\\\n+\t\txor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1];\t\t\\\n+\t\txor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2];\t\t\\\n+\t\txor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3];\t\t\\\n+\t\tor = xor[0] | xor[1] | xor[2] | xor[3];\t\t\t\\\n+\t\tmatch_key = 0;\t\t\t\t\t\t\\\n+\t\tif (or == 0)\t\t\t\t\t\t\\\n+\t\t\tmatch_key = 1;\t\t\t\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n+\tbreak;\t\t\t\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\tcase 64:\t\t\t\t\t\t\t\\\n+\t{\t\t\t\t\t\t\t\t\\\n+\t\tuint64_t xor[8], or;\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\t\txor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0];\t\t\\\n+\t\txor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1];\t\t\\\n+\t\txor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2];\t\t\\\n+\t\txor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3];\t\t\\\n+\t\txor[4] = (pkt_key[4] & key_mask[4]) ^ bkt_key[4];\t\t\\\n+\t\txor[5] = (pkt_key[5] & key_mask[5]) ^ bkt_key[5];\t\t\\\n+\t\txor[6] = (pkt_key[6] & key_mask[6]) ^ bkt_key[6];\t\t\\\n+\t\txor[7] = (pkt_key[7] & key_mask[7]) ^ bkt_key[7];\t\t\\\n+\t\tor = xor[0] | xor[1] | xor[2] | xor[3] |\t\t\\\n+\t\t\txor[4] | xor[5] | xor[6] | xor[7];\t\t\\\n+\t\tmatch_key = 0;\t\t\t\t\t\t\\\n+\t\tif (or == 0)\t\t\t\t\t\t\\\n+\t\t\tmatch_key = 1;\t\t\t\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n+\tbreak;\t\t\t\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\tdefault:\t\t\t\t\t\t\t\\\n+\t\tmatch_key = 0;\t\t\t\t\t\t\\\n+\t\tif (keycmp(bkt_key, pkt_key, key_mask, f->key_size) == 0)\t\\\n+\t\t\tmatch_key = 1;\t\t\t\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n }\n \n #define lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index)\\\n@@ -616,38 +665,7 @@ static int rte_table_hash_lru_lookup_unoptimized(\n \trte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\\\n }\n \n-#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index)\t\\\n-{\t\t\t\t\t\t\t\t\\\n-\tstruct grinder *g10, *g11;\t\t\t\t\\\n-\tuint64_t sig10, sig11, bkt10_index, bkt11_index;\t\\\n-\tstruct rte_mbuf *mbuf10, *mbuf11;\t\t\t\\\n-\tstruct bucket *bkt10, *bkt11, *buckets = t->buckets;\t\\\n-\tuint64_t bucket_mask = t->bucket_mask;\t\t\t\\\n-\tuint32_t signature_offset = t->signature_offset;\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tmbuf10 = pkts[pkt10_index];\t\t\t\t\\\n-\tsig10 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf10, signature_offset);\\\n-\tbkt10_index = sig10 & bucket_mask;\t\t\t\\\n-\tbkt10 = &buckets[bkt10_index];\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tmbuf11 = pkts[pkt11_index];\t\t\t\t\\\n-\tsig11 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf11, signature_offset);\\\n-\tbkt11_index = sig11 & bucket_mask;\t\t\t\\\n-\tbkt11 = &buckets[bkt11_index];\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\trte_prefetch0(bkt10);\t\t\t\t\t\\\n-\trte_prefetch0(bkt11);\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tg10 = &g[pkt10_index];\t\t\t\t\t\\\n-\tg10->sig = sig10;\t\t\t\t\t\\\n-\tg10->bkt = bkt10;\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\\\n-\tg11 = &g[pkt11_index];\t\t\t\t\t\\\n-\tg11->sig = sig11;\t\t\t\t\t\\\n-\tg11->bkt = bkt11;\t\t\t\t\t\\\n-}\n-\n-#define lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index)\\\n+#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index)\\\n {\t\t\t\t\t\t\t\t\\\n \tstruct grinder *g10, *g11;\t\t\t\t\\\n \tuint64_t sig10, sig11, bkt10_index, bkt11_index;\t\\\n@@ -662,13 +680,13 @@ static int rte_table_hash_lru_lookup_unoptimized(\n \t\t\t\t\t\t\t\t\\\n \tmbuf10 = pkts[pkt10_index];\t\t\t\t\\\n \tkey10 = RTE_MBUF_METADATA_UINT8_PTR(mbuf10, key_offset);\\\n-\tsig10 = (uint64_t) f_hash(key10, key_size, seed);\t\\\n+\tsig10 = (uint64_t) f_hash(key10, t->key_mask, key_size, seed);\\\n \tbkt10_index = sig10 & bucket_mask;\t\t\t\\\n \tbkt10 = &buckets[bkt10_index];\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n \tmbuf11 = pkts[pkt11_index];\t\t\t\t\\\n \tkey11 = RTE_MBUF_METADATA_UINT8_PTR(mbuf11, key_offset);\\\n-\tsig11 = (uint64_t) f_hash(key11, key_size, seed);\t\\\n+\tsig11 = (uint64_t) f_hash(key11, t->key_mask, key_size, seed);\\\n \tbkt11_index = sig11 & bucket_mask;\t\t\t\\\n \tbkt11 = &buckets[bkt11_index];\t\t\t\t\\\n \t\t\t\t\t\t\t\t\\\n@@ -819,7 +837,7 @@ static int rte_table_hash_lru_lookup(\n \t/* Cannot run the pipeline with less than 7 packets */\n \tif (__builtin_popcountll(pkts_mask) < 7)\n \t\treturn rte_table_hash_lru_lookup_unoptimized(table, pkts,\n-\t\t\tpkts_mask, lookup_hit_mask, entries, 0);\n+\t\t\tpkts_mask, lookup_hit_mask, entries);\n \n \t/* Pipeline stage 0 */\n \tlookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);\n@@ -923,140 +941,7 @@ static int rte_table_hash_lru_lookup(\n \t\tuint64_t pkts_mask_out_slow = 0;\n \n \t\tstatus = rte_table_hash_lru_lookup_unoptimized(table, pkts,\n-\t\t\tpkts_mask_match_many, &pkts_mask_out_slow, entries, 0);\n-\t\tpkts_mask_out |= pkts_mask_out_slow;\n-\t}\n-\n-\t*lookup_hit_mask = pkts_mask_out;\n-\tRTE_TABLE_HASH_LRU_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in - __builtin_popcountll(pkts_mask_out));\n-\treturn status;\n-}\n-\n-static int rte_table_hash_lru_lookup_dosig(\n-\tvoid *table,\n-\tstruct rte_mbuf **pkts,\n-\tuint64_t pkts_mask,\n-\tuint64_t *lookup_hit_mask,\n-\tvoid **entries)\n-{\n-\tstruct rte_table_hash *t = (struct rte_table_hash *) table;\n-\tstruct grinder *g = t->grinders;\n-\tuint64_t pkt00_index, pkt01_index, pkt10_index, pkt11_index;\n-\tuint64_t pkt20_index, pkt21_index, pkt30_index, pkt31_index;\n-\tuint64_t pkts_mask_out = 0, pkts_mask_match_many = 0;\n-\tint status = 0;\n-\n-\t__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);\n-\tRTE_TABLE_HASH_LRU_STATS_PKTS_IN_ADD(t, n_pkts_in);\n-\n-\t/* Cannot run the pipeline with less than 7 packets */\n-\tif (__builtin_popcountll(pkts_mask) < 7)\n-\t\treturn rte_table_hash_lru_lookup_unoptimized(table, pkts,\n-\t\t\tpkts_mask, lookup_hit_mask, entries, 1);\n-\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);\n-\n-\t/* Pipeline feed */\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);\n-\n-\t/* Pipeline feed */\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 0 */\n-\tlookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);\n-\n-\t/*\n-\t* Pipeline run\n-\t*\n-\t*/\n-\tfor ( ; pkts_mask; ) {\n-\t\t/* Pipeline feed */\n-\t\tpkt30_index = pkt20_index;\n-\t\tpkt31_index = pkt21_index;\n-\t\tpkt20_index = pkt10_index;\n-\t\tpkt21_index = pkt11_index;\n-\t\tpkt10_index = pkt00_index;\n-\t\tpkt11_index = pkt01_index;\n-\n-\t\t/* Pipeline stage 0 */\n-\t\tlookup2_stage0_with_odd_support(t, g, pkts, pkts_mask,\n-\t\t\tpkt00_index, pkt01_index);\n-\n-\t\t/* Pipeline stage 1 */\n-\t\tlookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);\n-\n-\t\t/* Pipeline stage 2 */\n-\t\tlookup2_stage2(t, g, pkt20_index, pkt21_index,\n-\t\t\tpkts_mask_match_many);\n-\n-\t\t/* Pipeline stage 3 */\n-\t\tlookup2_stage3(t, g, pkts, pkt30_index, pkt31_index,\n-\t\t\tpkts_mask_out, entries);\n-\t}\n-\n-\t/* Pipeline feed */\n-\tpkt30_index = pkt20_index;\n-\tpkt31_index = pkt21_index;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\tpkt10_index = pkt00_index;\n-\tpkt11_index = pkt01_index;\n-\n-\t/* Pipeline stage 1 */\n-\tlookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);\n-\n-\t/* Pipeline stage 3 */\n-\tlookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,\n-\t\tentries);\n-\n-\t/* Pipeline feed */\n-\tpkt30_index = pkt20_index;\n-\tpkt31_index = pkt21_index;\n-\tpkt20_index = pkt10_index;\n-\tpkt21_index = pkt11_index;\n-\n-\t/* Pipeline stage 2 */\n-\tlookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);\n-\n-\t/* Pipeline stage 3 */\n-\tlookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,\n-\t\tentries);\n-\n-\t/* Pipeline feed */\n-\tpkt30_index = pkt20_index;\n-\tpkt31_index = pkt21_index;\n-\n-\t/* Pipeline stage 3 */\n-\tlookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,\n-\t\tentries);\n-\n-\t/* Slow path */\n-\tpkts_mask_match_many &= ~pkts_mask_out;\n-\tif (pkts_mask_match_many) {\n-\t\tuint64_t pkts_mask_out_slow = 0;\n-\n-\t\tstatus = rte_table_hash_lru_lookup_unoptimized(table, pkts,\n-\t\t\tpkts_mask_match_many, &pkts_mask_out_slow, entries, 1);\n+\t\t\tpkts_mask_match_many, &pkts_mask_out_slow, entries);\n \t\tpkts_mask_out |= pkts_mask_out_slow;\n \t}\n \n@@ -1089,14 +974,3 @@ struct rte_table_ops rte_table_hash_lru_ops = {\n \t.f_lookup = rte_table_hash_lru_lookup,\n \t.f_stats = rte_table_hash_lru_stats_read,\n };\n-\n-struct rte_table_ops rte_table_hash_lru_dosig_ops = {\n-\t.f_create = rte_table_hash_lru_create,\n-\t.f_free = rte_table_hash_lru_free,\n-\t.f_add = rte_table_hash_lru_entry_add,\n-\t.f_delete = rte_table_hash_lru_entry_delete,\n-\t.f_add_bulk = NULL,\n-\t.f_delete_bulk = NULL,\n-\t.f_lookup = rte_table_hash_lru_lookup_dosig,\n-\t.f_stats = rte_table_hash_lru_stats_read,\n-};\n",
    "prefixes": [
        "dpdk-dev",
        "1/5"
    ]
}