Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/137506/?format=api
http://patches.dpdk.org/api/patches/137506/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/patch/20240229160308.365277-4-paul.szczepanek@arm.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20240229160308.365277-4-paul.szczepanek@arm.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20240229160308.365277-4-paul.szczepanek@arm.com", "date": "2024-02-29T16:03:07", "name": "[v6,3/4] docs: add pointer compression to the EAL guide", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "7a807d451af890b6037d54a77a4453fbd4c62cd2", "submitter": { "id": 3199, "url": "http://patches.dpdk.org/api/people/3199/?format=api", "name": "Paul Szczepanek", "email": "paul.szczepanek@arm.com" }, "delegate": { "id": 1, "url": "http://patches.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "http://patches.dpdk.org/project/dpdk/patch/20240229160308.365277-4-paul.szczepanek@arm.com/mbox/", "series": [ { "id": 31302, "url": "http://patches.dpdk.org/api/series/31302/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=31302", "date": "2024-02-29T16:03:04", "name": "add pointer compression API", "version": 6, "mbox": "http://patches.dpdk.org/series/31302/mbox/" } ], "comments": "http://patches.dpdk.org/api/patches/137506/comments/", "check": "success", "checks": "http://patches.dpdk.org/api/patches/137506/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id B04F043C3B;\n\tThu, 29 Feb 2024 17:04:02 +0100 (CET)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id C219842E5C;\n\tThu, 29 Feb 2024 17:03:45 +0100 (CET)", "from foss.arm.com (foss.arm.com [217.140.110.172])\n by mails.dpdk.org (Postfix) with ESMTP id 9655C42D78\n for <dev@dpdk.org>; Thu, 29 Feb 2024 17:03:39 +0100 (CET)", "from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])\n by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C1CC61516;\n Thu, 29 Feb 2024 08:04:17 -0800 (PST)", "from ampere-altra-2-1.usa.Arm.com (ampere-altra-2-1.usa.arm.com\n [10.118.91.158])\n by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DC89C3F6C4;\n Thu, 29 Feb 2024 08:03:38 -0800 (PST)" ], "From": "Paul Szczepanek <paul.szczepanek@arm.com>", "To": "dev@dpdk.org", "Cc": "Paul Szczepanek <paul.szczepanek@arm.com>,\n Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>", "Subject": "[PATCH v6 3/4] docs: add pointer compression to the EAL guide", "Date": "Thu, 29 Feb 2024 16:03:07 +0000", "Message-Id": "<20240229160308.365277-4-paul.szczepanek@arm.com>", "X-Mailer": "git-send-email 2.25.1", "In-Reply-To": "<20240229160308.365277-1-paul.szczepanek@arm.com>", "References": "<20230927150854.3670391-2-paul.szczepanek@arm.com>\n <20240229160308.365277-1-paul.szczepanek@arm.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Documentation added in the EAL guide for the new\nutility functions for pointer compression\nshowing example code and potential usecases.\n\nSigned-off-by: Paul Szczepanek <paul.szczepanek@arm.com>\nReviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>\n---\n .../prog_guide/env_abstraction_layer.rst | 142 ++++++++++++++++++\n 1 file changed, 142 insertions(+)\n\n--\n2.25.1", "diff": "diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst\nindex 6debf54efb..f04d032442 100644\n--- a/doc/guides/prog_guide/env_abstraction_layer.rst\n+++ b/doc/guides/prog_guide/env_abstraction_layer.rst\n@@ -1192,3 +1192,145 @@ will not be deallocated.\n\n Any successful deallocation event will trigger a callback, for which user\n applications and other DPDK subsystems can register.\n+\n+.. _pointer_compression:\n+\n+Pointer Compression\n+-------------------\n+\n+Use ``rte_ptr_compress_16()`` and ``rte_ptr_decompress_16()`` to compress and\n+decompress pointers into 16-bit offsets. Use ``rte_ptr_compress_32()`` and\n+``rte_ptr_decompress_32()`` to compress and decompress pointers into 32-bit\n+offsets.\n+\n+Compression takes advantage of the fact that pointers are usually located in a\n+limited memory region (like a mempool). By converting them to offsets from a\n+base memory address they can be stored in fewer bytes. How many bytes are needed\n+to store the offset is dictated by the memory region size and alignment of\n+objects the pointers point to.\n+\n+For example, a pointer which is part of a 4GB memory pool can be stored as 32\n+bit offset. If the pointer points to memory that is 8 bytes aligned then 3 bits\n+can be dropped from the offset and a 32GB memory pool can now fit in 32 bits.\n+\n+For performance reasons these requirements are not enforced programmatically.\n+The programmer is responsible for ensuring that the combination of distance\n+from the base pointer and memory alignment allow for storing of the offset in\n+the number of bits indicated by the function name (16 or 32). Start of mempool\n+memory would be a good candidate for the base pointer. Otherwise any pointer\n+that precedes all pointers, is close enough and has the same alignment as the\n+pointers being compressed will work.\n+\n+.. note::\n+\n+ Performance gains depend on the batch size of pointers and CPU capabilities\n+ such as vector extensions. It's important to measure the performance\n+ increase on target hardware. A test called ``ring_perf_autotest`` in\n+ ``dpdk-test`` can provide the measurements.\n+\n+Example usage\n+~~~~~~~~~~~~~\n+\n+In this example we send pointers between two cores through a ring. While this\n+is a realistic use case the code is simplified for demonstration purposes and\n+does not have error handling.\n+\n+.. code-block:: c\n+\n+ #include <rte_launch.h>\n+ #include <rte_ring.h>\n+ #include <rte_ring_elem.h>\n+ #include <rte_ptr_compress.h>\n+\n+ #define ITEMS_ARRAY_SIZE (1024)\n+ #define BATCH_SIZE (128)\n+ #define ALIGN_EXPONENT (3)\n+ #define ITEM_ALIGN (1<<ALIGN_EXPONENT)\n+ #define CORE_SEND (1)\n+ #define CORE_RECV (2)\n+\n+ struct item {\n+ int a;\n+ } __rte_aligned(ITEM_ALIGN);\n+\n+ static struct item items[ITEMS_ARRAY_SIZE] = {0};\n+ static struct rte_ring *ring = NULL;\n+\n+ static int\n+ send_compressed(void *args)\n+ {\n+ struct item *ptrs_send[BATCH_SIZE] = {0};\n+ unsigned int n_send = 0;\n+ struct rte_ring_zc_data zcd = {0};\n+\n+ /* in this example we only fill the ptrs_send once and reuse */\n+ for (;n_send < BATCH_SIZE; n_send++)\n+ ptrs_send[n_send] = &items[n_send];\n+\n+ for(;;) {\n+ n_send = rte_ring_enqueue_zc_burst_elem_start(\n+ ring, sizeof(uint32_t), BATCH_SIZE, &zcd, NULL);\n+\n+ /* compress ptrs_send into offsets */\n+ rte_ptr_compress_32(items, /* base pointer */\n+ ptrs_send, /* source array to be compressed */\n+ zcd.ptr1, /* destination array to store offsets */\n+ zcd.n1, /* how many pointers to compress */\n+ ALIGN_EXPONENT /* how many bits can we drop from the offset */);\n+\n+ if (zcd.ptr2 != NULL)\n+ rte_ptr_compress_32(items, ptrs_send + zcd.n1,\n+ zcd.ptr2, n_send - zcd.n1, ALIGN_EXPONENT);\n+\n+ rte_ring_enqueue_zc_finish(ring, n_send);\n+ }\n+ return 1;\n+ }\n+\n+ static int\n+ recv_compressed(void *args)\n+ {\n+ struct item *ptrs_recv[BATCH_SIZE] = {0};\n+ unsigned int n_recv;\n+ struct rte_ring_zc_data zcd = {0};\n+\n+ for(;;) {\n+ /* receive compressed pointers from the ring */\n+ n_recv = rte_ring_dequeue_zc_burst_elem_start(\n+ ring, sizeof(uint32_t), BATCH_SIZE, &zcd, NULL);\n+\n+ rte_ptr_decompress_32(items, /* base pointer */\n+ zcd.ptr1, /* source array to decompress */\n+ ptrs_recv, /* destination array to store pointers */\n+ zcd.n1, /* how many pointers to decompress */\n+ ALIGN_EXPONENT /* how many bits were dropped from the offset */);\n+\n+ /* handle the potential secondary buffer (caused by ring boundary) */\n+ if (zcd.ptr2 != NULL)\n+ rte_ptr_decompress_32(items,\n+ zcd.ptr2,\n+ ptrs_recv + zcd.n1,\n+ n_recv - zcd.n1,\n+ ALIGN_EXPONENT);\n+\n+ rte_ring_dequeue_zc_finish(ring, n_recv);\n+\n+ /* ptrs_recv contains what ptrs_send contained in the other thread */\n+ /* (...) */\n+ }\n+ return 1;\n+ }\n+\n+ void\n+ compression_example(void)\n+ {\n+ ring = rte_ring_create_elem(\n+ \"COMPR_PTRS\", sizeof(uint32_t),\n+ 1024, rte_socket_id(),\n+ RING_F_SP_ENQ | RING_F_SC_DEQ);\n+\n+ rte_eal_remote_launch(send_compressed, NULL, CORE_SEND);\n+ rte_eal_remote_launch(recv_compressed, NULL, CORE_RECV);\n+\n+ for(;;) {}\n+ }\n", "prefixes": [ "v6", "3/4" ] }{ "id": 137506, "url": "