Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/133677/?format=api
{ "id": 133677, "url": "http://patches.dpdk.org/api/patches/133677/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/patch/20231031181046.1203681-4-paul.szczepanek@arm.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20231031181046.1203681-4-paul.szczepanek@arm.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20231031181046.1203681-4-paul.szczepanek@arm.com", "date": "2023-10-31T18:10:46", "name": "[v3,3/3] docs: add pointer compression to the EAL guide", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "ada409d343679132c5f1a08b466ecd6cdfe66971", "submitter": { "id": 3199, "url": "http://patches.dpdk.org/api/people/3199/?format=api", "name": "Paul Szczepanek", "email": "paul.szczepanek@arm.com" }, "delegate": { "id": 1, "url": "http://patches.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "http://patches.dpdk.org/project/dpdk/patch/20231031181046.1203681-4-paul.szczepanek@arm.com/mbox/", "series": [ { "id": 30077, "url": "http://patches.dpdk.org/api/series/30077/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=30077", "date": "2023-10-31T18:10:43", "name": "add pointer compression API", "version": 3, "mbox": "http://patches.dpdk.org/series/30077/mbox/" } ], "comments": "http://patches.dpdk.org/api/patches/133677/comments/", "check": "fail", "checks": "http://patches.dpdk.org/api/patches/133677/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id BDB7043255;\n\tTue, 31 Oct 2023 19:11:24 +0100 (CET)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 46A0E4067A;\n\tTue, 31 Oct 2023 19:11:19 +0100 (CET)", "from foss.arm.com (foss.arm.com [217.140.110.172])\n by mails.dpdk.org (Postfix) with ESMTP id 477C940294\n for <dev@dpdk.org>; Tue, 31 Oct 2023 19:11:16 +0100 (CET)", "from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])\n by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6C2041480;\n Tue, 31 Oct 2023 11:11:57 -0700 (PDT)", "from ampere-altra-2-2.usa.Arm.com (unknown [10.118.91.160])\n by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A0D093F7F4;\n Tue, 31 Oct 2023 11:11:15 -0700 (PDT)" ], "From": "Paul Szczepanek <paul.szczepanek@arm.com>", "To": "dev@dpdk.org", "Cc": "Paul Szczepanek <paul.szczepanek@arm.com>,\n Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>", "Subject": "[PATCH v3 3/3] docs: add pointer compression to the EAL guide", "Date": "Tue, 31 Oct 2023 18:10:46 +0000", "Message-Id": "<20231031181046.1203681-4-paul.szczepanek@arm.com>", "X-Mailer": "git-send-email 2.25.1", "In-Reply-To": "<20231031181046.1203681-1-paul.szczepanek@arm.com>", "References": "<20230927150854.3670391-2-paul.szczepanek@arm.com>\n <20231031181046.1203681-1-paul.szczepanek@arm.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Documentation added in the EAL guide for the new\nutility functions for pointer compression\nshowing example code and potential usecases\n\nSigned-off-by: Paul Szczepanek <paul.szczepanek@arm.com>\nReviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>\n---\n .../prog_guide/env_abstraction_layer.rst | 142 ++++++++++++++++++\n 1 file changed, 142 insertions(+)\n\n--\n2.25.1", "diff": "diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst\nindex 89014789de..88cf1f16e2 100644\n--- a/doc/guides/prog_guide/env_abstraction_layer.rst\n+++ b/doc/guides/prog_guide/env_abstraction_layer.rst\n@@ -1192,3 +1192,145 @@ will not be deallocated.\n\n Any successful deallocation event will trigger a callback, for which user\n applications and other DPDK subsystems can register.\n+\n+.. _pointer_compression:\n+\n+Pointer Compression\n+-------------------\n+\n+Use ``rte_ptr_compress_16()`` and ``rte_ptr_decompress_16()`` to compress and\n+decompress pointers into 16-bit offsets. Use ``rte_ptr_compress_32()`` and\n+``rte_ptr_decompress_32()`` to compress and decompress pointers into 32-bit\n+offsets.\n+\n+Compression takes advantage of the fact that pointers are usually located in a\n+limited memory region (like a mempool). By converting them to offsets from a\n+base memory address they can be stored in fewer bytes. How many bytes are needed\n+to store the offset is dictated by the memory region size and alignment of\n+objects the pointers point to.\n+\n+For example, a pointer which is part of a 4GB memory pool can be stored as 32\n+bit offset. If the pointer points to memory that is 8 bytes aligned then 3 bits\n+can be dropped from the offset and a 32GB memory pool can now fit in 32 bits.\n+\n+For performance reasons these requirements are not enforced programmatically.\n+The programmer is responsible for ensuring that the combination of distance\n+from the base pointer and memory alignment allow for storing of the offset in\n+the number of bits indicated by the function name (16 or 32). Start of mempool\n+memory would be a good candidate for the base pointer. Otherwise any pointer\n+that precedes all pointers, is close enough and has the same alignment as the\n+pointers being compressed will work.\n+\n+.. note::\n+\n+ Performance gains depend on the batch size of pointers and CPU capabilities\n+ such as vector extensions. It's important to measure the performance\n+ increase on target hardware. A test called ``ring_perf_autotest`` in\n+ ``dpdk-test`` can provide the measurements.\n+\n+Example usage\n+~~~~~~~~~~~~~\n+\n+In this example we send pointers between two cores through a ring. While this\n+is a realistic use case the code is simplified for demonstration purposes and\n+does not have error handling.\n+\n+.. code-block:: c\n+\n+ #include <rte_launch.h>\n+ #include <rte_ring.h>\n+ #include <rte_ring_elem.h>\n+ #include <rte_ptr_compress.h>\n+\n+ #define ITEMS_ARRAY_SIZE (1024)\n+ #define BATCH_SIZE (128)\n+ #define ALIGN_EXPONENT (3)\n+ #define ITEM_ALIGN (1<<ALIGN_EXPONENT)\n+ #define CORE_SEND (1)\n+ #define CORE_RECV (2)\n+\n+ struct item {\n+ int a;\n+ } __rte_aligned(ITEM_ALIGN);\n+\n+ static struct item items[ITEMS_ARRAY_SIZE] = {0};\n+ static struct rte_ring *ring = NULL;\n+\n+ static int\n+ send_compressed(void *args)\n+ {\n+ struct item *ptrs_send[BATCH_SIZE] = {0};\n+ unsigned int n_send = 0;\n+ struct rte_ring_zc_data zcd = {0};\n+\n+ /* in this toy example we only fill the ptrs_send once and re-use */\n+ for (;n_send < BATCH_SIZE; n_send++)\n+ ptrs_send[n_send] = &items[n_send];\n+\n+ for(;;) {\n+ n_send = rte_ring_enqueue_zc_burst_elem_start(\n+ ring, sizeof(uint32_t), BATCH_SIZE, &zcd, NULL);\n+\n+ /* compress ptrs_send into offsets */\n+ rte_ptr_compress_32(items, /* base pointer */\n+ ptrs_send, /* source array to be compressed */\n+ zcd.ptr1, /* destination array to store offsets */\n+ zcd.n1, /* how many pointers to compress */\n+ ALIGN_EXPONENT /* how many bits can we drop from the offset */);\n+\n+ if (zcd.ptr2 != NULL)\n+ rte_ptr_compress_32(items, ptrs_send + zcd.n1,\n+ zcd.ptr2, n_send - zcd.n1, ALIGN_EXPONENT);\n+\n+ rte_ring_enqueue_zc_finish(ring, n_send);\n+ }\n+ return 1;\n+ }\n+\n+ static int\n+ recv_compressed(void *args)\n+ {\n+ struct item *ptrs_recv[BATCH_SIZE] = {0};\n+ unsigned int n_recv;\n+ struct rte_ring_zc_data zcd = {0};\n+\n+ for(;;) {\n+ /* receive compressed pointers from the ring */\n+ n_recv = rte_ring_dequeue_zc_burst_elem_start(\n+ ring, sizeof(uint32_t), BATCH_SIZE, &zcd, NULL);\n+\n+ rte_ptr_decompress_32(items, /* base pointer */\n+ zcd.ptr1, /* source array to decompress */\n+ ptrs_recv, /* destination array to store pointers */\n+ zcd.n1, /* how many pointers to decompress */\n+ ALIGN_EXPONENT /* how many bits were dropped from the offset */);\n+\n+ /* handle the potential secondary buffer (caused by ring boundary) */\n+ if (zcd.ptr2 != NULL)\n+ rte_ptr_decompress_32(items,\n+ zcd.ptr2,\n+ ptrs_recv + zcd.n1,\n+ n_recv - zcd.n1,\n+ ALIGN_EXPONENT);\n+\n+ rte_ring_dequeue_zc_finish(ring, n_recv);\n+\n+ /* ptrs_recv contains what ptrs_send contained in the other thread */\n+ /* (...) */\n+ }\n+ return 1;\n+ }\n+\n+ void\n+ compression_example(void)\n+ {\n+ ring = rte_ring_create_elem(\n+ \"COMPR_PTRS\", sizeof(uint32_t),\n+ 1024, rte_socket_id(),\n+ RING_F_SP_ENQ | RING_F_SC_DEQ);\n+\n+ rte_eal_remote_launch(send_compressed, NULL, CORE_SEND);\n+ rte_eal_remote_launch(recv_compressed, NULL, CORE_RECV);\n+\n+ for(;;) {}\n+ }\n", "prefixes": [ "v3", "3/3" ] }