Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/138112/?format=api
http://patches.dpdk.org/api/patches/138112/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/patch/20240307203943.188101-4-paul.szczepanek@arm.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20240307203943.188101-4-paul.szczepanek@arm.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20240307203943.188101-4-paul.szczepanek@arm.com", "date": "2024-03-07T20:39:42", "name": "[v8,3/4] docs: add pointer compression guide", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "bc11deaf6f39090205000427d5706fe89be0a712", "submitter": { "id": 3199, "url": "http://patches.dpdk.org/api/people/3199/?format=api", "name": "Paul Szczepanek", "email": "paul.szczepanek@arm.com" }, "delegate": { "id": 1, "url": "http://patches.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "http://patches.dpdk.org/project/dpdk/patch/20240307203943.188101-4-paul.szczepanek@arm.com/mbox/", "series": [ { "id": 31429, "url": "http://patches.dpdk.org/api/series/31429/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=31429", "date": "2024-03-07T20:39:40", "name": "[v8,1/4] ptr_compress: add pointer compression library", "version": 8, "mbox": "http://patches.dpdk.org/series/31429/mbox/" } ], "comments": "http://patches.dpdk.org/api/patches/138112/comments/", "check": "warning", "checks": "http://patches.dpdk.org/api/patches/138112/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id B202A43C21;\n\tThu, 7 Mar 2024 21:40:14 +0100 (CET)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 0C7AD42F15;\n\tThu, 7 Mar 2024 21:39:58 +0100 (CET)", "from foss.arm.com (foss.arm.com [217.140.110.172])\n by mails.dpdk.org (Postfix) with ESMTP id 83D2642EF5\n for <dev@dpdk.org>; Thu, 7 Mar 2024 21:39:53 +0100 (CET)", "from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])\n by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E17C51474;\n Thu, 7 Mar 2024 12:40:29 -0800 (PST)", "from ampere-altra-2-1.usa.Arm.com (ampere-altra-2-1.usa.arm.com\n [10.118.91.158])\n by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D5D093F762;\n Thu, 7 Mar 2024 12:39:52 -0800 (PST)" ], "From": "Paul Szczepanek <paul.szczepanek@arm.com>", "To": "dev@dpdk.org", "Cc": "Paul Szczepanek <paul.szczepanek@arm.com>,\n Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>,\n Nathan Brown <Nathan.Brown@arm.com>", "Subject": "[PATCH v8 3/4] docs: add pointer compression guide", "Date": "Thu, 7 Mar 2024 20:39:42 +0000", "Message-Id": "<20240307203943.188101-4-paul.szczepanek@arm.com>", "X-Mailer": "git-send-email 2.25.1", "In-Reply-To": "<20240307203943.188101-1-paul.szczepanek@arm.com>", "References": "<20230927150854.3670391-2-paul.szczepanek@arm.com>\n <20240307203943.188101-1-paul.szczepanek@arm.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Documentation added in the prog guide for the new\nutility functions for pointer compression\nshowing example code and potential usecases.\n\nSigned-off-by: Paul Szczepanek <paul.szczepanek@arm.com>\nReviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>\nReviewed-by: Nathan Brown <Nathan.Brown@arm.com>\n---\n doc/guides/prog_guide/ptr_compress_lib.rst | 144 +++++++++++++++++++++\n 1 file changed, 144 insertions(+)\n create mode 100644 doc/guides/prog_guide/ptr_compress_lib.rst\n\n--\n2.25.1", "diff": "diff --git a/doc/guides/prog_guide/ptr_compress_lib.rst b/doc/guides/prog_guide/ptr_compress_lib.rst\nnew file mode 100644\nindex 0000000000..00bc8a3697\n--- /dev/null\n+++ b/doc/guides/prog_guide/ptr_compress_lib.rst\n@@ -0,0 +1,144 @@\n+.. SPDX-License-Identifier: BSD-3-Clause\n+ Copyright(c) 2024 Arm Limited.\n+\n+.. _Pointer_Compression_Library:\n+\n+Pointer Compression Library\n+===========================\n+\n+Use ``rte_ptr_compress_16()`` and ``rte_ptr_decompress_16()`` to compress and\n+decompress pointers into 16-bit offsets. Use ``rte_ptr_compress_32()`` and\n+``rte_ptr_decompress_32()`` to compress and decompress pointers into 32-bit\n+offsets.\n+\n+Compression takes advantage of the fact that pointers are usually located in a\n+limited memory region (like a mempool). By converting them to offsets from a\n+base memory address they can be stored in fewer bytes. How many bytes are needed\n+to store the offset is dictated by the memory region size and alignment of\n+objects the pointers point to.\n+\n+For example, a pointer which is part of a 4GB memory pool can be stored as 32\n+bit offset. If the pointer points to memory that is 8 bytes aligned then 3 bits\n+can be dropped from the offset and a 32GB memory pool can now fit in 32 bits.\n+\n+For performance reasons these requirements are not enforced programmatically.\n+The programmer is responsible for ensuring that the combination of distance\n+from the base pointer and memory alignment allow for storing of the offset in\n+the number of bits indicated by the function name (16 or 32). Start of mempool\n+memory would be a good candidate for the base pointer. Otherwise any pointer\n+that precedes all pointers, is close enough and has the same alignment as the\n+pointers being compressed will work.\n+\n+.. note::\n+\n+ Performance gains depend on the batch size of pointers and CPU capabilities\n+ such as vector extensions. It's important to measure the performance\n+ increase on target hardware. A test called ``ring_perf_autotest`` in\n+ ``dpdk-test`` can provide the measurements.\n+\n+Example usage\n+~~~~~~~~~~~~~\n+\n+In this example we send pointers between two cores through a ring. While this\n+is a realistic use case the code is simplified for demonstration purposes and\n+does not have error handling.\n+\n+.. code-block:: c\n+\n+ #include <rte_launch.h>\n+ #include <rte_ring.h>\n+ #include <rte_ring_elem.h>\n+ #include <rte_ptr_compress.h>\n+\n+ #define ITEMS_ARRAY_SIZE (1024)\n+ #define BATCH_SIZE (128)\n+ #define ALIGN_EXPONENT (3)\n+ #define ITEM_ALIGN (1<<ALIGN_EXPONENT)\n+ #define CORE_SEND (1)\n+ #define CORE_RECV (2)\n+\n+ struct item {\n+ int a;\n+ } __rte_aligned(ITEM_ALIGN);\n+\n+ static struct item items[ITEMS_ARRAY_SIZE] = {0};\n+ static struct rte_ring *ring = NULL;\n+\n+ static int\n+ send_compressed(void *args)\n+ {\n+ struct item *ptrs_send[BATCH_SIZE] = {0};\n+ unsigned int n_send = 0;\n+ struct rte_ring_zc_data zcd = {0};\n+\n+ /* in this example we only fill the ptrs_send once and reuse */\n+ for (;n_send < BATCH_SIZE; n_send++)\n+ ptrs_send[n_send] = &items[n_send];\n+\n+ for(;;) {\n+ n_send = rte_ring_enqueue_zc_burst_elem_start(\n+ ring, sizeof(uint32_t), BATCH_SIZE, &zcd, NULL);\n+\n+ /* compress ptrs_send into offsets */\n+ rte_ptr_compress_32(items, /* base pointer */\n+ ptrs_send, /* source array to be compressed */\n+ zcd.ptr1, /* destination array to store offsets */\n+ zcd.n1, /* how many pointers to compress */\n+ ALIGN_EXPONENT /* how many bits can we drop from the offset */);\n+\n+ if (zcd.ptr2 != NULL)\n+ rte_ptr_compress_32(items, ptrs_send + zcd.n1,\n+ zcd.ptr2, n_send - zcd.n1, ALIGN_EXPONENT);\n+\n+ rte_ring_enqueue_zc_finish(ring, n_send);\n+ }\n+ return 1;\n+ }\n+\n+ static int\n+ recv_compressed(void *args)\n+ {\n+ struct item *ptrs_recv[BATCH_SIZE] = {0};\n+ unsigned int n_recv;\n+ struct rte_ring_zc_data zcd = {0};\n+\n+ for(;;) {\n+ /* receive compressed pointers from the ring */\n+ n_recv = rte_ring_dequeue_zc_burst_elem_start(\n+ ring, sizeof(uint32_t), BATCH_SIZE, &zcd, NULL);\n+\n+ rte_ptr_decompress_32(items, /* base pointer */\n+ zcd.ptr1, /* source array to decompress */\n+ ptrs_recv, /* destination array to store pointers */\n+ zcd.n1, /* how many pointers to decompress */\n+ ALIGN_EXPONENT /* how many bits were dropped from the offset */);\n+\n+ /* handle the potential secondary buffer (caused by ring boundary) */\n+ if (zcd.ptr2 != NULL)\n+ rte_ptr_decompress_32(items,\n+ zcd.ptr2,\n+ ptrs_recv + zcd.n1,\n+ n_recv - zcd.n1,\n+ ALIGN_EXPONENT);\n+\n+ rte_ring_dequeue_zc_finish(ring, n_recv);\n+\n+ /* ptrs_recv contains what ptrs_send contained in the other thread */\n+ /* (...) */\n+ }\n+ return 1;\n+ }\n+\n+ void\n+ compression_example(void)\n+ {\n+ ring = rte_ring_create_elem(\n+ \"COMPR_PTRS\", sizeof(uint32_t),\n+ 1024, rte_socket_id(),\n+ RING_F_SP_ENQ | RING_F_SC_DEQ);\n+\n+ rte_eal_remote_launch(send_compressed, NULL, CORE_SEND);\n+ rte_eal_remote_launch(recv_compressed, NULL, CORE_RECV);\n+\n+ for(;;) {}\n+ }\n", "prefixes": [ "v8", "3/4" ] }{ "id": 138112, "url": "