Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/5714/?format=api
{ "id": 5714, "url": "https://patches.dpdk.org/api/patches/5714/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/1435088014-18973-2-git-send-email-nhorman@tuxdriver.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<1435088014-18973-2-git-send-email-nhorman@tuxdriver.com>", "list_archive_url": "https://inbox.dpdk.org/dev/1435088014-18973-2-git-send-email-nhorman@tuxdriver.com", "date": "2015-06-23T19:33:34", "name": "[dpdk-dev,2/2] ABI: Add some documentation", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "3d5393b9fa9f869b794d4c5c463cc44ad75fcf4c", "submitter": { "id": 32, "url": "https://patches.dpdk.org/api/people/32/?format=api", "name": "Neil Horman", "email": "nhorman@tuxdriver.com" }, "delegate": null, "mbox": "https://patches.dpdk.org/project/dpdk/patch/1435088014-18973-2-git-send-email-nhorman@tuxdriver.com/mbox/", "series": [], "comments": "https://patches.dpdk.org/api/patches/5714/comments/", "check": "pending", "checks": "https://patches.dpdk.org/api/patches/5714/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 2402AC6EC;\n\tTue, 23 Jun 2015 21:34:05 +0200 (CEST)", "from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58])\n\tby dpdk.org (Postfix) with ESMTP id 9892CC6DE\n\tfor <dev@dpdk.org>; Tue, 23 Jun 2015 21:34:03 +0200 (CEST)", "from hmsreliant.think-freely.org\n\t([2001:470:8:a08:7aac:c0ff:fec2:933b] helo=localhost)\n\tby smtp.tuxdriver.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63)\n\t(envelope-from <nhorman@tuxdriver.com>)\n\tid 1Z7TxE-0005uX-3L; Tue, 23 Jun 2015 15:34:01 -0400" ], "From": "Neil Horman <nhorman@tuxdriver.com>", "To": "dev@dpdk.org", "Date": "Tue, 23 Jun 2015 15:33:34 -0400", "Message-Id": "<1435088014-18973-2-git-send-email-nhorman@tuxdriver.com>", "X-Mailer": "git-send-email 2.1.0", "In-Reply-To": "<1435088014-18973-1-git-send-email-nhorman@tuxdriver.com>", "References": "<1435088014-18973-1-git-send-email-nhorman@tuxdriver.com>", "X-Spam-Score": "-2.9 (--)", "X-Spam-Status": "No", "Subject": "[dpdk-dev] [PATCH 2/2] ABI: Add some documentation", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "patches and discussions about DPDK <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "People have been asking for ways to use the ABI macros, heres some docs to\nclarify their use. Included is:\n\n* An overview of what ABI is\n* Details of the ABI deprecation process\n* Details of the versioning macros\n* Examples of their use\n* Details of how to use the ABI validator\n\nThanks to John Mcnamara, who duplicated much of this effort at Intel while I was\nworking on it. Much of the introductory material was gathered and cleaned up by\nhim\n\nSigned-off-by: Neil Horman <nhorman@tuxdriver.com>\nCC: john.mcnamara@intel.com\nCC: thomas.monjalon@6wind.com\n---\n doc/guides/guidelines/index.rst | 1 +\n doc/guides/guidelines/versioning.rst | 456 +++++++++++++++++++++++++++++++++++\n 2 files changed, 457 insertions(+)\n create mode 100644 doc/guides/guidelines/versioning.rst", "diff": "diff --git a/doc/guides/guidelines/index.rst b/doc/guides/guidelines/index.rst\nindex b2b0a92..251062e 100644\n--- a/doc/guides/guidelines/index.rst\n+++ b/doc/guides/guidelines/index.rst\n@@ -6,3 +6,4 @@ Guidelines\n :numbered:\n \n coding_style\n+ versioning\ndiff --git a/doc/guides/guidelines/versioning.rst b/doc/guides/guidelines/versioning.rst\nnew file mode 100644\nindex 0000000..79af924\n--- /dev/null\n+++ b/doc/guides/guidelines/versioning.rst\n@@ -0,0 +1,456 @@\n+Managing ABI updates\n+====================\n+\n+Description\n+-----------\n+\n+This document details some methods for handling ABI management in the DPDK.\n+Note this document is not exhaustive, in that C library versioning is flexible\n+allowing multiple methods to acheive various goals, but it will provide the user\n+with some introductory methods\n+\n+General Guidelines\n+------------------\n+\n+#. Whenever possible, ABI should be preserved\n+#. The addition of symbols is generally not problematic\n+#. The modification of symbols can generally be managed with versioning\n+#. The removal of symbols generally is an ABI break and requires bumping of the\n+ LIBABIVER macro\n+\n+What is an ABI\n+--------------\n+\n+An ABI (Application Binary Interface) is the set of runtime interfaces exposed\n+by a library. It is similar to an API (Application Programming Interface) but\n+is the result of compilation. It is also effectively cloned when applications\n+link to dynamic libraries. That is to say when an application is compiled to\n+link against dynamic libraries, it is assumed that the ABI remains constant\n+between the time the application is compiled/linked, and the time that it runs.\n+Therefore, in the case of dynamic linking, it is critical that an ABI is\n+preserved, or (when modified), done in such a way that the application is unable\n+to behave improperly or in an unexpected fashion.\n+\n+The DPDK ABI policy\n+-------------------\n+\n+ABI versions are set at the time of major release labeling, and the ABI may\n+change multiple times, without warning, between the last release label and the\n+HEAD label of the git tree.\n+\n+ABI versions, once released, are available until such time as their\n+deprecation has been noted in the Release Notes for at least one major release\n+cycle. For example consider the case where the ABI for DPDK 2.0 has been\n+shipped and then a decision is made to modify it during the development of\n+DPDK 2.1. The decision will be recorded in the Release Notes for the DPDK 2.1\n+release and the modification will be made available in the DPDK 2.2 release.\n+\n+ABI versions may be deprecated in whole or in part as needed by a given\n+update.\n+\n+Some ABI changes may be too significant to reasonably maintain multiple\n+versions. In those cases ABI's may be updated without backward compatibility\n+being provided. The requirements for doing so are:\n+\n+#. At least 3 acknowledgments of the need to do so must be made on the\n+ dpdk.org mailing list.\n+\n+#. A full deprecation cycle, as explained above, must be made to offer\n+ downstream consumers sufficient warning of the change.\n+\n+#. The ``LIBABIVER`` variable in the makefile(s) where the ABI changes are\n+ incorporated must be incremented in parallel with the ABI changes\n+ themselves.\n+\n+Note that the above process for ABI deprecation should not be undertaken\n+lightly. ABI stability is extremely important for downstream consumers of the\n+DPDK, especially when distributed in shared object form. Every effort should\n+be made to preserve the ABI whenever possible. The ABI should only be changed\n+for significant reasons, such as performance enhancements. ABI breakage due to\n+changes such as reorganizing public structure fields for aesthetic or\n+readability purposes should be avoided.\n+\n+Examples of Deprecation Notices\n+-------------------------------\n+\n+The following are some examples of ABI deprecation notices which would be\n+added to the Release Notes:\n+\n+* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,\n+ to be replaced with the inline function ``rte_foo()``.\n+\n+* The function ``rte_mbuf_grok()`` has been updated to include a new parameter\n+ in version 2.0. Backwards compatibility will be maintained for this function\n+ until the release of version 2.1\n+\n+* The members of ``struct rte_foo`` have been reorganized in release 2.0 for\n+ performance reasons. Existing binary applications will have backwards\n+ compatibility in release 2.0, while newly built binaries will need to\n+ reference the new structure variant ``struct rte_foo2``. Compatibility will\n+ be removed in release 2.2, and all applications will require updating and\n+ rebuilding to the new structure at that time, which will be renamed to the\n+ original ``struct rte_foo``.\n+\n+* Significant ABI changes are planned for the ``librte_dostuff`` library. The\n+ upcoming release 2.0 will not contain these changes, but release 2.1 will,\n+ and no backwards compatibility is planned due to the extensive nature of\n+ these changes. Binaries using this library built prior to version 2.1 will\n+ require updating and recompilation.\n+\n+Versioning Macros\n+-----------------\n+\n+When a symbol is exported from a library to provide an API, it also provides a\n+calling convention (ABI) that is embodied in its name, return type and\n+arguments. Occasionally that function may need to change to accommodate new\n+functionality or behavior. When that occurs, it is desirable to allow for\n+backward compatibility for a time with older binaries that are dynamically\n+linked to the DPDK.\n+\n+To support backward compatibility the ``lib/librte_compat/rte_compat.h``\n+header file provides macros to use when updating exported functions. These\n+macros are used in conjunction with the ``rte_<library>_version.map`` file for\n+a given library to allow multiple versions of a symbol to exist in a shared\n+library so that older binaries need not be immediately recompiled.\n+\n+The macros exported are:\n+\n+* ``VERSION_SYMBOL(b, e, n)``: Creates a symbol version table entry binding\n+ unversioned symbol ``b`` to the internal function ``b_e``.\n+\n+\n+* ``BASE_SYMBOL(b, e)``: Creates a symbol version table entry binding\n+ unversioned symbol ``b`` to the internal function ``b_e``.\n+\n+* ``BIND_DEFAULT_SYMBOL(b, e, n)``: Creates a symbol version entry instructing\n+ the linker to bind references to symbol ``b`` to the internal symbol\n+ ``b_e``.\n+\n+\n+Examples of ABI Macro use\n+-------------------------\n+\n+Updating a public API\n+~~~~~~~~~~~~~~~~~~~~~\n+\n+Assume we have a function as follows\n+\n+.. code-block:: c\n+\n+ /*\n+ * Create an acl context object for apps to \n+ * manipulate\n+ */\n+ struct rte_acl_ctx *\n+ rte_acl_create(const struct rte_acl_param *param)\n+ {\n+ ...\n+ }\n+\n+\n+Assume that struct rte_acl_ctx is a private structure, and that a developer\n+wishes to enhance the acl api so that a debugging flag can be enabled on a\n+per-context basis. This requires an addition to the structure (which, being\n+private, is safe), but it also requries modifying the code as follows\n+\n+.. code-block:: c\n+\n+ /*\n+ * Create an acl context object for apps to \n+ * manipulate\n+ */\n+ struct rte_acl_ctx *\n+ rte_acl_create(const struct rte_acl_param *param, int debug)\n+ {\n+ ...\n+ }\n+\n+\n+Note also that, being a public function, the header file prototype must also be\n+changed, as must all the call sites, to reflect the new ABI footprint. We will\n+maintain previous ABI versions that are accessible only to previously compiled\n+binaries\n+\n+The addition of a parameter to the function is ABI breaking as the function is\n+public, and existing application may use it in its current form. However, the\n+compatibility macros in DPDK alow a developer to use symbol versioning so that\n+multiple functions can be mapped to the same public symbol based on when an\n+application was linked to it. To see how this is done, we start with the\n+requisite libraries version map file. Initially the version map file for the\n+acl library looks like this\n+\n+.. code-block:: none \n+\n+ DPDK_2.0 {\n+ global:\n+\n+ rte_acl_add_rules;\n+ rte_acl_build;\n+ rte_acl_classify;\n+ rte_acl_classify_alg;\n+ rte_acl_classify_scalar;\n+ rte_acl_create;\n+ rte_acl_dump;\n+ rte_acl_find_existing;\n+ rte_acl_free;\n+ rte_acl_ipv4vlan_add_rules;\n+ rte_acl_ipv4vlan_build;\n+ rte_acl_list_dump;\n+ rte_acl_reset;\n+ rte_acl_reset_rules;\n+ rte_acl_set_ctx_classify;\n+\n+ local: *;\n+ };\n+\n+This file needs to be modified as follows\n+\n+.. code-block:: none\n+\n+ DPDK_2.0 {\n+ global:\n+\n+ rte_acl_add_rules;\n+ rte_acl_build;\n+ rte_acl_classify;\n+ rte_acl_classify_alg;\n+ rte_acl_classify_scalar;\n+ rte_acl_create;\n+ rte_acl_dump;\n+ rte_acl_find_existing;\n+ rte_acl_free;\n+ rte_acl_ipv4vlan_add_rules;\n+ rte_acl_ipv4vlan_build;\n+ rte_acl_list_dump;\n+ rte_acl_reset;\n+ rte_acl_reset_rules;\n+ rte_acl_set_ctx_classify;\n+\n+ local: *;\n+ };\n+\n+ DPDK_2.1 {\n+ global:\n+ rte_acl_create;\n+\n+ } DPDK_2.0;\n+\n+The addition of the new block tells the linker that a new version node is\n+available (DPDK_2.1), whcih contains the symbol rte_acl_create, and inherits the\n+symbols from the DPDK_2.0 node. This list is directly translated into a list of\n+exported symbols when DPDK is compiled as a shared library\n+\n+Next, we need to specify in the code which function map to the rte_acl_create\n+symbol at which versions. First, at the site of the initial symbol definition,\n+we need to update the function so that it is uniquely named, and not in conflict\n+with the public symbol name\n+\n+.. code-block:: c\n+\n+ struct rte_acl_ctx *\n+ -rte_acl_create(const struct rte_acl_param *param)\n+ +rte_acl_create_v20(const struct rte_acl_param *param)\n+ {\n+ size_t sz;\n+ struct rte_acl_ctx *ctx;\n+ ...\n+\n+Note that the base name of the symbol was kept in tact, as this is condusive to\n+the macros used for versioning symbols. That is our next step, mapping this new\n+symbol name to the initial symbol name at version node 2.0. Immediately after\n+the function, we add this line of code\n+\n+.. code-block:: c\n+\n+ VERSION_SYMBOL(rte_acl_create, _v20, 2.0);VERSION_SYMBOL(rte_acl_create, _v20, 2.0);\n+\n+Remembering to also add the rte_compat.h header to the requisite c file where\n+these changes are being made. The above macro instructs the linker to create a\n+new symbol rte_acl_create@DPDK_2.0, which matches the symbol created in older\n+builds, but now points to the above newly named function. We have now mapped\n+the origional rte_acl_create symbol to the origional function (but with a new\n+name)\n+\n+Next, we need to create the 2.1 version of the symbol. We create a new function\n+name, with a different suffix, and implement it appropriately\n+\n+.. code-block:: c\n+\n+ struct rte_acl_ctx *\n+ rte_acl_create_v21(const struct rte_acl_param *param, int debug);\n+ {\n+ struct rte_acl_ctx *ctx = rte_acl_create_v20(param);\n+\n+ ctx->debug = debug;\n+\n+ return ctx;\n+ }\n+\n+This code serves as our new API call. Its the same as our old call, but adds\n+the new parameter in place. Next we need to map this function to the symbol\n+rte_acl_create@DPDK_2.1. To do this, we modify the public prototype of the call\n+in the header file, adding the macro there to inform all including applications,\n+that on re-link, the default rte_acl_create symbol should point to this\n+function. Note that we could do this by simply naming the fuction above\n+rte_acl_create, and the linker would chose the most recent version tag to apply\n+in the version script, but we can also do this in the header file\n+\n+.. code-block:: c\n+\n+ struct rte_acl_ctx *\n+ -rte_acl_create(const struct rte_acl_param *param);\n+ +rte_acl_create(const struct rte_acl_param *param, int debug);\n+ +BIND_DEFAULT_SYMBOL(rte_acl_create, _v21, 2.1);\n+\n+The BIND_DEFAULT_SYMBOL macro explicitly tells applications that include this\n+header, to link to the rte_acl_create_v21 function and apply the DPDK_2.1\n+version node to it. This method is more explicit and flexible than just\n+reimplementing the exact symbol name, and allows for other features (such as\n+linking to the old symbol version by default, when the new ABI is to be opt-in\n+for a period.\n+\n+Thats it, on the next shared library rebuild, there will be two versions of\n+rte_acl_create, an old DPDK_2.0 version, used by previously built applications,\n+and a new DPDK_2.1 version, used by future built applications.\n+\n+\n+Deprecating part of a public API\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+Lets assume that you've done the above update, and after a few releases have\n+passed you decide you would like to retire the old version of the function.\n+After having gone through the ABI deprecation annoucement process, removal is\n+easy. Start by removing the symbol from the requisite version map file:\n+\n+.. code-block:: none\n+\n+ DPDK_2.0 {\n+ global:\n+\n+ rte_acl_add_rules;\n+ rte_acl_build;\n+ rte_acl_classify;\n+ rte_acl_classify_alg;\n+ rte_acl_classify_scalar;\n+ rte_acl_dump;\n+ - rte_acl_create\n+ rte_acl_find_existing;\n+ rte_acl_free;\n+ rte_acl_ipv4vlan_add_rules;\n+ rte_acl_ipv4vlan_build;\n+ rte_acl_list_dump;\n+ rte_acl_reset;\n+ rte_acl_reset_rules;\n+ rte_acl_set_ctx_classify;\n+\n+ local: *;\n+ };\n+\n+ DPDK_2.1 {\n+ global:\n+ rte_acl_create;\n+ } DPDK_2.0;\n+\n+\n+Next remove the corresponding versioned export\n+.. code-block:: c\n+\n+ -VERSION_SYMBOL(rte_acl_create, _v20, 2.0);\n+\n+\n+Note that the internal function definition could also be removed, but its used\n+in our example by the newer version _v21, so we leave it in place. This is a\n+coding style choice.\n+\n+Lastly, we need to bump the LIBABIVER number for this library in the Makefile to\n+indicate to applications doing dynamic linking that this is a later, and\n+possibly incompatible library version:\n+\n+.. code-block:: c\n+\n+ -LIBABIVER := 1\n+ +LIBABIVER := 2\n+\n+Deprecating an entire ABI version\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+While removing a symbol from and ABI may be useful, it is often more practical\n+to remove an entire version node at once. If a version node completely\n+specifies an API, then removing part of it, typically makes it incomplete. In\n+those cases it is better to remove the entire node\n+ \n+To do this, start by modifying the version map file, such that all symbols from\n+the node to be removed are merged into the next node in the map\n+\n+In the case of our map above, it would transform to look as follows\n+\n+.. code-block:: none\n+\n+ DPDK_2.1 { \n+ global:\n+ \n+ rte_acl_add_rules;\n+ rte_acl_build;\n+ rte_acl_classify;\n+ rte_acl_classify_alg;\n+ rte_acl_classify_scalar;\n+ rte_acl_dump;\n+ rte_acl_create\n+ rte_acl_find_existing;\n+ rte_acl_free;\n+ rte_acl_ipv4vlan_add_rules;\n+ rte_acl_ipv4vlan_build;\n+ rte_acl_list_dump;\n+ rte_acl_reset;\n+ rte_acl_reset_rules;\n+ rte_acl_set_ctx_classify;\n+ \n+ local: *;\n+ }; \n+\n+Then any uses of BIND_DEFAULT_SYMBOL that ponited to the old node should be\n+updated to point to the new version node in any header files for all affected\n+symbols.\n+\n+.. code-block:: c\n+\n+ -BIND_DEFAULT_SYMBOL(rte_acl_create, _v20, 2.0);\n+ +BIND_DEFAULT_SYMBOL(rte_acl_create, _v21, 2.1);\n+\n+Lastly, any VERSION_SYMBOL macros that point to the old version node should be\n+removed, taking care to keep, where neeed old code in place to support newer\n+versions of the symbol.\n+\n+Running the ABI Validator\n+-------------------------\n+\n+The ``scripts`` directory in the DPDK source tree contains a utility program,\n+``validate-abi.sh``, for validating the DPDK ABI based on the Linux `ABI\n+Compliance Checker\n+<http://ispras.linuxbase.org/index.php/ABI_compliance_checker>`_.\n+\n+This has a dependency on the ``abi-compliance-checker`` and ``and abi-dumper``\n+utilities which can be installed via a package manager. For example::\n+\n+ sudo yum install abi-compliance-checker\n+ sudo yum install abi-dumper\n+\n+The syntax of the ``validate-abi.sh`` utility is::\n+\n+ ./scripts/validate-abi.sh <TAG1> <TAG2> <TARGET>\n+\n+Where ``TAG1`` and ``TAG2`` are valid git tags on the local repo and target is\n+the usual DPDK compilation target.\n+\n+For example to test the current committed HEAD against a previous release tag\n+we could add a temporary tag and run the utility as follows::\n+\n+ git tag MY_TEMP_TAG\n+ ./scripts/validate-abi.sh v2.0.0 MY_TEMP_TAG x86_64-native-linuxapp-gcc\n+\n+After the validation script completes (it can take a while since it need to\n+compile both tags) it will create compatibility reports in the\n+``./compat_report`` directory. Listed incompatibilities can be found as\n+follows::\n+\n+ grep -lr Incompatible compat_reports/\n", "prefixes": [ "dpdk-dev", "2/2" ] }