Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/68907/?format=api
{ "id": 68907, "url": "https://patches.dpdk.org/api/patches/68907/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/20200419100133.3232316-34-jerinj@marvell.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20200419100133.3232316-34-jerinj@marvell.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20200419100133.3232316-34-jerinj@marvell.com", "date": "2020-04-19T10:01:33", "name": "[v6,33/33] doc: add trace library guide", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "4b824c9daa17ed7ce0b0be29f9bf1648df39a6b4", "submitter": { "id": 1188, "url": "https://patches.dpdk.org/api/people/1188/?format=api", "name": "Jerin Jacob Kollanukkaran", "email": "jerinj@marvell.com" }, "delegate": { "id": 24651, "url": "https://patches.dpdk.org/api/users/24651/?format=api", "username": "dmarchand", "first_name": "David", "last_name": "Marchand", "email": "david.marchand@redhat.com" }, "mbox": "https://patches.dpdk.org/project/dpdk/patch/20200419100133.3232316-34-jerinj@marvell.com/mbox/", "series": [ { "id": 9495, "url": "https://patches.dpdk.org/api/series/9495/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/list/?series=9495", "date": "2020-04-19T10:01:02", "name": "DPDK Trace support", "version": 6, "mbox": "https://patches.dpdk.org/series/9495/mbox/" } ], "comments": "https://patches.dpdk.org/api/patches/68907/comments/", "check": "success", "checks": "https://patches.dpdk.org/api/patches/68907/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from dpdk.org (dpdk.org [92.243.14.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id D91C5A0561;\n\tSun, 19 Apr 2020 12:12:11 +0200 (CEST)", "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id D488A1D698;\n\tSun, 19 Apr 2020 12:04:02 +0200 (CEST)", "from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com\n [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 24DC81D5C0\n for <dev@dpdk.org>; Sun, 19 Apr 2020 12:03:50 +0200 (CEST)", "from pps.filterd (m0045849.ppops.net [127.0.0.1])\n by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id\n 03J9wnD4032579; Sun, 19 Apr 2020 03:03:49 -0700", "from sc-exch03.marvell.com ([199.233.58.183])\n by mx0a-0016f401.pphosted.com with ESMTP id 30fxwp3d9p-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT);\n Sun, 19 Apr 2020 03:03:49 -0700", "from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH03.marvell.com\n (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2;\n Sun, 19 Apr 2020 03:03:47 -0700", "from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com\n (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2;\n Sun, 19 Apr 2020 03:03:46 -0700", "from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com\n (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend\n Transport; Sun, 19 Apr 2020 03:03:46 -0700", "from jerin-lab.marvell.com (jerin-lab.marvell.com [10.28.34.14])\n by maili.marvell.com (Postfix) with ESMTP id F3C8C3F703F;\n Sun, 19 Apr 2020 03:03:43 -0700 (PDT)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com;\n h=from : to : cc :\n subject : date : message-id : in-reply-to : references : mime-version :\n content-transfer-encoding : content-type; s=pfpt0818;\n bh=xs+iEC9gohLSI3mAcEq0XqImRrSNxDW/q6PiCwpSeho=;\n b=QCWidgvzYO1osxNTE5XtKxHtI486KQ7ISA/GuSpqS93wkRDoW8V0kg5sKINM8wPDZaEm\n DyDzTnNe98UB8UO+3jtMEIdP6xGQkYx5hD7P5KuiHUmXf4C0I87KMTsvKNV3UN7EFlHL\n yS1GCwXqjUjuIIXD1LMKSecHxFhEjupLVEhYg7lhnF97KXJ08aifqg+V4FZfmHm5TPW2\n Sv5dVPAudZLC8sCDd2niApBXiaPqUOcgTYUuX7Hi2ROqQhFGDRovA/Gp4aR7QMgbLabX\n 1B/8ovRMLHv+BXE3xw2MMWAwQn+SEXbx1BMc5JIdavqUWeemrBbij09nCyir8iw2vkk6 qw==", "From": "<jerinj@marvell.com>", "To": "Thomas Monjalon <thomas@monjalon.net>, John McNamara\n <john.mcnamara@intel.com>, Marko Kovacevic <marko.kovacevic@intel.com>,\n \"Jerin Jacob\" <jerinj@marvell.com>, Sunil Kumar Kori <skori@marvell.com>", "CC": "<dev@dpdk.org>, <bruce.richardson@intel.com>, <david.marchand@redhat.com>,\n <mattias.ronnblom@ericsson.com>", "Date": "Sun, 19 Apr 2020 15:31:33 +0530", "Message-ID": "<20200419100133.3232316-34-jerinj@marvell.com>", "X-Mailer": "git-send-email 2.25.1", "In-Reply-To": "<20200419100133.3232316-1-jerinj@marvell.com>", "References": "<20200413150116.734047-1-jerinj@marvell.com>\n <20200419100133.3232316-1-jerinj@marvell.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "Content-Type": "text/plain", "X-Proofpoint-Virus-Version": "vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676\n definitions=2020-04-19_02:2020-04-17,\n 2020-04-19 signatures=0", "Subject": "[dpdk-dev] [PATCH v6 33/33] doc: add trace library guide", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "From: Jerin Jacob <jerinj@marvell.com>\n\nAdd programmar's guide for trace library support.\n\nSigned-off-by: Jerin Jacob <jerinj@marvell.com>\nSigned-off-by: Sunil Kumar Kori <skori@marvell.com>\n---\n MAINTAINERS | 1 +\n doc/guides/prog_guide/index.rst | 1 +\n doc/guides/prog_guide/trace_lib.rst | 346 +++++++++++++++++++++++++\n doc/guides/rel_notes/release_20_05.rst | 9 +\n 4 files changed, 357 insertions(+)\n create mode 100644 doc/guides/prog_guide/trace_lib.rst", "diff": "diff --git a/MAINTAINERS b/MAINTAINERS\nindex 4592abede..b08ffd482 100644\n--- a/MAINTAINERS\n+++ b/MAINTAINERS\n@@ -200,6 +200,7 @@ M: Sunil Kumar Kori <skori@marvell.com>\n F: lib/librte_eal/include/rte_trace*.h\n F: lib/librte_eal/common/eal_common_trace*.c\n F: lib/librte_eal/common/eal_trace.h\n+F: doc/guides/prog_guide/trace_lib.rst\n \n Memory Allocation\n M: Anatoly Burakov <anatoly.burakov@intel.com>\ndiff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst\nindex fb250abf5..0daa08acc 100644\n--- a/doc/guides/prog_guide/index.rst\n+++ b/doc/guides/prog_guide/index.rst\n@@ -35,6 +35,7 @@ Programmer's Guide\n lpm_lib\n lpm6_lib\n flow_classify_lib\n+ trace_lib\n packet_distrib_lib\n reorder_lib\n ip_fragment_reassembly_lib\ndiff --git a/doc/guides/prog_guide/trace_lib.rst b/doc/guides/prog_guide/trace_lib.rst\nnew file mode 100644\nindex 000000000..fe0fe2335\n--- /dev/null\n+++ b/doc/guides/prog_guide/trace_lib.rst\n@@ -0,0 +1,346 @@\n+.. SPDX-License-Identifier: BSD-3-Clause\n+ Copyright(C) 2020 Marvell International Ltd.\n+\n+Trace Library\n+=============\n+\n+Overview\n+--------\n+\n+*Tracing* is a technique used to understand what goes on in a running software\n+system. The software used for tracing is called a *tracer*, which is conceptually\n+similar to a tape recorder. When recording, specific instrumentation points\n+placed in the software source code generate events that are saved on a giant\n+tape: a trace file. The trace file then later can be opened in *trace viewers* to\n+visualize and analyze the trace events with timestamps and multi-core views.\n+Such a mechanism will be useful for resolving a wide range of problems such as\n+multi-core synchronization issues, latency measurements, finding out the\n+post analyses information like CPU idle time, etc that would otherwise be\n+extremely challenging.\n+\n+Tracing is often compared to *logging*. However, tracers and loggers are two\n+different tools, serving two different purposes. Tracers are designed to record\n+much lower-level events that occur much more frequently than log messages, often\n+in the range of thousands per second, with very little execution overhead.\n+Logging is more appropriate for a very high-level analysis of less frequent\n+events: user accesses, exceptional conditions (errors and warnings, for\n+example), database transactions, instant messaging communications, and such.\n+Simply put, logging is one of the many use cases that can be satisfied with\n+tracing.\n+\n+DPDK tracing library features\n+-----------------------------\n+\n+- A framework to add tracepoints in control and fast APIs with minimum impact on\n+ performance. Typical trace overhead is ~20 cycles and instrumentation\n+ overhead is 1 cycle.\n+- Enable and disable the tracepoints at runtime.\n+- Save the trace buffer to the filesystem at any point in time.\n+- Supports ``overwrite`` and ``discard`` trace mode operations.\n+- String-based tracepoint object lookup.\n+- Enable and disable a set of tracepoints based on regular expression and/or\n+ globbing.\n+- Generate trace in ``common trace format(CTF)``. ``CTF`` is an open-source trace\n+ format and it is compatible with ``LTTng``.\n+ For detailed information, refer `Common Trace Format <https://diamon.org/ctf/>`_\n+\n+How to add a tracepoint?\n+------------------------\n+\n+This section steps you through the details of adding a simple tracepoint.\n+\n+.. _create_provider_header_file:\n+\n+Create the tracepoint provider header file\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+.. code-block:: c\n+\n+ #include <rte_trace_point.h>\n+\n+ RTE_TRACE_POINT(app_trace_string,\n+ RTE_TRACE_POINT_ARGS(const char *str),\n+ rte_trace_point_emit_string(str);\n+ )\n+\n+The above macro creates ``app_trace_string`` tracepoint. The user can choose\n+any name for the tracepoint. However, when adding the tracepoint in the DPDK\n+library, the ``rte_trace_lib_<library_name>_[<domain>]_<name>`` naming convention\n+must be followed. The examples are ``rte_trace_lib_eal_generic_str``,\n+``rte_trace_lib_mempool_create``.\n+\n+When ``RTE_TRACE_POINT`` expands for above the definition, it creates the\n+following function template. The consumer of this tracepoint can invoke\n+``app_trace_string(const char *str)`` to emit the trace event to the trace buffer.\n+\n+.. code-block:: c\n+\n+ static __rte_always_inline void\n+ app_trace_string(const char *str)\n+ {\n+ /* Trace subsystem hooks */\n+ ...\n+ rte_trace_point_emit_string(str);\n+ }\n+\n+Register the tracepoint\n+~~~~~~~~~~~~~~~~~~~~~~~\n+\n+.. code-block:: c\n+\n+ #define RTE_TRACE_POINT_REGISTER_SELECT /* Select trace point register macros */\n+\n+ #include <tracepoint_provider_header_file.h>\n+\n+ RTE_TRACE_POINT_DEFINE(app_trace_string);\n+\n+ RTE_INIT(eal_trace_init)\n+ {\n+ RTE_TRACE_POINT_REGISTER(app_trace_string, app.trace.string);\n+ }\n+\n+The above code snippet registers the ``app_trace_string`` tracepoint to\n+trace library. Here, the ``tracepoint_provider_header_file.h`` is the header file\n+that user created in the first step :ref:`create_provider_header_file`\n+\n+The second argument for the ``RTE_TRACE_POINT_REGISTER`` is the name for the\n+tracepoint. This string will be used for tracepoint lookup or regular expression\n+and/or glob based tracepoint operations. There is no requirement for\n+the trace function and its name to be similar. However, it is recommended to\n+have a similar name for a better naming convention.\n+\n+The user must register the tracepoint before the ``rte_eal_init`` invocation.\n+The user can use the ``RTE_INIT`` construction scheme to achieve the same.\n+\n+.. note::\n+\n+ The ``RTE_TRACE_POINT_DEFINE`` defines the tracepoint of ``rte_trace_point_t``\n+ type. When the tracepoint defined in the shared library, the user must\n+ update the ``.map`` file with ``__<trace_function_name>`` symbol.\n+ For example, ``__app_trace_string`` will be the exported symbol in the\n+ above example.\n+\n+Datapath tracepoint\n+-------------------\n+\n+In order to avoid performance impact for the datapath tracepoint, the library\n+introduced ``RTE_TRACE_POINT_FP``. When adding the tracepoint in datapath code,\n+The user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``.\n+\n+``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using\n+``CONFIG_RTE_ENABLE_TRACE_FP`` configuration parameter. The ``enable_trace_fp``\n+build option shall be used for the same for meson build .The application may use\n+``rte_trace_fp_is_enabled()`` to get status of ``CONFIG_RTE_ENABLE_TRACE_FP``.\n+\n+Event record mode\n+-----------------\n+\n+Event record mode is an attribute of trace buffers. Trace library exposes the\n+following modes:\n+\n+.. _table_event_record_modes:\n+\n+.. table:: Event record modes.\n+\n+ +-----------+-----------------------------------------------------------------+\n+ | Mode | Description |\n+ | | |\n+ +===========+=================================================================+\n+ | Overwrite | When trace buffer is full, new trace events overwrites the |\n+ | | existing captured events in the trace buffer. |\n+ +-----------+-----------------------------------------------------------------+\n+ | Discard | When trace buffer is full, new trace events will be discarded. |\n+ +-----------+-----------------------------------------------------------------+\n+\n+The mode can be configured either using EAL command line parameters on\n+application boot up or use ``rte_trace_mode_set()`` API to configure at runtime.\n+Refer :doc:`../linux_gsg/linux_eal_parameters` for EAL command line options.\n+\n+Trace file location\n+-------------------\n+\n+On ``rte_trace_save()`` or ``rte_eal_cleanup()`` invocation, the library saves\n+the trace buffers to the filesystem. By default, library saves trace files at\n+``$HOME/dpdk-traces/rte-yyyy-mm-dd-[AP]M-hh-mm-ss/``. It can be overridden by\n+the ``--trace-dir=<directory path>`` EAL command line option.\n+\n+For more information, refer :doc:`../linux_gsg/linux_eal_parameters` for trace\n+EAL command line options.\n+\n+\n+View and analyze the recorded events\n+------------------------------------\n+\n+Once the trace directory is available, the user can view/inspect the recorded events.\n+\n+There are many tools you can use to read DPDK traces:\n+\n+1. ``babeltrace`` is a command-line utility that converts trace formats; it\n+supports the format that DPDK trace library produces, CTF, as well as a\n+basic text output that can be grep ed. The babeltrace command is part of the\n+opensource ``Babeltrace`` project.\n+\n+2. ``Trace Compass`` is a graphical user interface for viewing and analyzing any\n+type of logs or traces, including DPDK traces.\n+\n+Use the babeltrace command-line tool\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+The simplest way to list all the recorded events of a trace is to pass its path\n+to babeltrace with no options::\n+\n+ babeltrace </path-to-trace-events/rte-yyyy-mm-dd-[AP]M-hh-mm-ss/>\n+\n+``babeltrace`` finds all traces recursively within the given path and prints all\n+their events, merging them in chronological order.\n+\n+You can pipe the output of the babeltrace into a tool like grep(1) for further\n+filtering. Below example grep the events for ``ethdev`` only::\n+\n+ babeltrace /tmp/my-dpdk-trace | grep ethdev\n+\n+You can pipe the output of babeltrace into a tool like wc(1) to count the\n+recorded events. Below example count the number of ``ethdev`` events::\n+\n+ babeltrace /tmp/my-dpdk-trace | grep ethdev | wc --lines\n+\n+Use the tracecompass GUI tool\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+``Tracecompass`` is another tool to view/analyze the DPDK traces which gives\n+a graphical view of events. Like ``babeltrace``, tracecompass also provides\n+an interface to search for a particular event. To use ``tracecompass``,\n+following are the minimum required steps:\n+\n+- Install ``tracecompass`` to the localhost. Variants are available for Linux,\n+ Windows, and OS-X.\n+- Launch ``tracecompass`` which will open a graphical window with trace management\n+ interfaces.\n+- Open a trace using ``File->Open Trace`` option and select metadata file which\n+ is to be viewed/analyzed.\n+\n+For more details, refer `Trace Compass <https://www.eclipse.org/tracecompass/>`_\n+\n+Quick start\n+-----------\n+\n+This section steps you through the details of generating trace and viewing it.\n+\n+- Start the dpdk-test::\n+\n+ echo \"quit\" | ./build/app/test/dpdk-test --no-huge --trace=.*\n+\n+- View the traces with babeltrace viewer::\n+\n+ babeltrace $HOME/dpdk-traces/rte-yyyy-mm-dd-[AP]M-hh-mm-ss/\n+\n+Implementation details\n+----------------------\n+\n+As DPDK trace library is designed to generate traces that uses ``Common Trace\n+Format (CTF)``. ``CTF`` specification consists of the following units to create\n+a trace.\n+\n+- ``Stream`` Sequence of packets.\n+- ``Packet`` Header and one or more events.\n+- ``Event`` Header and payload.\n+\n+For detailed information, refer `Common Trace Format <https://diamon.org/ctf/>`_\n+\n+The implementation details broadly divided into the following areas:\n+\n+Trace metadata creation\n+~~~~~~~~~~~~~~~~~~~~~~~\n+\n+Based on the ``CTF`` specification, One of a CTF trace's streams is mandatory:\n+the metadata stream. It contains exactly what you would expect: data about the\n+trace itself. The metadata stream contains a textual description of the binary\n+layouts of all the other streams.\n+\n+This description is written using the Trace Stream Description Language (TSDL),\n+a declarative language that exists only in the realm of CTF. The purpose of the\n+metadata stream is to make CTF readers know how to parse a trace's binary\n+streams of events without CTF specifying any fixed layout. The only stream\n+layout known in advance is, in fact, the metadata stream's one.\n+\n+The internal ``trace_metadata_create()`` function generates the metadata.\n+\n+Trace memory\n+~~~~~~~~~~~~\n+\n+The trace memory will be allocated through an internal function\n+``__rte_trace_mem_per_thread_alloc()``. The trace memory will be allocated\n+per thread to enable lock less trace-emit function. The memory for the\n+trace memory for DPDK lcores will be allocated on ``rte_eal_init()`` if the trace\n+is enabled through a EAL option. For non DPDK threads, on the first trace\n+emission, the memory will be allocated.\n+\n+Trace memory layout\n+~~~~~~~~~~~~~~~~~~~\n+\n+.. _table_trace_mem_layout:\n+\n+.. table:: Trace memory layout.\n+\n+ +-------------------+\n+ | packet.header |\n+ +-------------------+\n+ | packet.context |\n+ +-------------------+\n+ | trace 0 header |\n+ +-------------------+\n+ | trace 0 payload |\n+ +-------------------+\n+ | trace 1 header |\n+ +-------------------+\n+ | trace 1 payload |\n+ +-------------------+\n+ | trace N header |\n+ +-------------------+\n+ | trace N payload |\n+ +-------------------+\n+\n+packet.header\n+^^^^^^^^^^^^^\n+\n+.. _table_packet_header:\n+\n+.. table:: Packet header layout.\n+\n+ +-------------------+\n+ | uint32_t magic |\n+ +-------------------+\n+ | rte_uuid_t uuid |\n+ +-------------------+\n+\n+packet.context\n+^^^^^^^^^^^^^^\n+\n+.. _table_packet_context:\n+\n+.. table:: Packet context layout.\n+\n+ +----------------------+\n+ | uint32_t thread_id |\n+ +----------------------+\n+ | char thread_name[32] |\n+ +----------------------+\n+\n+trace.header\n+^^^^^^^^^^^^\n+\n+.. _table_trace_header:\n+\n+.. table:: Packet context layout.\n+\n+ +----------------------+\n+ | event_id [63:48] |\n+ +----------------------+\n+ | timestamp [47:0] |\n+ +----------------------+\n+\n+The trace header is 64bit, it consists of 48bit of timestamp and 16bit event ID.\n+\n+The ``packet.header`` and ``packet.context`` will be written in the slow path\n+at the time of trace memory creation. The ``trace.header`` and trace payout\n+will be emitted when the trace function invoked.\ndiff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst\nindex 184967844..6a108b104 100644\n--- a/doc/guides/rel_notes/release_20_05.rst\n+++ b/doc/guides/rel_notes/release_20_05.rst\n@@ -81,6 +81,15 @@ New Features\n by making use of the event device capabilities. The event mode currently supports\n only inline IPsec protocol offload.\n \n+* **Added Trace Library and Tracepoints**\n+\n+ A native implementation of ``common trace format(CTF)`` based trace library added\n+ to provide the ability to add tracepoints in application/library to get runtime\n+ trace/debug information for control and fast APIs with minimum impact on\n+ fast path performance. Typical trace overhead is ~20 cycles and instrumentation\n+ overhead is 1 cycle. Added tracepoints in ``EAL``, ``ethdev``, ``cryptodev``,\n+ ``eventdev`` and ``mempool`` libraries for important functions.\n+\n \n Removed Items\n -------------\n", "prefixes": [ "v6", "33/33" ] }