Cover Detail
Show a cover letter.
GET /api/covers/90508/?format=api
{ "id": 90508, "url": "http://patches.dpdk.org/api/covers/90508/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/cover/1617413948-10504-1-git-send-email-navasile@linux.microsoft.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<1617413948-10504-1-git-send-email-navasile@linux.microsoft.com>", "list_archive_url": "https://inbox.dpdk.org/dev/1617413948-10504-1-git-send-email-navasile@linux.microsoft.com", "date": "2021-04-03T01:38:58", "name": "[v6,00/10] eal: Add new API for threading", "submitter": { "id": 1668, "url": "http://patches.dpdk.org/api/people/1668/?format=api", "name": "Narcisa Ana Maria Vasile", "email": "navasile@linux.microsoft.com" }, "mbox": "http://patches.dpdk.org/project/dpdk/cover/1617413948-10504-1-git-send-email-navasile@linux.microsoft.com/mbox/", "series": [ { "id": 16091, "url": "http://patches.dpdk.org/api/series/16091/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=16091", "date": "2021-04-03T01:38:59", "name": "eal: Add new API for threading", "version": 6, "mbox": "http://patches.dpdk.org/series/16091/mbox/" } ], "comments": "http://patches.dpdk.org/api/covers/90508/comments/", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 801ECA0548;\n\tSat, 3 Apr 2021 03:39:21 +0200 (CEST)", "from [217.70.189.124] (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 528A7140EA7;\n\tSat, 3 Apr 2021 03:39:16 +0200 (CEST)", "from linux.microsoft.com (linux.microsoft.com [13.77.154.182])\n by mails.dpdk.org (Postfix) with ESMTP id 27EB8140EA1\n for <dev@dpdk.org>; Sat, 3 Apr 2021 03:39:13 +0200 (CEST)", "by linux.microsoft.com (Postfix, from userid 1059)\n id 562E020ABC4A; Fri, 2 Apr 2021 18:39:12 -0700 (PDT)" ], "DKIM-Filter": "OpenDKIM Filter v2.11.0 linux.microsoft.com 562E020ABC4A", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com;\n s=default; t=1617413952;\n bh=ATI/x9FkBMq/hwGsBpzO5aZi9fpQ3nziw3cpysafhV8=;\n h=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n b=sQ7QNjtMq6ILHyMlFTU8zstyrNLvwMQv3z9L5yq3WxQ3h6Zy1rXQOXnsYB1oKx4Aa\n ANcJzp+PNezrQud/CC6aK21bGoZr4TMQA8n9zetpvT+p34bDfRSyrHyhkfmO9E3nTV\n N0fhwgsYZsWXIo46jTPAV8XstU33cAPSwM+oDpIw=", "From": "Narcisa Ana Maria Vasile <navasile@linux.microsoft.com>", "To": "dev@dpdk.org, thomas@monjalon.net, dmitry.kozliuk@gmail.com,\n khot@microsoft.com, navasile@microsoft.com, dmitrym@microsoft.com,\n roretzla@microsoft.com, talshn@nvidia.com, ocardona@microsoft.com", "Cc": "bruce.richardson@intel.com, david.marchand@redhat.com,\n pallavi.kadam@intel.com", "Date": "Fri, 2 Apr 2021 18:38:58 -0700", "Message-Id": "<1617413948-10504-1-git-send-email-navasile@linux.microsoft.com>", "X-Mailer": "git-send-email 1.8.3.1", "In-Reply-To": "<1617057640-24301-2-git-send-email-navasile@linux.microsoft.com>", "References": "<1617057640-24301-2-git-send-email-navasile@linux.microsoft.com>", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=UTF-8", "Content-Transfer-Encoding": "8bit", "Subject": "[dpdk-dev] [PATCH v6 00/10] eal: Add new API for threading", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "From: Narcisa Vasile <navasile@microsoft.com>\n\nEAL thread API\n\n**Problem Statement**\nDPDK currently uses the pthread interface to create and manage threads.\nWindows does not support the POSIX thread programming model, so it currently\nrelies on a header file that hides the Windows calls under\npthread matched interfaces. Given that EAL should isolate the environment\nspecifics from the applications and libraries and mediate\nall the communication with the operating systems, a new EAL interface\nis needed for thread management.\n\n**Goals**\n* Introduce a generic EAL API for threading support that will remove\n the current Windows pthread.h shim.\n* Replace references to pthread_* across the DPDK codebase with the new\n RTE_THREAD_* API.\n* Allow users to choose between using the RTE_THREAD_* API or a\n 3rd party thread library through a configuration option.\n\n**Design plan**\nNew API main files:\n* rte_thread.h (librte_eal/include)\n* rte_thread_types.h (librte_eal/include)\n* rte_thread_windows_types.h (librte_eal/windows/include)\n* rte_thread.c (librte_eal/windows)\n* rte_thread.c (librte_eal/common)\n\nFor flexibility, the user is offered the option of either using the RTE_THREAD_* API or\na 3rd party thread library, through a meson flag “use_external_thread_lib”.\nBy default, this flag is set to FALSE, which means Windows libraries and applications\nwill use the RTE_THREAD_* API for managing threads.\n\nIf compiling on Windows and the “use_external_thread_lib” is *not* set,\nthe following files will be parsed: \n* include/rte_thread.h\n* windows/include/rte_thread_windows_types.h\n* windows/rte_thread.c\nIn all other cases, the compilation/parsing includes the following files:\n* include/rte_thread.h \n* include/rte_thread_types.h\n* common/rte_thread.c\n\n**A schematic example of the design**\n--------------------------------------------------\nlib/librte_eal/include/rte_thread.h\nint rte_thread_create();\n\nlib/librte_eal/common/rte_thread.c\nint rte_thread_create() \n{\n\treturn pthread_create();\n}\n\nlib/librte_eal/windows/rte_thread.c\nint rte_thread_create() \n{\n\treturn CreateThread();\n}\n\nlib/librte_eal/windows/meson.build\nif get_option('use_external_thread_lib')\n\tsources += 'librte_eal/common/rte_thread.c'\nelse\n\tsources += 'librte_eal/windows/rte_thread.c'\nendif\n-----------------------------------------------------\n\n**Thread attributes**\n\nWhen or after a thread is created, specific characteristics of the thread\ncan be adjusted. Given that the thread characteristics that are of interest\nfor DPDK applications are affinity and priority, the following structure\nthat represents thread attributes has been defined:\n\ntypedef struct\n{\n\tenum rte_thread_priority priority;\n\trte_cpuset_t cpuset;\n} rte_thread_attr_t;\n\nThe *rte_thread_create()* function can optionally receive an rte_thread_attr_t\nobject that will cause the thread to be created with the affinity and priority\ndescribed by the attributes object. If no rte_thread_attr_t is passed\n(parameter is NULL), the default affinity and priority are used.\nAn rte_thread_attr_t object can also be set to the default values\nby calling *rte_thread_attr_init()*.\n\n*Priority* is represented through an enum that currently advertises\ntwo values for priority:\n\t- RTE_THREAD_PRIORITY_NORMAL\n\t- RTE_THREAD_PRIORITY_REALTIME_CRITICAL\nThe enum can be extended to allow for multiple priority levels.\nrte_thread_set_priority - sets the priority of a thread\nrte_thread_attr_set_priority - updates an rte_thread_attr_t object\n with a new value for priority\n\nThe user can choose thread priority through an EAL parameter,\nwhen starting an application. If EAL parameter is not used,\nthe per-platform default value for thread priority is used.\nOtherwise administrator has an option to set one of available options:\n --thread-prio normal\n --thread-prio realtime\n\nExample:\n./dpdk-l2fwd -l 0-3 -n 4 –thread-prio normal -- -q 8 -p ffff\n\n*Affinity* is described by the already known “rte_cpuset_t” type.\nrte_thread_attr_set/get_affinity - sets/gets the affinity field in a\n rte_thread_attr_t object\nrte_thread_set/get_affinity – sets/gets the affinity of a thread\n\n**Errors**\nA translation function that maps Windows error codes to errno-style\nerror codes is provided. \n\n**Future work**\nNote that this patchset was focused on introducing new API that will\nremove the Windows pthread.h shim. In DPDK, there are still a few references\nto pthread_* that were not implemented in the shim.\nThe long term plan is for EAL to provide full threading support:\n* Adding support for conditional variables\n* Additional functionality offered by pthread_* (such as pthread_setname_np, etc.)\n* Static mutex initializers are not used on Windows. If we must continue\n using them, they need to be platform dependent and an implementation will\n need to be provided for Windows.\n\nv6:\n- improve error-translation function\n- call the error translation function in rte_thread_value_get()\n\nv5:\n- update cover letter with more details on the priority argument\n\nv4:\n- fix function description\n- rebase\n\nv3:\n- rebase\n\nv2:\n- revert changes that break ABI \n- break up changes into smaller patches\n- fix coding style issues\n- fix issues with errors\n- fix parameter type in examples/kni.c\n\nNarcisa Vasile (10):\n eal: add thread id and simple thread functions\n eal: add thread attributes\n windows/eal: translate Windows errors to errno-style errors\n eal: implement functions for thread affinity management\n eal: implement thread priority management functions\n eal: add thread lifetime management\n eal: implement functions for mutex management\n eal: implement functions for thread barrier management\n eal: add EAL argument for setting thread priority\n Enable the new EAL thread API\n\n app/test/process.h | 6 +-\n app/test/test_lcores.c | 16 +-\n app/test/test_link_bonding.c | 10 +-\n app/test/test_lpm_perf.c | 12 +-\n config/meson.build | 4 +\n drivers/bus/dpaa/base/qbman/bman_driver.c | 6 +-\n drivers/bus/dpaa/base/qbman/dpaa_sys.c | 14 +-\n drivers/bus/dpaa/base/qbman/process.c | 6 +-\n drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 6 +-\n drivers/compress/mlx5/mlx5_compress.c | 10 +-\n drivers/event/dlb/pf/base/dlb_osdep.h | 2 +-\n drivers/event/dlb2/pf/base/dlb2_osdep.h | 2 +-\n drivers/net/af_xdp/rte_eth_af_xdp.c | 18 +-\n drivers/net/ark/ark_ethdev.c | 4 +-\n drivers/net/atlantic/atl_ethdev.c | 4 +-\n drivers/net/atlantic/atl_types.h | 5 +-\n .../net/atlantic/hw_atl/hw_atl_utils_fw2x.c | 24 +-\n drivers/net/axgbe/axgbe_dev.c | 8 +-\n drivers/net/axgbe/axgbe_ethdev.c | 8 +-\n drivers/net/axgbe/axgbe_ethdev.h | 8 +-\n drivers/net/axgbe/axgbe_i2c.c | 4 +-\n drivers/net/axgbe/axgbe_mdio.c | 8 +-\n drivers/net/axgbe/axgbe_phy_impl.c | 6 +-\n drivers/net/bnxt/bnxt.h | 16 +-\n drivers/net/bnxt/bnxt_cpr.c | 4 +-\n drivers/net/bnxt/bnxt_ethdev.c | 52 +-\n drivers/net/bnxt/bnxt_irq.c | 8 +-\n drivers/net/bnxt/bnxt_reps.c | 10 +-\n drivers/net/bnxt/tf_ulp/bnxt_ulp.c | 34 +-\n drivers/net/bnxt/tf_ulp/bnxt_ulp.h | 4 +-\n drivers/net/bnxt/tf_ulp/ulp_fc_mgr.c | 24 +-\n drivers/net/bnxt/tf_ulp/ulp_fc_mgr.h | 2 +-\n drivers/net/ena/base/ena_plat_dpdk.h | 8 +-\n drivers/net/enic/enic.h | 2 +-\n drivers/net/hns3/hns3_ethdev.h | 2 +-\n drivers/net/hns3/hns3_ethdev_vf.c | 2 +-\n drivers/net/hns3/hns3_mbx.c | 2 +-\n drivers/net/ice/ice_dcf_parent.c | 4 +-\n drivers/net/ipn3ke/ipn3ke_representor.c | 6 +-\n drivers/net/ixgbe/ixgbe_ethdev.c | 2 +-\n drivers/net/ixgbe/ixgbe_ethdev.h | 2 +-\n drivers/net/kni/rte_eth_kni.c | 6 +-\n drivers/net/mlx5/linux/mlx5_os.c | 2 +-\n drivers/net/mlx5/mlx5.c | 20 +-\n drivers/net/mlx5/mlx5.h | 2 +-\n drivers/net/mlx5/mlx5_txpp.c | 8 +-\n drivers/net/mlx5/windows/mlx5_flow_os.c | 10 +-\n drivers/net/mlx5/windows/mlx5_os.c | 2 +-\n drivers/net/qede/base/bcm_osal.h | 8 +-\n drivers/net/vhost/rte_eth_vhost.c | 22 +-\n .../net/virtio/virtio_user/virtio_user_dev.c | 30 +-\n .../net/virtio/virtio_user/virtio_user_dev.h | 2 +-\n drivers/raw/ifpga/ifpga_rawdev.c | 8 +-\n drivers/vdpa/ifc/ifcvf_vdpa.c | 36 +-\n drivers/vdpa/mlx5/mlx5_vdpa.c | 24 +-\n drivers/vdpa/mlx5/mlx5_vdpa.h | 6 +-\n drivers/vdpa/mlx5/mlx5_vdpa_event.c | 67 +--\n examples/kni/main.c | 6 +-\n examples/vhost/main.c | 2 +-\n examples/vhost_blk/vhost_blk.c | 10 +-\n lib/librte_eal/common/eal_common_options.c | 36 +-\n lib/librte_eal/common/eal_common_proc.c | 48 +-\n lib/librte_eal/common/eal_common_thread.c | 43 +-\n lib/librte_eal/common/eal_common_trace.c | 2 +-\n lib/librte_eal/common/eal_internal_cfg.h | 2 +\n lib/librte_eal/common/eal_options.h | 2 +\n lib/librte_eal/common/eal_private.h | 2 +-\n lib/librte_eal/common/malloc_mp.c | 32 +-\n lib/librte_eal/common/meson.build | 1 +\n lib/librte_eal/common/rte_thread.c | 342 +++++++++++\n lib/librte_eal/freebsd/eal.c | 37 +-\n lib/librte_eal/freebsd/eal_alarm.c | 12 +-\n lib/librte_eal/freebsd/eal_interrupts.c | 4 +-\n lib/librte_eal/freebsd/eal_thread.c | 8 +-\n lib/librte_eal/include/meson.build | 1 +\n lib/librte_eal/include/rte_lcore.h | 8 +-\n lib/librte_eal/include/rte_per_lcore.h | 2 -\n lib/librte_eal/include/rte_thread.h | 342 ++++++++++-\n lib/librte_eal/include/rte_thread_types.h | 20 +\n lib/librte_eal/linux/eal.c | 42 +-\n lib/librte_eal/linux/eal_alarm.c | 10 +-\n lib/librte_eal/linux/eal_interrupts.c | 4 +-\n lib/librte_eal/linux/eal_thread.c | 8 +-\n lib/librte_eal/linux/eal_timer.c | 2 +-\n lib/librte_eal/rte_eal_exports.def | 20 +\n lib/librte_eal/unix/meson.build | 1 -\n lib/librte_eal/unix/rte_thread.c | 92 ---\n lib/librte_eal/version.map | 21 +\n lib/librte_eal/windows/eal.c | 26 +-\n lib/librte_eal/windows/eal_interrupts.c | 6 +-\n lib/librte_eal/windows/eal_lcore.c | 170 ++++--\n lib/librte_eal/windows/eal_thread.c | 24 +-\n lib/librte_eal/windows/eal_windows.h | 20 +-\n lib/librte_eal/windows/include/meson.build | 1 +\n lib/librte_eal/windows/include/pthread.h | 186 ------\n .../include/rte_windows_thread_types.h | 19 +\n lib/librte_eal/windows/include/sched.h | 2 +-\n lib/librte_eal/windows/meson.build | 7 +-\n lib/librte_eal/windows/rte_thread.c | 532 +++++++++++++++++-\n lib/librte_ethdev/rte_ethdev.c | 4 +-\n lib/librte_ethdev/rte_ethdev_core.h | 3 +-\n lib/librte_ethdev/rte_flow.c | 4 +-\n .../rte_event_eth_rx_adapter.c | 6 +-\n lib/librte_vhost/fd_man.c | 40 +-\n lib/librte_vhost/fd_man.h | 6 +-\n lib/librte_vhost/socket.c | 124 ++--\n lib/librte_vhost/vhost.c | 10 +-\n meson_options.txt | 2 +\n 108 files changed, 2048 insertions(+), 934 deletions(-)\n create mode 100644 lib/librte_eal/common/rte_thread.c\n create mode 100644 lib/librte_eal/include/rte_thread_types.h\n delete mode 100644 lib/librte_eal/unix/rte_thread.c\n delete mode 100644 lib/librte_eal/windows/include/pthread.h\n create mode 100644 lib/librte_eal/windows/include/rte_windows_thread_types.h" }