Cover Detail
Show a cover letter.
GET /api/covers/66001/?format=api
{ "id": 66001, "url": "http://patches.dpdk.org/api/covers/66001/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/cover/20200224113515.1744-1-konstantin.ananyev@intel.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20200224113515.1744-1-konstantin.ananyev@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20200224113515.1744-1-konstantin.ananyev@intel.com", "date": "2020-02-24T11:35:09", "name": "[RFC,0/6] New sync modes for ring", "submitter": { "id": 33, "url": "http://patches.dpdk.org/api/people/33/?format=api", "name": "Ananyev, Konstantin", "email": "konstantin.ananyev@intel.com" }, "mbox": "http://patches.dpdk.org/project/dpdk/cover/20200224113515.1744-1-konstantin.ananyev@intel.com/mbox/", "series": [ { "id": 8659, "url": "http://patches.dpdk.org/api/series/8659/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=8659", "date": "2020-02-24T11:35:09", "name": "New sync modes for ring", "version": 1, "mbox": "http://patches.dpdk.org/series/8659/mbox/" } ], "comments": "http://patches.dpdk.org/api/covers/66001/comments/", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from dpdk.org (dpdk.org [92.243.14.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id A73C5A0524;\n\tMon, 24 Feb 2020 12:35:24 +0100 (CET)", "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 4E5FE1BE85;\n\tMon, 24 Feb 2020 12:35:23 +0100 (CET)", "from mga02.intel.com (mga02.intel.com [134.134.136.20])\n by dpdk.org (Postfix) with ESMTP id 650881F1C\n for <dev@dpdk.org>; Mon, 24 Feb 2020 12:35:22 +0100 (CET)", "from fmsmga005.fm.intel.com ([10.253.24.32])\n by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n 24 Feb 2020 03:35:20 -0800", "from sivswdev08.ir.intel.com ([10.237.217.47])\n by fmsmga005.fm.intel.com with ESMTP; 24 Feb 2020 03:35:18 -0800" ], "X-Amp-Result": "SKIPPED(no attachment in message)", "X-Amp-File-Uploaded": "False", "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.70,480,1574150400\"; d=\"scan'208\";a=\"435878522\"", "From": "Konstantin Ananyev <konstantin.ananyev@intel.com>", "To": "dev@dpdk.org", "Cc": "olivier.matz@6wind.com, Konstantin Ananyev <konstantin.ananyev@intel.com>", "Date": "Mon, 24 Feb 2020 11:35:09 +0000", "Message-Id": "<20200224113515.1744-1-konstantin.ananyev@intel.com>", "X-Mailer": "git-send-email 2.18.0", "Subject": "[dpdk-dev] [RFC 0/6] New sync modes for ring", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "Upfront note - that RFC is not a complete patch.\nIt introduces an ABI breakage, plus it doesn't update ring_elem\ncode properly, etc.\nI plan to deal with all these things in later versions.\nRight now I seek an initial feedback about proposed ideas.\nWould also ask people to repeat performance tests (see below)\non their platforms to confirm the impact.\n\nMore and more customers use(/try to use) DPDK based apps within\novercommitted systems (multiple acttive threads over same pysical cores):\nVM, container deployments, etc.\nOne quite common problem they hit: Lock-Holder-Preemption with rte_ring.\nLHP is quite a common problem for spin-based sync primitives\n(spin-locks, etc.) on overcommitted systems.\nThe situation gets much worse when some sort of\nfair-locking technique is used (ticket-lock, etc.).\nAs now not only lock-owner but also lock-waiters scheduling\norder matters a lot.\nThis is a well-known problem for kernel within VMs:\nhttp://www-archive.xenproject.org/files/xensummitboston08/LHP.pdf\nhttps://www.cs.hs-rm.de/~kaiser/events/wamos2017/Slides/selcuk.pdf\nThe problem with rte_ring is that while head accusion is sort of\nun-fair locking, waiting on tail is very similar to ticket lock schema -\ntail has to be updated in particular order.\nThat makes current rte_ring implementation to perform\nreally pure on some overcommited scenarios.\nWhile it is probably not possible to completely resolve this problem in\nuserspace only (without some kernel communication/intervention),\nremoving fairness in tail update can mitigate it significantly.\nSo this RFC proposes two new optional ring synchronization modes:\n1) Head/Tail Sync (HTS) mode\nIn that mode enqueue/dequeue operation is fully serialized:\n only one thread at a time is allowed to perform given op.\n As another enhancement provide ability to split enqueue/dequeue\n operation into two phases:\n - enqueue/dequeue start\n - enqueue/dequeue finish\n That allows user to inspect objects in the ring without removing\n them from it (aka MT safe peek).\n2) Relaxed Tail Sync (RTS)\nThe main difference from original MP/MC algorithm is that\ntail value is increased not by every thread that finished enqueue/dequeue,\nbut only by the last one.\nThat allows threads to avoid spinning on ring tail value,\nleaving actual tail value change to the last thread in the update queue.\n\nTest results on IA (see below) show significant improvements\nfor average enqueue/dequeue op times on overcommitted systems.\nFor 'classic' DPDK deployments (one thread per core) original MP/MC\nalgorithm still shows best numbers, though for 64-bit target\nRTS numbers are not that far away.\nNumbers were produced by ring_stress_*autotest\n(first patch in these series).\n\nX86_64 @ Intel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHz\nDEQ+ENQ average cycles/obj\n\n MP/MC HTS RTS\n1thread@1core(--lcores=6-7) 8.00 8.15 8.99\n2thread@2core(--lcores=6-8) 19.14 19.61 20.35\n4thread@4core(--lcores=6-10) 29.43 29.79 31.82\n8thread@8core(--lcores=6-14) 110.59 192.81 119.50\n16thread@16core(--lcores=6-22) 461.03 813.12 495.59\n32thread/@32core(--lcores='6-22,55-70') 982.90 1972.38 1160.51\n\n2thread@1core(--lcores='6,(10-11)@7' 20140.50 23.58 25.14\n4thread@2core(--lcores='6,(10-11)@7,(20-21)@8' 153680.60 76.88 80.05\n8thread@2core(--lcores='6,(10-13)@7,(20-23)@8' 280314.32 294.72 318.79\n16thread@2core(--lcores='6,(10-17)@7,(20-27)@8' 643176.59 1144.02 1175.14\n32thread@2core(--lcores='6,(10-25)@7,(30-45)@8' 4264238.80 4627.48 4892.68\n\n8thread@2core(--lcores='6,(10-17)@(7,8))' 321085.98 298.59 307.47\n16thread@4core(--lcores='6,(20-35)@(7-10))' 1900705.61 575.35 678.29\n32thread@4core(--lcores='6,(20-51)@(7-10))' 5510445.85 2164.36 2714.12\n\ni686 @ Intel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHz\nDEQ+ENQ average cycles/obj\n\n MP/MC HTS RTS\n1thread@1core(--lcores=6-7) 7.85 12.13 11.31\n2thread@2core(--lcores=6-8) 17.89 24.52 21.86\n8thread@8core(--lcores=6-14) 32.58 354.20 54.58\n32thread/@32core(--lcores='6-22,55-70') 813.77 6072.41 2169.91\n\n2thread@1core(--lcores='6,(10-11)@7' 16095.00 36.06 34.74\n8thread@2core(--lcores='6,(10-13)@7,(20-23)@8' 1140354.54 346.61 361.57\n16thread@2core(--lcores='6,(10-17)@7,(20-27)@8' 1920417.86 1314.90 1416.65\n\n8thread@2core(--lcores='6,(10-17)@(7,8))' 594358.61 332.70 357.74\n32thread@4core(--lcores='6,(20-51)@(7-10))' 5319896.86 2836.44 3028.87\n\nKonstantin Ananyev (6):\n test/ring: add contention stress test\n ring: rework ring layout to allow new sync schemes\n ring: introduce RTS ring mode\n test/ring: add contention stress test for RTS ring\n ring: introduce HTS ring mode\n test/ring: add contention stress test for HTS ring\n\n app/test/Makefile | 3 +\n app/test/meson.build | 3 +\n app/test/test_pdump.c | 6 +-\n app/test/test_ring_hts_stress.c | 28 ++\n app/test/test_ring_rts_stress.c | 28 ++\n app/test/test_ring_stress.c | 27 ++\n app/test/test_ring_stress.h | 477 +++++++++++++++++++\n lib/librte_pdump/rte_pdump.c | 2 +-\n lib/librte_port/rte_port_ring.c | 12 +-\n lib/librte_ring/Makefile | 4 +-\n lib/librte_ring/meson.build | 4 +-\n lib/librte_ring/rte_ring.c | 84 +++-\n lib/librte_ring/rte_ring.h | 619 +++++++++++++++++++++++--\n lib/librte_ring/rte_ring_elem.h | 8 +-\n lib/librte_ring/rte_ring_hts_generic.h | 228 +++++++++\n lib/librte_ring/rte_ring_rts_generic.h | 240 ++++++++++\n 16 files changed, 1721 insertions(+), 52 deletions(-)\n create mode 100644 app/test/test_ring_hts_stress.c\n create mode 100644 app/test/test_ring_rts_stress.c\n create mode 100644 app/test/test_ring_stress.c\n create mode 100644 app/test/test_ring_stress.h\n create mode 100644 lib/librte_ring/rte_ring_hts_generic.h\n create mode 100644 lib/librte_ring/rte_ring_rts_generic.h" }