Cover Comment List
List cover comments
GET /api/covers/53180/comments/?format=api&order=date
[ { "id": 95715, "web_url": "http://patches.dpdk.org/comment/95715/", "msgid": "<20190501121545.GA26521@hmswarspite.think-freely.org>", "list_archive_url": "https://inbox.dpdk.org/dev/20190501121545.GA26521@hmswarspite.think-freely.org", "date": "2019-05-01T12:15:45", "subject": "Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "submitter": { "id": 32, "url": "http://patches.dpdk.org/api/people/32/?format=api", "name": "Neil Horman", "email": "nhorman@tuxdriver.com" }, "content": "On Tue, Apr 30, 2019 at 10:54:15PM -0500, Honnappa Nagarahalli wrote:\n> Lock-less data structures provide scalability and determinism.\n> They enable use cases where locking may not be allowed\n> (for ex: real-time applications).\n> \nI know this is version 9 of the patch, so I'm sorry for the late comment, but I\nhave to ask: Why re-invent this wheel? There are already several Userspace RCU\nlibraries that are mature and carried by Linux and BSD distributions. Why would\nwe throw another one into DPDK instead of just using whats already available,\nmature and stable?\n\nNeil", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id D71132A6C;\n\tWed, 1 May 2019 14:16:45 +0200 (CEST)", "from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58])\n\tby dpdk.org (Postfix) with ESMTP id EF54F1F28\n\tfor <dev@dpdk.org>; Wed, 1 May 2019 14:16:44 +0200 (CEST)", "from cpe-2606-a000-111b-405a-0-0-0-162e.dyn6.twc.com\n\t([2606:a000:111b:405a::162e] helo=localhost)\n\tby smtp.tuxdriver.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.63)\n\t(envelope-from <nhorman@tuxdriver.com>)\n\tid 1hLo9n-00017s-IA; Wed, 01 May 2019 08:16:30 -0400" ], "Date": "Wed, 1 May 2019 08:15:45 -0400", "From": "Neil Horman <nhorman@tuxdriver.com>", "To": "Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>", "Cc": "konstantin.ananyev@intel.com, stephen@networkplumber.org,\n\tpaulmck@linux.ibm.com, marko.kovacevic@intel.com, dev@dpdk.org,\n\tgavin.hu@arm.com, dharmik.thakkar@arm.com, malvika.gupta@arm.com", "Message-ID": "<20190501121545.GA26521@hmswarspite.think-freely.org>", "References": "<20181122033055.3431-1-honnappa.nagarahalli@arm.com>\n\t<20190501035419.33524-1-honnappa.nagarahalli@arm.com>", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=us-ascii", "Content-Disposition": "inline", "In-Reply-To": "<20190501035419.33524-1-honnappa.nagarahalli@arm.com>", "User-Agent": "Mutt/1.11.3 (2019-02-01)", "X-Spam-Score": "-2.9 (--)", "X-Spam-Status": "No", "Subject": "Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 95718, "web_url": "http://patches.dpdk.org/comment/95718/", "msgid": "<VE1PR08MB51491622B8794D6E8A1DA833983B0@VE1PR08MB5149.eurprd08.prod.outlook.com>", "list_archive_url": "https://inbox.dpdk.org/dev/VE1PR08MB51491622B8794D6E8A1DA833983B0@VE1PR08MB5149.eurprd08.prod.outlook.com", "date": "2019-05-01T14:56:48", "subject": "Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "submitter": { "id": 1045, "url": "http://patches.dpdk.org/api/people/1045/?format=api", "name": "Honnappa Nagarahalli", "email": "honnappa.nagarahalli@arm.com" }, "content": "> \n> On Tue, Apr 30, 2019 at 10:54:15PM -0500, Honnappa Nagarahalli wrote:\n> > Lock-less data structures provide scalability and determinism.\n> > They enable use cases where locking may not be allowed (for ex:\n> > real-time applications).\n> >\n> I know this is version 9 of the patch, so I'm sorry for the late comment, but I\n> have to ask: Why re-invent this wheel? There are already several Userspace\nThanks Neil, for asking the question. This has been debated before. Please refer to [2] for more details.\n\nliburcu [1] was explored as it seemed to be familiar to others in the community . I am not aware of any other library.\n\nThere are unique requirements in DPDK and there is still scope for improvement from what is available. I have explained this in the cover letter without making a direct comparison to liburcu. May be it is worth tweaking the documentation to call this out explicitly.\n\n[1] https://liburcu.org/\n[2] http://mails.dpdk.org/archives/dev/2018-November/119875.html\n\n> RCU libraries that are mature and carried by Linux and BSD distributions.\n> Why would we throw another one into DPDK instead of just using whats\n> already available, mature and stable?\n> \n> Neil", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id DE6262BC1;\n\tWed, 1 May 2019 16:56:53 +0200 (CEST)", "from EUR04-HE1-obe.outbound.protection.outlook.com\n\t(mail-eopbgr70077.outbound.protection.outlook.com [40.107.7.77])\n\tby dpdk.org (Postfix) with ESMTP id 1AA132BAE\n\tfor <dev@dpdk.org>; Wed, 1 May 2019 16:56:51 +0200 (CEST)", "from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.152) by\n\tVE1PR08MB5104.eurprd08.prod.outlook.com (20.179.30.14) with Microsoft\n\tSMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id\n\t15.20.1835.15; Wed, 1 May 2019 14:56:48 +0000", "from VE1PR08MB5149.eurprd08.prod.outlook.com\n\t([fe80::9b6:3403:4386:78]) by VE1PR08MB5149.eurprd08.prod.outlook.com\n\t([fe80::9b6:3403:4386:78%2]) with mapi id 15.20.1835.018;\n\tWed, 1 May 2019 14:56:48 +0000" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;\n\ts=selector1-arm-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n\tbh=nqLjvQqB6ANLKJMyDaUaMxBpt/Xn0dnXBpwgFYhpZY8=;\n\tb=ZxWknLGCbiIpMQNv7cCzgvLsiU8p3ViRKGLUEnB2FPq3Vyw9P8AeWIULEEVQDdv3HA5N7fTdPh09vqD2qmEd0Rv6EstnUh2Ke2Oaz/dRwIdsdL8SmZ//wnxNw4VKe6a+ohBaSlP8XwKw7tGGRYRhjZ6k0Zw0Bjsahla3jNfawxo=", "From": "Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>", "To": "Neil Horman <nhorman@tuxdriver.com>", "CC": "\"konstantin.ananyev@intel.com\" <konstantin.ananyev@intel.com>,\n\t\"stephen@networkplumber.org\" <stephen@networkplumber.org>,\n\t\"paulmck@linux.ibm.com\" <paulmck@linux.ibm.com>,\n\t\"marko.kovacevic@intel.com\"\n\t<marko.kovacevic@intel.com>, \"dev@dpdk.org\" <dev@dpdk.org>,\n\t\"Gavin Hu (Arm Technology China)\" <Gavin.Hu@arm.com>, Dharmik Thakkar\n\t<Dharmik.Thakkar@arm.com>, Malvika Gupta <Malvika.Gupta@arm.com>,\n\tHonnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>, nd <nd@arm.com>,\n\tnd <nd@arm.com>", "Thread-Topic": "[dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "Thread-Index": "AQHVABe82XMyTZzopkuFSkjHfNlwJqZWQCDQ", "Date": "Wed, 1 May 2019 14:56:48 +0000", "Message-ID": "<VE1PR08MB51491622B8794D6E8A1DA833983B0@VE1PR08MB5149.eurprd08.prod.outlook.com>", "References": "<20181122033055.3431-1-honnappa.nagarahalli@arm.com>\n\t<20190501035419.33524-1-honnappa.nagarahalli@arm.com>\n\t<20190501121545.GA26521@hmswarspite.think-freely.org>", "In-Reply-To": "<20190501121545.GA26521@hmswarspite.think-freely.org>", "Accept-Language": "en-US", "Content-Language": "en-US", "X-MS-Has-Attach": "", "X-MS-TNEF-Correlator": "", "authentication-results": "spf=none (sender IP is )\n\tsmtp.mailfrom=Honnappa.Nagarahalli@arm.com; ", "x-originating-ip": "[217.140.111.135]", "x-ms-publictraffictype": "Email", "x-ms-office365-filtering-correlation-id": "efa6a3d4-967c-43bc-d56d-08d6ce453cc3", "x-ms-office365-filtering-ht": "Tenant", "x-microsoft-antispam": "BCL:0; PCL:0;\n\tRULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(4618075)(2017052603328)(7193020);\n\tSRVR:VE1PR08MB5104; ", "x-ms-traffictypediagnostic": "VE1PR08MB5104:", "x-ms-exchange-purlcount": "2", "nodisclaimer": "True", "x-microsoft-antispam-prvs": "<VE1PR08MB51040A6425E7DB2687810B89983B0@VE1PR08MB5104.eurprd08.prod.outlook.com>", "x-ms-oob-tlc-oobclassifiers": "OLM:10000;", "x-forefront-prvs": "00246AB517", "x-forefront-antispam-report": "SFV:NSPM;\n\tSFS:(10009020)(366004)(199004)(189003)(55016002)(53936002)(6306002)(33656002)(9686003)(6246003)(229853002)(66476007)(54906003)(4326008)(99286004)(25786009)(7696005)(6436002)(476003)(6916009)(52536014)(14454004)(11346002)(5660300002)(446003)(26005)(6506007)(102836004)(66066001)(486006)(2906002)(71190400001)(73956011)(256004)(8936002)(76176011)(81166006)(3846002)(14444005)(305945005)(498600001)(68736007)(76116006)(74316002)(66946007)(66446008)(966005)(86362001)(66556008)(64756008)(8676002)(72206003)(81156014)(71200400001)(7736002)(6116002)(186003);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB5104;\n\tH:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en;\n\tPTR:InfoNoRecords; A:1; MX:1; ", "received-spf": "None (protection.outlook.com: arm.com does not designate\n\tpermitted sender hosts)", "x-ms-exchange-senderadcheck": "1", "x-microsoft-antispam-message-info": "Cab4Vhc5ohfbsN8TklR6y32RmPbKpA1/V1ErPAWR07L+7VCRGlEDFIa9XlBFusJ/vvznTZdMmX0H0+R409xZwOkw7jATANan/X5ZrfP+yuXLorxUolxkhQQy4+VurVEFt8KDKxTR5Yy7fScvZz3UwFSdwjPdo3AuIEFsT34CEorQ8cMy3QQLmDG/v2WM2eF67gaaIzD8w9Jh3oboNV/4nbsGyCQR8BMrnLlSj4i6e1qnVovuSy3T6AzsrOfqPOJ9+mhA2A6f01/63ZVSqD0B3N+vksJYh6cArKVcsiRFOcOvlPWL+bc7qO5eXQjEa90usupSlPzIWvX+bHJFqyMbKwuD1Wl4GcBt9rHjPBVRgxEAcZy1+oOm0pbIe5Y9RktCYOJskt+spJEWRQ1dXX7QC+fsv+qsTfK8+VE1jkITY6U=", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "quoted-printable", "MIME-Version": "1.0", "X-OriginatorOrg": "arm.com", "X-MS-Exchange-CrossTenant-Network-Message-Id": "efa6a3d4-967c-43bc-d56d-08d6ce453cc3", "X-MS-Exchange-CrossTenant-originalarrivaltime": "01 May 2019 14:56:48.5244\n\t(UTC)", "X-MS-Exchange-CrossTenant-fromentityheader": "Hosted", "X-MS-Exchange-CrossTenant-id": "f34e5979-57d9-4aaa-ad4d-b122a662184d", "X-MS-Exchange-CrossTenant-mailboxtype": "HOSTED", "X-MS-Exchange-Transport-CrossTenantHeadersStamped": "VE1PR08MB5104", "Subject": "Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 95720, "web_url": "http://patches.dpdk.org/comment/95720/", "msgid": "<20190501180524.GC26521@hmswarspite.think-freely.org>", "list_archive_url": "https://inbox.dpdk.org/dev/20190501180524.GC26521@hmswarspite.think-freely.org", "date": "2019-05-01T18:05:24", "subject": "Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "submitter": { "id": 32, "url": "http://patches.dpdk.org/api/people/32/?format=api", "name": "Neil Horman", "email": "nhorman@tuxdriver.com" }, "content": "On Wed, May 01, 2019 at 02:56:48PM +0000, Honnappa Nagarahalli wrote:\n> > \n> > On Tue, Apr 30, 2019 at 10:54:15PM -0500, Honnappa Nagarahalli wrote:\n> > > Lock-less data structures provide scalability and determinism.\n> > > They enable use cases where locking may not be allowed (for ex:\n> > > real-time applications).\n> > >\n> > I know this is version 9 of the patch, so I'm sorry for the late comment, but I\n> > have to ask: Why re-invent this wheel? There are already several Userspace\n> Thanks Neil, for asking the question. This has been debated before. Please refer to [2] for more details.\n> \n> liburcu [1] was explored as it seemed to be familiar to others in the community . I am not aware of any other library.\n> \n> There are unique requirements in DPDK and there is still scope for improvement from what is available. I have explained this in the cover letter without making a direct comparison to liburcu. May be it is worth tweaking the documentation to call this out explicitly.\n> \nI think what you're referring to here is the need for multiple QSBR variables,\nyes? I'm not sure thats, strictly speaking, a requirement. It seems like its a\nperformance improvement, but I'm not sure thats the case (see performance\nnumbers below).\n\nRegarding performance, we can't keep using raw performance as a trump card for\nall other aspects of the DPDK. This entire patch is meant to improve\nperformance, it seems like it would be worthwhile to gain the code consolidation\nand reuse benefits for the minor performance hit.\n\nFurther to performance, I may be misreading this, but I ran the integrated\nperformance test you provided in this patch, as well as the benchmark tests for\nliburcw (trimmed for easier reading here)\n\nliburcw:\n[nhorman@hmswarspite benchmark]$ ./test_urcu 7 1 1 -v -a 0 -a 1 -a 2 -a 3 -a 4 -a 5 -a 6 -a 7 -a 0\nAdding CPU 0 affinity\nAdding CPU 1 affinity\nAdding CPU 2 affinity\nAdding CPU 3 affinity\nAdding CPU 4 affinity\nAdding CPU 5 affinity\nAdding CPU 6 affinity\nAdding CPU 7 affinity\nAdding CPU 0 affinity\nrunning test for 1 seconds, 7 readers, 1 writers.\nWriter delay : 0 loops.\nReader duration : 0 loops.\nthread main , tid 22712\nthread_begin reader, tid 22726\nthread_begin reader, tid 22729\nthread_begin reader, tid 22728\nthread_begin reader, tid 22727\nthread_begin reader, tid 22731\nthread_begin reader, tid 22730\nthread_begin reader, tid 22732\nthread_begin writer, tid 22733\nthread_end reader, tid 22729\nthread_end reader, tid 22731\nthread_end reader, tid 22730\nthread_end reader, tid 22728\nthread_end reader, tid 22727\nthread_end writer, tid 22733\nthread_end reader, tid 22726\nthread_end reader, tid 22732\ntotal number of reads : 1185640774, writes 264444\nSUMMARY /home/nhorman/git/userspace-rcu/tests/benchmark/.libs/lt-test_urcu testdur 1 nr_readers 7 rdur 0 wdur 0 nr_writers 1 wdelay 0 nr_reads 1185640774 nr_writes 264444 nr_ops 1185905218\n\nDPDK:\nPerf test: 1 writer, 7 readers, 1 QSBR variable, 1 QSBR Query, Non-Blocking QSBR check\nFollowing numbers include calls to rte_hash functions\nCycles per 1 update(online/update/offline): 813407\nCycles per 1 check(start, check): 859679\n\n\nBoth of these tests qsbr rcu in each library using 7 readers and 1 writer. Its\na little bit of an apples to oranges comparison, as the tests run using slightly\ndifferent parameters, and produce different output statistics, but I think they\ncan be somewhat normalized. Primarily the stat that stuck out to me was the\nDPDK Cycles per 1 update statistic, which I believe is effectively the number of\ncycles spent in the test / the number of writer updates. On DPDK that number in\nthis test run works out to 813407. In the liburcw test, it reports the total\nnumber of ops (cycles), and the number of writes completed within those cycles.\nIf we do the same division there we get 185905218 / 264444 = 4484. I may be\nmisreading something here, but that seems like a pretty significant write side\nperformance improvement over this implementation.\n\nNeil\n \n> [1] https://liburcu.org/\n> [2] http://mails.dpdk.org/archives/dev/2018-November/119875.html\n> \n> > RCU libraries that are mature and carried by Linux and BSD distributions.\n> > Why would we throw another one into DPDK instead of just using whats\n> > already available, mature and stable?\n> > \n> > Neil\n> \n>", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 323271DBF;\n\tWed, 1 May 2019 20:06:11 +0200 (CEST)", "from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58])\n\tby dpdk.org (Postfix) with ESMTP id 74F4A1DBC\n\tfor <dev@dpdk.org>; Wed, 1 May 2019 20:06:10 +0200 (CEST)", "from cpe-2606-a000-111b-405a-0-0-0-162e.dyn6.twc.com\n\t([2606:a000:111b:405a::162e] helo=localhost)\n\tby smtp.tuxdriver.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.63)\n\t(envelope-from <nhorman@tuxdriver.com>)\n\tid 1hLtcA-0005Gz-0r; Wed, 01 May 2019 14:05:59 -0400" ], "Date": "Wed, 1 May 2019 14:05:24 -0400", "From": "Neil Horman <nhorman@tuxdriver.com>", "To": "Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>", "Cc": "\"konstantin.ananyev@intel.com\" <konstantin.ananyev@intel.com>,\n\t\"stephen@networkplumber.org\" <stephen@networkplumber.org>,\n\t\"paulmck@linux.ibm.com\" <paulmck@linux.ibm.com>,\n\t\"marko.kovacevic@intel.com\" <marko.kovacevic@intel.com>,\n\t\"dev@dpdk.org\" <dev@dpdk.org>,\n\t\"Gavin Hu (Arm Technology China)\" <Gavin.Hu@arm.com>,\n\tDharmik Thakkar <Dharmik.Thakkar@arm.com>,\n\tMalvika Gupta <Malvika.Gupta@arm.com>, nd <nd@arm.com>", "Message-ID": "<20190501180524.GC26521@hmswarspite.think-freely.org>", "References": "<20181122033055.3431-1-honnappa.nagarahalli@arm.com>\n\t<20190501035419.33524-1-honnappa.nagarahalli@arm.com>\n\t<20190501121545.GA26521@hmswarspite.think-freely.org>\n\t<VE1PR08MB51491622B8794D6E8A1DA833983B0@VE1PR08MB5149.eurprd08.prod.outlook.com>", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=us-ascii", "Content-Disposition": "inline", "In-Reply-To": "<VE1PR08MB51491622B8794D6E8A1DA833983B0@VE1PR08MB5149.eurprd08.prod.outlook.com>", "User-Agent": "Mutt/1.11.3 (2019-02-01)", "X-Spam-Score": "-2.9 (--)", "X-Spam-Status": "No", "Subject": "Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 95723, "web_url": "http://patches.dpdk.org/comment/95723/", "msgid": "<VE1PR08MB5149FC5995E423C89C793AC8983B0@VE1PR08MB5149.eurprd08.prod.outlook.com>", "list_archive_url": "https://inbox.dpdk.org/dev/VE1PR08MB5149FC5995E423C89C793AC8983B0@VE1PR08MB5149.eurprd08.prod.outlook.com", "date": "2019-05-01T21:18:05", "subject": "Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "submitter": { "id": 1045, "url": "http://patches.dpdk.org/api/people/1045/?format=api", "name": "Honnappa Nagarahalli", "email": "honnappa.nagarahalli@arm.com" }, "content": "> Subject: Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n> QSBR mechanism\n> \n> On Wed, May 01, 2019 at 02:56:48PM +0000, Honnappa Nagarahalli wrote:\n> > >\n> > > On Tue, Apr 30, 2019 at 10:54:15PM -0500, Honnappa Nagarahalli wrote:\n> > > > Lock-less data structures provide scalability and determinism.\n> > > > They enable use cases where locking may not be allowed (for ex:\n> > > > real-time applications).\n> > > >\n> > > I know this is version 9 of the patch, so I'm sorry for the late\n> > > comment, but I have to ask: Why re-invent this wheel? There are\n> > > already several Userspace\n> > Thanks Neil, for asking the question. This has been debated before. Please\n> refer to [2] for more details.\n> >\n> > liburcu [1] was explored as it seemed to be familiar to others in the\n> community . I am not aware of any other library.\n> >\n> > There are unique requirements in DPDK and there is still scope for\n> improvement from what is available. I have explained this in the cover letter\n> without making a direct comparison to liburcu. May be it is worth tweaking the\n> documentation to call this out explicitly.\n> >\n> I think what you're referring to here is the need for multiple QSBR variables,\n> yes? I'm not sure thats, strictly speaking, a requirement. It seems like its a\n> performance improvement, but I'm not sure thats the case (see performance\n> numbers below).\nDPDK supports service cores feature and pipeline mode where a particular data structure is used by a subset of readers. These use cases affect the writer and reader (which are on the data plane) in the following ways:\n\n1) The writer does not need to wait for all the readers to complete the quiescent state. Writer does not need to spend CPU cycles and add to memory bandwidth polling the unwanted readers. DPDK has uses cases where the writer is on the data plane as well.\n2) The readers that do not use the data structure do not have to spend cycles reporting their quiescent state. Note that these are data plane cycles\n\nOther than this, please read about how grace period and critical section affect the over head introduced by QSBR mechanism in the cover letter. It also explains how this library solves this issue.\n\nThis is discussed in the discussion thread I provided earlier.\n\n> \n> Regarding performance, we can't keep using raw performance as a trump card\nIMO, performance is NOT a 'trump card'. The whole essence of DPDK is performance. If not for performance, would DPDK exist?\n\n> for all other aspects of the DPDK. This entire patch is meant to improve\n> performance, it seems like it would be worthwhile to gain the code\n> consolidation and reuse benefits for the minor performance hit.\nApologies, I did not understand this. Can you please elaborate code consolidation part?\n\n> \n> Further to performance, I may be misreading this, but I ran the integrated\n> performance test you provided in this patch, as well as the benchmark tests for\n> liburcw (trimmed for easier reading here)\nJust to be sure, I believe you are referring to *liburcu*\n\n> \n> liburcw:\n> [nhorman@hmswarspite benchmark]$ ./test_urcu 7 1 1 -v -a 0 -a 1 -a 2 -a 3 -a\n> 4 -a 5 -a 6 -a 7 -a 0 Adding CPU 0 affinity Adding CPU 1 affinity Adding CPU 2\n> affinity Adding CPU 3 affinity Adding CPU 4 affinity Adding CPU 5 affinity\n> Adding CPU 6 affinity Adding CPU 7 affinity Adding CPU 0 affinity running test\n> for 1 seconds, 7 readers, 1 writers.\n> Writer delay : 0 loops.\n> Reader duration : 0 loops.\n> thread main , tid 22712\n> thread_begin reader, tid 22726\n> thread_begin reader, tid 22729\n> thread_begin reader, tid 22728\n> thread_begin reader, tid 22727\n> thread_begin reader, tid 22731\n> thread_begin reader, tid 22730\n> thread_begin reader, tid 22732\n> thread_begin writer, tid 22733\n> thread_end reader, tid 22729\n> thread_end reader, tid 22731\n> thread_end reader, tid 22730\n> thread_end reader, tid 22728\n> thread_end reader, tid 22727\n> thread_end writer, tid 22733\n> thread_end reader, tid 22726\n> thread_end reader, tid 22732\n> total number of reads : 1185640774, writes 264444\n> SUMMARY /home/nhorman/git/userspace-rcu/tests/benchmark/.libs/lt-\n> test_urcu testdur 1 nr_readers 7 rdur 0 wdur 0 nr_writers 1 wdelay\n> 0 nr_reads 1185640774 nr_writes 264444 nr_ops 1185905218\n> \n> DPDK:\n> Perf test: 1 writer, 7 readers, 1 QSBR variable, 1 QSBR Query, Non-Blocking\n> QSBR check Following numbers include calls to rte_hash functions Cycles per 1\n> update(online/update/offline): 813407 Cycles per 1 check(start, check):\n> 859679\n> \n> \n> Both of these tests qsbr rcu in each library using 7 readers and 1 writer. Its a\n> little bit of an apples to oranges comparison, as the tests run using slightly\nThanks for running the test. Yes, it is apples to oranges comparison:\n1) The test you are running is not the correct test assuming the code for this test is [3]\n2) This is not QSBR\n\nI suggest you use [4] for your testing. It also need further changes to match the test case in this patch. The function 'thr_reader' reports quiescent state every 1024 iterations, please change it to report every iteration.\n\nAfter this you need to compare these results with the first test case in this patch.\n\n[3] https://github.com/urcu/userspace-rcu/blob/master/tests/benchmark/test_urcu.c\n[4] https://github.com/urcu/userspace-rcu/blob/master/tests/benchmark/test_urcu_qsbr.c\n\n> different parameters, and produce different output statistics, but I think they\n> can be somewhat normalized. Primarily the stat that stuck out to me was the\n> DPDK Cycles per 1 update statistic, which I believe is effectively the number of\n> cycles spent in the test / the number of writer updates. On DPDK that number\n> in this test run works out to 813407. In the liburcw test, it reports the total\n> number of ops (cycles), and the number of writes completed within those\n> cycles.\n> If we do the same division there we get 185905218 / 264444 = 4484. I may be\n> misreading something here, but that seems like a pretty significant write side\nYes, you are misreading. 'number of ops' is not cycles. It is sum of 'nr_writes' and 'nr_reads'. The test runs for 1 sec (uses 'sleep'), so these are number of operations done in 1 sec. You need to normalize to number of cycles using this data.\n\n> performance improvement over this implementation.\n> \n> Neil\n> \n> > [1] https://liburcu.org/\n> > [2] http://mails.dpdk.org/archives/dev/2018-November/119875.html\n> >\n> > > RCU libraries that are mature and carried by Linux and BSD distributions.\n> > > Why would we throw another one into DPDK instead of just using whats\n> > > already available, mature and stable?\n> > >\n> > > Neil\n> >\n> >", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 8BC15137C;\n\tWed, 1 May 2019 23:18:10 +0200 (CEST)", "from EUR01-VE1-obe.outbound.protection.outlook.com\n\t(mail-eopbgr140050.outbound.protection.outlook.com [40.107.14.50])\n\tby dpdk.org (Postfix) with ESMTP id 231C41041\n\tfor <dev@dpdk.org>; Wed, 1 May 2019 23:18:08 +0200 (CEST)", "from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.152) by\n\tVE1PR08MB4847.eurprd08.prod.outlook.com (10.255.113.87) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id\n\t15.20.1835.15; Wed, 1 May 2019 21:18:06 +0000", "from VE1PR08MB5149.eurprd08.prod.outlook.com\n\t([fe80::9b6:3403:4386:78]) by VE1PR08MB5149.eurprd08.prod.outlook.com\n\t([fe80::9b6:3403:4386:78%2]) with mapi id 15.20.1835.018;\n\tWed, 1 May 2019 21:18:06 +0000" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;\n\ts=selector1-arm-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n\tbh=pmaww8u+Ap+NPOQfXvcm8bPGzTHdqTO3ICkgqDOnWBg=;\n\tb=FwzJkLW3M1NIYQVdVkjicEso4+ykyuUqGA3OsvZhG+rf+Q7hD9WTgVCg29ETgNLnhBVsjHAIaDkciufu3o/evpyowIAcW4A1iS31kOX3mD/Z7qNZ8fs228mqxSkbf03u7d3Cq/O7jd1Edv8eiZaRfFwKX2UzcO1cdHP6fyTUiZA=", "From": "Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>", "To": "Neil Horman <nhorman@tuxdriver.com>", "CC": "\"konstantin.ananyev@intel.com\" <konstantin.ananyev@intel.com>,\n\t\"stephen@networkplumber.org\" <stephen@networkplumber.org>,\n\t\"paulmck@linux.ibm.com\" <paulmck@linux.ibm.com>,\n\t\"marko.kovacevic@intel.com\"\n\t<marko.kovacevic@intel.com>, \"dev@dpdk.org\" <dev@dpdk.org>,\n\t\"Gavin Hu (Arm Technology China)\" <Gavin.Hu@arm.com>, Dharmik Thakkar\n\t<Dharmik.Thakkar@arm.com>, Malvika Gupta <Malvika.Gupta@arm.com>,\n\tHonnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>, nd <nd@arm.com>,\n\tnd <nd@arm.com>", "Thread-Topic": "[dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "Thread-Index": "AQHVABe82XMyTZzopkuFSkjHfNlwJqZWQCDQgABQlgCAABhQwA==", "Date": "Wed, 1 May 2019 21:18:05 +0000", "Message-ID": "<VE1PR08MB5149FC5995E423C89C793AC8983B0@VE1PR08MB5149.eurprd08.prod.outlook.com>", "References": "<20181122033055.3431-1-honnappa.nagarahalli@arm.com>\n\t<20190501035419.33524-1-honnappa.nagarahalli@arm.com>\n\t<20190501121545.GA26521@hmswarspite.think-freely.org>\n\t<VE1PR08MB51491622B8794D6E8A1DA833983B0@VE1PR08MB5149.eurprd08.prod.outlook.com>\n\t<20190501180524.GC26521@hmswarspite.think-freely.org>", "In-Reply-To": "<20190501180524.GC26521@hmswarspite.think-freely.org>", "Accept-Language": "en-US", "Content-Language": "en-US", "X-MS-Has-Attach": "", "X-MS-TNEF-Correlator": "", "authentication-results": "spf=none (sender IP is )\n\tsmtp.mailfrom=Honnappa.Nagarahalli@arm.com; ", "x-originating-ip": "[217.140.111.135]", "x-ms-publictraffictype": "Email", "x-ms-office365-filtering-correlation-id": "8f088416-fd07-41f5-1a2b-08d6ce7a80bf", "x-ms-office365-filtering-ht": "Tenant", "x-microsoft-antispam": "BCL:0; PCL:0;\n\tRULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(4618075)(2017052603328)(7193020);\n\tSRVR:VE1PR08MB4847; ", "x-ms-traffictypediagnostic": "VE1PR08MB4847:", "x-ms-exchange-purlcount": "4", "nodisclaimer": "True", "x-microsoft-antispam-prvs": "<VE1PR08MB4847E1F11D20BE54815DD05A983B0@VE1PR08MB4847.eurprd08.prod.outlook.com>", "x-ms-oob-tlc-oobclassifiers": "OLM:10000;", "x-forefront-prvs": "00246AB517", "x-forefront-antispam-report": "SFV:NSPM;\n\tSFS:(10009020)(136003)(39860400002)(366004)(346002)(396003)(376002)(199004)(189003)(229853002)(66066001)(186003)(478600001)(256004)(14454004)(71190400001)(71200400001)(26005)(14444005)(446003)(476003)(11346002)(486006)(966005)(5660300002)(4326008)(6506007)(76176011)(7696005)(66476007)(2906002)(72206003)(6436002)(81156014)(99286004)(102836004)(25786009)(6246003)(305945005)(7736002)(54906003)(52536014)(9686003)(66446008)(64756008)(86362001)(73956011)(66556008)(55016002)(76116006)(53936002)(81166006)(316002)(33656002)(66946007)(6116002)(74316002)(8936002)(8676002)(6306002)(68736007)(3846002)(6916009);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4847;\n\tH:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en;\n\tPTR:InfoNoRecords; A:1; MX:1; ", "received-spf": "None (protection.outlook.com: arm.com does not designate\n\tpermitted sender hosts)", "x-ms-exchange-senderadcheck": "1", "x-microsoft-antispam-message-info": "TRK5wkyOkHEb2Qdyl0T3GNPyXevkAJYwYSaFt7qzO+HL51szgA7FSB/EW6+WjUdlvLQsQBZQduzLMN9tmS4zLMfk1SlDKfWeqRUbkCzaafvrJUfQykk08HVK9iWLEp4KevMWKAiNd7+if3+bzVrlQR6VDaoDqCv2mJZjYvMe5+GXucpwKiOxU9UkJub2xsMaT/bRanLE9EKH6WBVR3pZh8O44hfG3QpCmzWHrtXehsmmBt/1WuaW1ff0pp1I+rcgYJVxLG+pU2E1eXuQUstoh/k+0X7X8abtKPRmEJn2bJjvUcFhAFjrktSxNbAjPKv8ioAVV1ubfjfmWapFFtdRXf7lIbOVdilTRK73HiRQdZ2T22P3izLlBBYZOrX6EHlVlM0n33XkgQDTNMpV1ptVenteP++FdpRIgF9cGoHa4MM=", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "quoted-printable", "MIME-Version": "1.0", "X-OriginatorOrg": "arm.com", "X-MS-Exchange-CrossTenant-Network-Message-Id": "8f088416-fd07-41f5-1a2b-08d6ce7a80bf", "X-MS-Exchange-CrossTenant-originalarrivaltime": "01 May 2019 21:18:05.9342\n\t(UTC)", "X-MS-Exchange-CrossTenant-fromentityheader": "Hosted", "X-MS-Exchange-CrossTenant-id": "f34e5979-57d9-4aaa-ad4d-b122a662184d", "X-MS-Exchange-CrossTenant-mailboxtype": "HOSTED", "X-MS-Exchange-Transport-CrossTenantHeadersStamped": "VE1PR08MB4847", "Subject": "Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 95727, "web_url": "http://patches.dpdk.org/comment/95727/", "msgid": "<2437939.GGFfNdu4dS@xps>", "list_archive_url": "https://inbox.dpdk.org/dev/2437939.GGFfNdu4dS@xps", "date": "2019-05-01T23:36:37", "subject": "Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "submitter": { "id": 685, "url": "http://patches.dpdk.org/api/people/685/?format=api", "name": "Thomas Monjalon", "email": "thomas@monjalon.net" }, "content": "01/05/2019 05:54, Honnappa Nagarahalli:\n> Dharmik Thakkar (1):\n> test/rcu_qsbr: add API and functional tests\n> \n> Honnappa Nagarahalli (3):\n> rcu: add RCU library supporting QSBR mechanism\n> doc/rcu: add lib_rcu documentation\n> doc: added RCU to the release notes\n\nApplied with discussed doc changes, thanks.", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 0A7C7374;\n\tThu, 2 May 2019 01:36:44 +0200 (CEST)", "from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com\n\t[66.111.4.28]) by dpdk.org (Postfix) with ESMTP id 7AAE6A3\n\tfor <dev@dpdk.org>; Thu, 2 May 2019 01:36:43 +0200 (CEST)", "from compute1.internal (compute1.nyi.internal [10.202.2.41])\n\tby mailout.nyi.internal (Postfix) with ESMTP id A651C23943;\n\tWed, 1 May 2019 19:36:41 -0400 (EDT)", "from mailfrontend2 ([10.202.2.163])\n\tby compute1.internal (MEProxy); Wed, 01 May 2019 19:36:41 -0400", "from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184])\n\tby mail.messagingengine.com (Postfix) with ESMTPA id 1337B103CC;\n\tWed, 1 May 2019 19:36:38 -0400 (EDT)" ], "DKIM-Signature": [ "v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h=\n\tfrom:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding:content-type; s=mesmtp;\n\tbh=S7eC87qcVGsvJJxj/QtDhKVS8lNdGoYVXI8dO5Zio0E=; b=qQou2ukwOgVF\n\tYtHauZpCV+EmaFokaiiwr7+ElmBU71Q0le2KhKoud3vy63aVHc7Se1VL3AZ45tQA\n\tu4bQnMe83kiL0F0cNkIj2R0WLnuKs6MRonF6D5OcBBEOAw6cIy0IbwVGIOKlztCC\n\tC9+t7yumkNOS18hUgga72jixriMex8U=", "v=1; a=rsa-sha256; c=relaxed/relaxed; d=\n\tmessagingengine.com; h=cc:content-transfer-encoding:content-type\n\t:date:from:in-reply-to:message-id:mime-version:references\n\t:subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender\n\t:x-sasl-enc; s=fm2; bh=S7eC87qcVGsvJJxj/QtDhKVS8lNdGoYVXI8dO5Zio\n\t0E=; b=QYjbN3xrkAvaTNAETwRDtoENp9Fg/RfZ29gLEAVUDccBZn4Bk9Gk1358i\n\tIgBoLp+J7lqKqCKCQSkXafJPvMupc78QpG/B05hKwt8NFWtefQ9kYOENsFoTLyLn\n\tsEiatRZvtaCMryfpno4UdnvdJjXQxaiN9ySmdMNlhzpzL8aKpYgfI0CO2xxhM57F\n\tfu94VW7jGIDjr5LuFIG4LZzR6olhy929lD1CNF7NdeI1S2P1kqX03OVgYwyy9yPK\n\twe3AoCanJLM+WuDuxGtbUii/tsXE6Y9UZC5PqssVysHq3N6Qa7im3lfOuJjREX59\n\tYCRdcD4egvGpvxCsb6cLaCQVjor/A==" ], "X-ME-Sender": "<xms:iC3KXDum_zDpQTt6cytXyuKIRV0r6xcdIRyPJHZhEGUd4G5malhCzg>", "X-ME-Proxy-Cause": "gggruggvucftvghtrhhoucdtuddrgeduuddrieekgdduhecutefuodetggdotefrodftvf\n\tcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu\n\tuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc\n\tfjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhmrghs\n\tucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucfkph\n\tepjeejrddufeegrddvtdefrddukeegnecurfgrrhgrmhepmhgrihhlfhhrohhmpehthhho\n\tmhgrshesmhhonhhjrghlohhnrdhnvghtnecuvehluhhsthgvrhfuihiivgeptd", "X-ME-Proxy": "<xmx:iC3KXJvSlQMq5EU_6WvBjs4q2IGapDDUSTs4wx3ZKPhIL-NgdHuxZQ>\n\t<xmx:iC3KXIeNeYteLCuRI8yVw9AiPI3A51yGwPfnt5ZYfPmHq5muuW0SLg>\n\t<xmx:iC3KXKaCpVYROGnmsLmAAJuXqd9s400vJzudw1XF6bdsbYpBHwjq0w>\n\t<xmx:iS3KXDUIsGLJVUp-I4TGctyyKpu89UI4k5_PjLdweLNoH4sNe2lH-Q>", "From": "Thomas Monjalon <thomas@monjalon.net>", "To": "Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>", "Cc": "dev@dpdk.org, konstantin.ananyev@intel.com, stephen@networkplumber.org, \n\tpaulmck@linux.ibm.com, marko.kovacevic@intel.com, gavin.hu@arm.com,\n\tdharmik.thakkar@arm.com, malvika.gupta@arm.com", "Date": "Thu, 02 May 2019 01:36:37 +0200", "Message-ID": "<2437939.GGFfNdu4dS@xps>", "In-Reply-To": "<20190501035419.33524-1-honnappa.nagarahalli@arm.com>", "References": "<20181122033055.3431-1-honnappa.nagarahalli@arm.com>\n\t<20190501035419.33524-1-honnappa.nagarahalli@arm.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "7Bit", "Content-Type": "text/plain; charset=\"us-ascii\"", "Subject": "Re: [dpdk-dev] [PATCH v9 0/4] lib/rcu: add RCU library supporting\n\tQSBR mechanism", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null } ]