From patchwork Tue Sep 15 19:34:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 77811 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5EE34A04C7; Tue, 15 Sep 2020 21:35:11 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 05F7D1C125; Tue, 15 Sep 2020 21:35:03 +0200 (CEST) Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id 6B0941C0DB for ; Tue, 15 Sep 2020 21:34:59 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20200915193458euoutp01510d4eaf4582566c7e6c131fef69a6bb~1DA5pRMOT1858218582euoutp012 for ; Tue, 15 Sep 2020 19:34:58 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20200915193458euoutp01510d4eaf4582566c7e6c131fef69a6bb~1DA5pRMOT1858218582euoutp012 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1600198498; bh=eymXJujej9PG1zGL1zp62MJRAkbOY/KHhs1VGhD++7o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Bi9W1JgsuO3coND+uz+Wcg+noBHmkQcHC2rdtZM4s1hbC9rsZWxh7bwrEGiU9sI7R gUwH2qnO7uCck8t5Ft5D8CHrceBvIe2Vj0kwx/yic5Xb2o9uG1MlIt0u/zZYHwqu4Q MjKuXaWJll/cJj1LhCExuIqbaQQ3wKOMFpxgdJio= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20200915193458eucas1p284fce0e944445a8b5178e11fee8345c4~1DA5bYxUg0572705727eucas1p2i; Tue, 15 Sep 2020 19:34:58 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id AE.C6.06318.167116F5; Tue, 15 Sep 2020 20:34:58 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20200915193457eucas1p2adbe25c41a0e4ef16c029e7bff104503~1DA4OMK9D0573405734eucas1p2o; Tue, 15 Sep 2020 19:34:57 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20200915193457eusmtrp157bc5606dc50e94efea6eed9a7f3ab6e~1DA4NAo-j1473114731eusmtrp18; Tue, 15 Sep 2020 19:34:57 +0000 (GMT) X-AuditID: cbfec7f5-371ff700000018ae-24-5f611761b579 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 59.6C.06017.167116F5; Tue, 15 Sep 2020 20:34:57 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20200915193456eusmtip2258564de43d3832eafc8bce3d3ea91d6~1DA3rid021759817598eusmtip2f; Tue, 15 Sep 2020 19:34:56 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Tue, 15 Sep 2020 21:34:44 +0200 Message-Id: <20200915193449.13310-2-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrKIsWRmVeSWpSXmKPExsWy7djP87pJ4onxBnP2MFncWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHMUl01Kak5mWWqRvl0CV0bLqgdM BR8lK1bu3cXYwHhTpIuRk0NCwETi7NW97F2MXBxCAisYJc7PesIK4XxhlFj27QxU5jOjxJR5 XYwwLX/P3IGqWs4o0TfhCQtIQkjgE6PEmReqIDabgK3EkZlfWUFsEYEwiebmvUA1HBzMAs4S T76ygZjCAk4ShxrSQCpYBFQlji3oZQKxeQVcJTau+csCsUpeYvWGA8wgNqeAm8TVj7OZQdZK CFxnk5h+YTcTyBwJAReJM0cKIeqFJV4d38IOYctInJ7cwwJRv41R4urvn4wQzn5Gieu9K6Cq rCUO//vNBnGbpsT6XfoQYUeJiX9OM0LM55O48VYQJMwMZE7aNp0ZIswr0dEmBFGtJ/G0Zyoj zNo/a59Ane8hsebVXWZISM1klPi4aAXrBEb5WQjLFjAyrmIUTy0tzk1PLTbOSy3XK07MLS7N S9dLzs/dxAiM/NP/jn/dwbjvT9IhRgEORiUe3gW3E+KFWBPLiitzDzFKcDArifA6nT0dJ8Sb klhZlVqUH19UmpNafIhRmoNFSZzXeNHLWCGB9MSS1OzU1ILUIpgsEwenVAPjbqPFim5Jxk9y q2vKLCTfZtdk6K5v2hC62kBj28HTCUdKi2ff/f13r5i+fkXFV4VYo4N6e38E/rZWj30j4mmp uyecw9x/1xfJm7OOTnwR6a5uu3DipOcVb96sXNYRyBaRe2KbpsB7iZdLfE/X2X20uDLRPG/O HP2tHxddmjsv2oTveMXHWq37SizFGYmGWsxFxYkAKvaY5/gCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrGLMWRmVeSWpSXmKPExsVy+t/xe7qJ4onxBidnsljcWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHOUnk1RfmlJqkJGfnGJrVK0oYWR nqGlhZ6RiaWeobF5rJWRqZK+nU1Kak5mWWqRvl2CXkbLqgdMBR8lK1bu3cXYwHhTpIuRk0NC wETi75k7rF2MXBxCAksZJV7euc7YxcgBlJCR+HBJAKJGWOLPtS42iJoPjBJvNxxhB0mwCdhK HJn5lRWkXkQgTOLESn+QMLOAu8SWxVOZQcLCAk4ShxrSQMIsAqoSxxb0MoHYvAKuEhvX/GWB GC8vsXrDAWYQm1PATeLqx9lgthBQzeKT59knMPItYGRYxSiSWlqcm55bbKRXnJhbXJqXrpec n7uJERiG24793LKDsetd8CFGAQ5GJR7eBbcT4oVYE8uKK3MPMUpwMCuJ8DqdPR0nxJuSWFmV WpQfX1Sak1p8iNEU6KiJzFKiyfnAGMkriTc0NTS3sDQ0NzY3NrNQEuftEDgYIySQnliSmp2a WpBaBNPHxMEp1cAorW817dAz3uViO1Z3JBi8rOrZuuCorLu9cxhzqGfGrM5cf+d5yusjDRgV XxT+mnn5qMONmt7+16v3fpp/X7eyjlUmcrHyrd7/L3sy7DlmL3y3KnrhZPdFfof8jZZWqHlN 1hbfeMUmvyiUk9GtZ0OpEXssb/CKMJeLHUt5F5VP9/TUDpx4jU+JpTgj0VCLuag4EQCu+8jB WQIAAA== X-CMS-MailID: 20200915193457eucas1p2adbe25c41a0e4ef16c029e7bff104503 X-Msg-Generator: CA X-RootMTR: 20200915193457eucas1p2adbe25c41a0e4ef16c029e7bff104503 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20200915193457eucas1p2adbe25c41a0e4ef16c029e7bff104503 References: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v1 1/6] app/test: fix deadlock in distributor test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The sanity test with worker shutdown delegates all bufs to be processed by a single lcore worker, then it freezes one of the lcore workers and continues to send more bufs. Problem occurred if freezed lcore is the same as the one that is processing the mbufs. The lcore processing mbufs might be different every time test is launched. This is caused by keeping the value of wkr static variable in rte_distributor_process function between running test cases. Test freezed always lcore with 0 id. The patch changes avoids possible collision by freezing lcore with zero_idx. The lcore that receives the data updates the zero_idx, so it is not freezed itself. To reproduce the issue fixed by this patch, please run distributor_autotest command in test app several times in a row. Fixes: c3eabff124e6 ("distributor: add unit tests") Cc: bruce.richardson@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Tested-by: David Hunt --- app/test/test_distributor.c | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index ba1f81cf8..35b25463a 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -28,6 +28,7 @@ struct worker_params worker_params; static volatile int quit; /**< general quit variable for all threads */ static volatile int zero_quit; /**< var for when we just want thr0 to quit*/ static volatile unsigned worker_idx; +static volatile unsigned zero_idx; struct worker_stats { volatile unsigned handled_packets; @@ -346,27 +347,43 @@ handle_work_for_shutdown_test(void *arg) unsigned int total = 0; unsigned int i; unsigned int returned = 0; + unsigned int zero_id = 0; const unsigned int id = __atomic_fetch_add(&worker_idx, 1, __ATOMIC_RELAXED); num = rte_distributor_get_pkt(d, id, buf, buf, num); + zero_id = __atomic_load_n(&zero_idx, __ATOMIC_ACQUIRE); + if (id == zero_id && num > 0) { + zero_id = (zero_id + 1) % __atomic_load_n(&worker_idx, + __ATOMIC_ACQUIRE); + __atomic_store_n(&zero_idx, zero_id, __ATOMIC_RELEASE); + } + /* wait for quit single globally, or for worker zero, wait * for zero_quit */ - while (!quit && !(id == 0 && zero_quit)) { + while (!quit && !(id == zero_id && zero_quit)) { worker_stats[id].handled_packets += num; count += num; for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, id, buf, buf, num); + + zero_id = __atomic_load_n(&zero_idx, __ATOMIC_ACQUIRE); + if (id == zero_id && num > 0) { + zero_id = (zero_id + 1) % __atomic_load_n(&worker_idx, + __ATOMIC_ACQUIRE); + __atomic_store_n(&zero_idx, zero_id, __ATOMIC_RELEASE); + } + total += num; } worker_stats[id].handled_packets += num; count += num; returned = rte_distributor_return_pkt(d, id, buf, num); - if (id == 0) { + if (id == zero_id) { /* for worker zero, allow it to restart to pick up last packet * when all workers are shutting down. */ @@ -586,6 +603,7 @@ quit_workers(struct worker_params *wp, struct rte_mempool *p) rte_eal_mp_wait_lcore(); quit = 0; worker_idx = 0; + zero_idx = 0; } static int From patchwork Tue Sep 15 19:34:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 77812 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7B5C7A04C7; Tue, 15 Sep 2020 21:35:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7E1631C12C; Tue, 15 Sep 2020 21:35:04 +0200 (CEST) Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id 6B2391C10A for ; Tue, 15 Sep 2020 21:34:59 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20200915193458euoutp010022215fef8fe2fd5ad2df9867b572cb~1DA5wbq9x1761317613euoutp01_ for ; Tue, 15 Sep 2020 19:34:58 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20200915193458euoutp010022215fef8fe2fd5ad2df9867b572cb~1DA5wbq9x1761317613euoutp01_ DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1600198498; bh=xPWUwWzs8KDd9GRIYyNVFLY6E9FQFwkbk3DM1rTtSlk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=k/kXUkpdkw7riFSrjmtY/fr+J8KnqOU1qk0srN1oaKOLkYI+OSKtsjjckeMHMmqoK p7WRTg62r1NRKXwSE3oVuFKmgF+Sbvd0xSbqQbo0eqqCD9g5K2XWJgbUj3yUS4XCgl 3hUZyyydbarM+/uqgw9O9dT3r4hKqZp/YYVDva/c= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20200915193458eucas1p28e3b5333e24e4c816caf2befb585c59e~1DA5mlBtG0563105631eucas1p2h; Tue, 15 Sep 2020 19:34:58 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 7E.60.05997.267116F5; Tue, 15 Sep 2020 20:34:58 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20200915193457eucas1p2321d28b6abf69f244cd7c1e61ed0620e~1DA5AZzrH3166631666eucas1p29; Tue, 15 Sep 2020 19:34:57 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20200915193457eusmtrp1cc25e5d5d131c08378a5506ffa7e755b~1DA4-2VW41473114731eusmtrp19; Tue, 15 Sep 2020 19:34:57 +0000 (GMT) X-AuditID: cbfec7f4-677ff7000000176d-0c-5f611762be36 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 63.7C.06314.167116F5; Tue, 15 Sep 2020 20:34:57 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20200915193457eusmtip24ddc8686f0599fd010fb3879d3b9e329~1DA4XrXV82767527675eusmtip2p; Tue, 15 Sep 2020 19:34:57 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Tue, 15 Sep 2020 21:34:45 +0200 Message-Id: <20200915193449.13310-3-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrCIsWRmVeSWpSXmKPExsWy7djP87pJ4onxBuuOy1rcWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHMUl01Kak5mWWqRvl0CV8aHhb0s BW91KqZv/c/awLhQpYuRk0NCwERi84ejLF2MXBxCAisYJSaueMYCkhAS+MIosXuHOETiM6PE qw1PmGA6NrWeYoMoWs4osXF3HIT9CajoBVicTcBW4sjMr6wgtohAmERz816goRwczALOEk++ gpUIC7hL/Fh8FGwki4CqxIUlB9hBbF4BV4nr675ArZKXWL3hADOIzSngJnH142xmkHskBC6z Saw4sYQFoshF4vu51+wQtrDEq+NboGwZidOTe1ggGrYxSlz9/ZMRwtnPKHG9dwVUlbXE4X+/ 2SCu05RYv0sfIuwoca1nOiNIWEKAT+LGW0GQMDOQOWnbdGaIMK9ER5sQRLWexNOeqYwwa/+s fcICUeIh0frSCRKEMxkldq3fzDyBUX4Wwq4FjIyrGMVTS4tz01OLjfJSy/WKE3OLS/PS9ZLz czcxAuP+9L/jX3Yw7vqTdIhRgINRiYd3we2EeCHWxLLiytxDjBIczEoivE5nT8cJ8aYkVlal FuXHF5XmpBYfYpTmYFES5zVe9DJWSCA9sSQ1OzW1ILUIJsvEwSnVwDjr0oLu6gkXK6r/1wSW 9jFIKYquZ4+dkTj74t3p1RcnOGWvDniXlfpzw8qI2bf2zq+ct4grPI/7/fpPBsy6FrbOk1r3 7C5nOPZfIaozR+VUwCuxK9HWz0sTwpweiaRK/ru2iquw68fxz5el/JdbCx7IdL3xYPHOE0cL 8vdOyPxcfnCJW976vodKLMUZiYZazEXFiQB/AuSi9wIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrOLMWRmVeSWpSXmKPExsVy+t/xe7qJ4onxBqvazS1urLK36Jv0kcni 3aftTBbPetYxWvzr+MPuwOrxa8FSVo/Fe14yeRx8t4cpgDlKz6Yov7QkVSEjv7jEVina0MJI z9DSQs/IxFLP0Ng81srIVEnfziYlNSezLLVI3y5BL+PDwl6Wgrc6FdO3/mdtYFyo0sXIySEh YCKxqfUUWxcjF4eQwFJGiXsbehm7GDmAEjISHy4JQNQIS/y51gVV84FRYmnLRxaQBJuArcSR mV9ZQepFBMIkTqz0BwkzC7hLbFk8lRnEFgayfyw+ygRiswioSlxYcoAdxOYVcJW4vu4LE8R8 eYnVGw6A1XMKuElc/TgbzBYCqll88jz7BEa+BYwMqxhFUkuLc9Nziw31ihNzi0vz0vWS83M3 MQIDcduxn5t3MF7aGHyIUYCDUYmHd8HthHgh1sSy4srcQ4wSHMxKIrxOZ0/HCfGmJFZWpRbl xxeV5qQWH2I0BTpqIrOUaHI+MErySuINTQ3NLSwNzY3Njc0slMR5OwQOxggJpCeWpGanphak FsH0MXFwSjUwBh47MSH3gaHjra6bFRcNn05Wm77o56mwVyHLDK04Kh8nLqxZysvF9L/6l9Wk t9r5Qi0iXMLXLzxru6I2s3eWrfjBb+nr1zx6HvNgc1p07g6B+1l8Ypp7DA+klXAs6FtwcPHp sjMTwnd+mXjRcO/hPcDg3zFrUnlj8G+b9T1zchfeWDmReX39TiWW4oxEQy3mouJEAL9RtPNa AgAA X-CMS-MailID: 20200915193457eucas1p2321d28b6abf69f244cd7c1e61ed0620e X-Msg-Generator: CA X-RootMTR: 20200915193457eucas1p2321d28b6abf69f244cd7c1e61ed0620e X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20200915193457eucas1p2321d28b6abf69f244cd7c1e61ed0620e References: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v1 2/6] app/test: synchronize statistics between lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Statistics of handled packets are cleared and read on main lcore, while they are increased in workers handlers on different lcores. Without synchronization occasionally showed invalid values. This patch uses atomic acquire/release mechanisms to synchronize. Fixes: c3eabff124e6 ("distributor: add unit tests") Cc: bruce.richardson@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- app/test/test_distributor.c | 39 ++++++++++++++++++++++++------------- 1 file changed, 26 insertions(+), 13 deletions(-) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index 35b25463a..0e49e3714 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -43,7 +43,8 @@ total_packet_count(void) { unsigned i, count = 0; for (i = 0; i < worker_idx; i++) - count += worker_stats[i].handled_packets; + count += __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE); return count; } @@ -52,6 +53,7 @@ static inline void clear_packet_count(void) { memset(&worker_stats, 0, sizeof(worker_stats)); + rte_atomic_thread_fence(__ATOMIC_RELEASE); } /* this is the basic worker function for sanity test @@ -72,13 +74,13 @@ handle_work(void *arg) num = rte_distributor_get_pkt(db, id, buf, buf, num); while (!quit) { __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_RELAXED); + __ATOMIC_ACQ_REL); count += num; num = rte_distributor_get_pkt(db, id, buf, buf, num); } __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_RELAXED); + __ATOMIC_ACQ_REL); count += num; rte_distributor_return_pkt(db, id, buf, num); return 0; @@ -134,7 +136,8 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); printf("Sanity test with all zero hashes done.\n"); /* pick two flows and check they go correctly */ @@ -159,7 +162,9 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n( + &worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); printf("Sanity test with two hash values done\n"); } @@ -185,7 +190,8 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); printf("Sanity test with non-zero hashes done\n"); rte_mempool_put_bulk(p, (void *)bufs, BURST); @@ -280,15 +286,17 @@ handle_work_with_free_mbufs(void *arg) buf[i] = NULL; num = rte_distributor_get_pkt(d, id, buf, buf, num); while (!quit) { - worker_stats[id].handled_packets += num; count += num; + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, id, buf, buf, num); } - worker_stats[id].handled_packets += num; count += num; + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); rte_distributor_return_pkt(d, id, buf, num); return 0; } @@ -363,8 +371,9 @@ handle_work_for_shutdown_test(void *arg) /* wait for quit single globally, or for worker zero, wait * for zero_quit */ while (!quit && !(id == zero_id && zero_quit)) { - worker_stats[id].handled_packets += num; count += num; + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, @@ -379,10 +388,11 @@ handle_work_for_shutdown_test(void *arg) total += num; } - worker_stats[id].handled_packets += num; count += num; returned = rte_distributor_return_pkt(d, id, buf, num); + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); if (id == zero_id) { /* for worker zero, allow it to restart to pick up last packet * when all workers are shutting down. @@ -394,10 +404,11 @@ handle_work_for_shutdown_test(void *arg) id, buf, buf, num); while (!quit) { - worker_stats[id].handled_packets += num; count += num; rte_pktmbuf_free(pkt); num = rte_distributor_get_pkt(d, id, buf, buf, num); + __atomic_fetch_add(&worker_stats[id].handled_packets, + num, __ATOMIC_ACQ_REL); } returned = rte_distributor_return_pkt(d, id, buf, num); @@ -461,7 +472,8 @@ sanity_test_with_worker_shutdown(struct worker_params *wp, for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); if (total_packet_count() != BURST * 2) { printf("Line %d: Error, not all packets flushed. " @@ -514,7 +526,8 @@ test_flush_with_worker_shutdown(struct worker_params *wp, zero_quit = 0; for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); if (total_packet_count() != BURST) { printf("Line %d: Error, not all packets flushed. " From patchwork Tue Sep 15 19:34:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 77813 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 65912A04C7; Tue, 15 Sep 2020 21:35:27 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E2B9B1C132; Tue, 15 Sep 2020 21:35:05 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 1202B1C10D for ; Tue, 15 Sep 2020 21:35:00 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20200915193459euoutp02dae9a1d50afb083c647a841a97d40238~1DA6jB63c2999829998euoutp02a for ; Tue, 15 Sep 2020 19:34:59 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20200915193459euoutp02dae9a1d50afb083c647a841a97d40238~1DA6jB63c2999829998euoutp02a DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1600198499; bh=BVJe8/A3whspe3s69DyHdj0CtB/By4/BhAUSBV03VnQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BkqHbjiw2R82Ra1wNNdDeyn0P6esoCkqUr83fESVC6V4zDVnR4hsmpapHgui4efYX mhuD/qNaP1bfnV+tkenxGND+i4c4cC2mWnndCcMHFz7c40UjQUVSL7SF5UwZjwI4cT v2NCBxevGMAo4rMBB859ryz94RcS+ly+c1+K/z3g= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20200915193458eucas1p1ba92f4c2431dd9bb73315dfa2c310f20~1DA5_ZrER2429224292eucas1p19; Tue, 15 Sep 2020 19:34:58 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id DE.60.05997.267116F5; Tue, 15 Sep 2020 20:34:58 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20200915193458eucas1p1d9308e63063eda28f96eedba3a361a2b~1DA5syDRT1351113511eucas1p1g; Tue, 15 Sep 2020 19:34:58 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20200915193458eusmtrp191bcd90d7dda133db6cd22d802b66ee5~1DA5sTFDj1473114731eusmtrp1-; Tue, 15 Sep 2020 19:34:58 +0000 (GMT) X-AuditID: cbfec7f4-65dff7000000176d-0d-5f611762ba67 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id D9.6C.06017.267116F5; Tue, 15 Sep 2020 20:34:58 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20200915193458eusmtip27bee4959e6e39ac8d0e521b1fdbc6e54~1DA5Ga2mP2964229642eusmtip2W; Tue, 15 Sep 2020 19:34:57 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Tue, 15 Sep 2020 21:34:46 +0200 Message-Id: <20200915193449.13310-4-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrOIsWRmVeSWpSXmKPExsWy7djP87pJ4onxBvvXWlvcWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHMUl01Kak5mWWqRvl0CV8bCPdEF PSoVSza3MTcwPpLpYuTkkBAwkWhs3s/YxcjFISSwglHi78cOVgjnC6NE99s1TBDOZ0aJH007 GWFaHu++wwyRWM4oMf/SfBYI5xOjxO7GR2BVbAK2EkdmfmUFsUUEwiSam/cCFXFwMAs4Szz5 ygYSFhbwkPjbdYMJxGYRUJXYfuUfM4jNK+Aq8WjJMzaIZfISqzccAItzCrhJXP04G2yxhMBl Nombf2axQxS5SPydeRXKFpZ4dXwLlC0jcXpyDwtEwzZGiau/fzJCOPsZJa73roCqspY4/O83 G8R1mhLrd+lDhB0lju5ZCHa0hACfxI23giBhZiBz0rbpzBBhXomONiGIaj2Jpz1TGWHW/ln7 hAXC9pB4fOUUNLBmMkpM3nWBdQKj/CyEZQsYGVcxiqeWFuempxYb5aWW6xUn5haX5qXrJefn bmIERv/pf8e/7GDc9SfpEKMAB6MSD++C2wnxQqyJZcWVuYcYJTiYlUR4nc6ejhPiTUmsrEot yo8vKs1JLT7EKM3BoiTOa7zoZayQQHpiSWp2ampBahFMlomDU6qBcd3NF2+2ZFl9fdntJyaY Pyfy2CPP6Wd1PZs6F3zfvJpHKO3s9Dsu8k8/2ck82JbOPllc/73GK1Ob819/Vn3gY93032ty 0ekUhvfnHvntS4x7udj26PyYjc+mnng1W6FSZenDa3kGZZJCJ/krYqpnF0S95v+1ucNh7n05 hcVt5jUrC/j5TAU5DymxFGckGmoxFxUnAgClaQ5C+gIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrGLMWRmVeSWpSXmKPExsVy+t/xe7pJ4onxBus3qFncWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHOUnk1RfmlJqkJGfnGJrVK0oYWR nqGlhZ6RiaWeobF5rJWRqZK+nU1Kak5mWWqRvl2CXsbCPdEFPSoVSza3MTcwPpLpYuTkkBAw kXi8+w5zFyMXh5DAUkaJ9jsrmboYOYASMhIfLglA1AhL/LnWxQZR84FRYu7jDnaQBJuArcSR mV9ZQepFBMIkTqz0BwkzC7hLbFk8lRnEFhbwkPjbdYMJxGYRUJXYfuUfWJxXwFXi0ZJnbBDz 5SVWbzgAFucUcJO4+nE2mC0EVLP45Hn2CYx8CxgZVjGKpJYW56bnFhvpFSfmFpfmpesl5+du YgSG4bZjP7fsYOx6F3yIUYCDUYmHd8HthHgh1sSy4srcQ4wSHMxKIrxOZ0/HCfGmJFZWpRbl xxeV5qQWH2I0BTpqIrOUaHI+MEbySuINTQ3NLSwNzY3Njc0slMR5OwQOxggJpCeWpGanphak FsH0MXFwSjUwThC+qNG4V2jWN++Nl50820oWde+/bvNo2Yf1BR0BCdP5Zkx9P6kh9oDInRc3 lk/W263n+7d0Z7Wu08mVt9h1XGSn+IbLXnQ/UCOZe7WD7VopV8HkVaEvJ8tzvm6Y3+Okqmm2 4m/nerY1h8T53VVZIk4ErdynGpzVcHLHujlHDIMkxcQTnSRmKbEUZyQaajEXFScCAIJrg5tZ AgAA X-CMS-MailID: 20200915193458eucas1p1d9308e63063eda28f96eedba3a361a2b X-Msg-Generator: CA X-RootMTR: 20200915193458eucas1p1d9308e63063eda28f96eedba3a361a2b X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20200915193458eucas1p1d9308e63063eda28f96eedba3a361a2b References: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v1 3/6] app/test: fix freeing mbufs in distributor tests X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Sanity tests with mbuf alloc and shutdown tests assume that mbufs passed to worker cores are freed in handlers. Such packets should not be returned to the distributor's main core. The only packets that should be returned are the packets send after completion of the tests in quit_workers function. This patch fixes freeing mbufs, stops returning them to distributor's core and cleans up unused variables. Fixes: c0de0eb82e40 ("distributor: switch over to new API") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- app/test/test_distributor.c | 37 +++++++++++-------------------------- 1 file changed, 11 insertions(+), 26 deletions(-) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index 0e49e3714..da13a9a3f 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -277,24 +277,21 @@ handle_work_with_free_mbufs(void *arg) struct rte_mbuf *buf[8] __rte_cache_aligned; struct worker_params *wp = arg; struct rte_distributor *d = wp->dist; - unsigned int count = 0; unsigned int i; unsigned int num = 0; unsigned int id = __atomic_fetch_add(&worker_idx, 1, __ATOMIC_RELAXED); for (i = 0; i < 8; i++) buf[i] = NULL; - num = rte_distributor_get_pkt(d, id, buf, buf, num); + num = rte_distributor_get_pkt(d, id, buf, buf, 0); while (!quit) { - count += num; __atomic_fetch_add(&worker_stats[id].handled_packets, num, __ATOMIC_ACQ_REL); for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, - id, buf, buf, num); + id, buf, buf, 0); } - count += num; __atomic_fetch_add(&worker_stats[id].handled_packets, num, __ATOMIC_ACQ_REL); rte_distributor_return_pkt(d, id, buf, num); @@ -322,7 +319,6 @@ sanity_test_with_mbuf_alloc(struct worker_params *wp, struct rte_mempool *p) rte_distributor_process(d, NULL, 0); for (j = 0; j < BURST; j++) { bufs[j]->hash.usr = (i+j) << 1; - rte_mbuf_refcnt_set(bufs[j], 1); } rte_distributor_process(d, bufs, BURST); @@ -346,20 +342,15 @@ sanity_test_with_mbuf_alloc(struct worker_params *wp, struct rte_mempool *p) static int handle_work_for_shutdown_test(void *arg) { - struct rte_mbuf *pkt = NULL; struct rte_mbuf *buf[8] __rte_cache_aligned; struct worker_params *wp = arg; struct rte_distributor *d = wp->dist; - unsigned int count = 0; unsigned int num = 0; - unsigned int total = 0; unsigned int i; - unsigned int returned = 0; unsigned int zero_id = 0; const unsigned int id = __atomic_fetch_add(&worker_idx, 1, __ATOMIC_RELAXED); - - num = rte_distributor_get_pkt(d, id, buf, buf, num); + num = rte_distributor_get_pkt(d, id, buf, buf, 0); zero_id = __atomic_load_n(&zero_idx, __ATOMIC_ACQUIRE); if (id == zero_id && num > 0) { @@ -371,13 +362,12 @@ handle_work_for_shutdown_test(void *arg) /* wait for quit single globally, or for worker zero, wait * for zero_quit */ while (!quit && !(id == zero_id && zero_quit)) { - count += num; __atomic_fetch_add(&worker_stats[id].handled_packets, num, __ATOMIC_ACQ_REL); for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, - id, buf, buf, num); + id, buf, buf, 0); zero_id = __atomic_load_n(&zero_idx, __ATOMIC_ACQUIRE); if (id == zero_id && num > 0) { @@ -385,12 +375,7 @@ handle_work_for_shutdown_test(void *arg) __ATOMIC_ACQUIRE); __atomic_store_n(&zero_idx, zero_id, __ATOMIC_RELEASE); } - - total += num; } - count += num; - returned = rte_distributor_return_pkt(d, id, buf, num); - __atomic_fetch_add(&worker_stats[id].handled_packets, num, __ATOMIC_ACQ_REL); if (id == zero_id) { @@ -400,20 +385,20 @@ handle_work_for_shutdown_test(void *arg) while (zero_quit) usleep(100); + for (i = 0; i < num; i++) + rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, - id, buf, buf, num); + id, buf, buf, 0); while (!quit) { - count += num; - rte_pktmbuf_free(pkt); - num = rte_distributor_get_pkt(d, id, buf, buf, num); __atomic_fetch_add(&worker_stats[id].handled_packets, num, __ATOMIC_ACQ_REL); + for (i = 0; i < num; i++) + rte_pktmbuf_free(buf[i]); + num = rte_distributor_get_pkt(d, id, buf, buf, 0); } - returned = rte_distributor_return_pkt(d, - id, buf, num); - printf("Num returned = %d\n", returned); } + rte_distributor_return_pkt(d, id, buf, num); return 0; } From patchwork Tue Sep 15 19:34:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 77814 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 302A1A04C7; Tue, 15 Sep 2020 21:35:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B31251C191; Tue, 15 Sep 2020 21:35:07 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 871871C119 for ; Tue, 15 Sep 2020 21:35:00 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20200915193500euoutp02926bf14f4df219c46981b1c1faa37e60~1DA7O1N9V2234022340euoutp02T for ; Tue, 15 Sep 2020 19:35:00 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20200915193500euoutp02926bf14f4df219c46981b1c1faa37e60~1DA7O1N9V2234022340euoutp02T DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1600198500; bh=uh3mh0YltAfFhD986e+iRP3syUOWOIh9uNZziEV6b5M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GAQ3IHEjCw2qSXLD/0sO2Pyuqwe/XyE76ogy9FZF33opDN1WiVl6hFcSSgZMh2oOd sHJitrB4iiyLFrdLlUpSwbOzJ+Yh1C3jjZr06pxswtanDffa24DCY4p8YkjDR0hWe9 4Iqsf4P8TYrgJNm24dWk4Nuplzm2T5kKs7GBhBXc= Received: from eusmges1new.samsung.com (unknown [203.254.199.242]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20200915193459eucas1p2e61cb1ffd83a145d6c0480e5ee9b11dd~1DA66QE530523505235eucas1p2o; Tue, 15 Sep 2020 19:34:59 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges1new.samsung.com (EUCPMTA) with SMTP id 8F.E3.06456.367116F5; Tue, 15 Sep 2020 20:34:59 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20200915193459eucas1p19f5d1cbea87d7dc3bbd2638cdb96a31b~1DA6fsrdp1351113511eucas1p1h; Tue, 15 Sep 2020 19:34:59 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20200915193459eusmtrp173f28568ee7ea0a20f0bdf36219ff412~1DA6fNKSP1473114731eusmtrp1A; Tue, 15 Sep 2020 19:34:59 +0000 (GMT) X-AuditID: cbfec7f2-809ff70000001938-01-5f611763c168 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id C3.7C.06314.367116F5; Tue, 15 Sep 2020 20:34:59 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20200915193458eusmtip29b508d080a4703d388778ef4d7168b08~1DA51OIKa1712017120eusmtip2f; Tue, 15 Sep 2020 19:34:58 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Tue, 15 Sep 2020 21:34:47 +0200 Message-Id: <20200915193449.13310-5-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrKIsWRmVeSWpSXmKPExsWy7djP87rJ4onxBhvnmlncWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHMUl01Kak5mWWqRvl0CV8bFNrGC ZZwVnV+vMjYwvmLvYuTkkBAwkTi7fgVjFyMXh5DACkaJQ3NbmEASQgJfGCX+r7CHSHxmlFjz /BIrTMfxo2eYIBLLGSXW9Wxgh+j4xChx9GQAiM0mYCtxZOZXsAYRgTCJ5ua9LF2MHBzMAs4S T76ygYSFBbwk3n7cAtbKIqAqsfLVRWYQm1fAVeL6knNQ18lLrN5wACzOKeAmcfXjbGaQvRIC Z9gknnw/ywxR5CJx7WQj1HHCEq+Ob4FqlpE4PbmHBaJhG6PE1d8/GSGc/YwS13tXQFVZSxz+ 95sN4jpNifW79CHCjhI3Lv1hBwlLCPBJ3HgrCBJmBjInbZvODBHmlehoE4Ko1pN42jOVEWbt n7VPWCBsD4nvR6+zQ8JqJqPEv3nnWScwys9CWLaAkXEVo3hqaXFuemqxYV5quV5xYm5xaV66 XnJ+7iZGYOSf/nf80w7Gr5eSDjEKcDAq8fAuuJ0QL8SaWFZcmXuIUYKDWUmE1+ns6Tgh3pTE yqrUovz4otKc1OJDjNIcLErivMaLXsYKCaQnlqRmp6YWpBbBZJk4OKUaGL2ltFv77p18W2Y0 71TJM+9svtXMYlkr0/oqfxyV2plTncVcwTrf1a7uXaPfQx6zPW+/BXOvf3DldXD+qWtstZM/ 6ClzXv6sddImenX4jPYn9hzsU3izv1y02bP6g+ukv7J7vpjLeknMsuq6Uj6/eYLe+Tfpwvq/ uiodH+lHhQau05jIYpM4XYmlOCPRUIu5qDgRACi42074AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrGLMWRmVeSWpSXmKPExsVy+t/xe7rJ4onxBqc75S1urLK36Jv0kcni 3aftTBbPetYxWvzr+MPuwOrxa8FSVo/Fe14yeRx8t4cpgDlKz6Yov7QkVSEjv7jEVina0MJI z9DSQs/IxFLP0Ng81srIVEnfziYlNSezLLVI3y5BL+Nim1jBMs6Kzq9XGRsYX7F3MXJySAiY SBw/eoapi5GLQ0hgKaPE4XmrWLsYOYASMhIfLglA1AhL/LnWxQZR84FR4lr3W7BmNgFbiSMz v4LViwiESZxY6Q8SZhZwl9iyeCoziC0s4CXx9uMWsHIWAVWJla8ugsV5BVwlri85B3WDvMTq DQfA4pwCbhJXP84Gs4WAahafPM8+gZFvASPDKkaR1NLi3PTcYkO94sTc4tK8dL3k/NxNjMAw 3Hbs5+YdjJc2Bh9iFOBgVOLhXXA7IV6INbGsuDL3EKMEB7OSCK/T2dNxQrwpiZVVqUX58UWl OanFhxhNgY6ayCwlmpwPjJG8knhDU0NzC0tDc2NzYzMLJXHeDoGDMUIC6YklqdmpqQWpRTB9 TBycUg2MWa88tt/tD/ENrGwK28MjwOgpHb+IJdPtwHW99Qyrrz46L1K4837w3/sZPSUJTnPn SGVfe31L4W2med9+1VaxyY2yd9e1+i3KkBI49cK/6lng37kr/302Xhf+QeXpPrE+Pp+IfBt9 lRmaqivrpK/eE3t07OMii8lbj104ftVLU4GlKTyJNWC9EktxRqKhFnNRcSIAeg262VkCAAA= X-CMS-MailID: 20200915193459eucas1p19f5d1cbea87d7dc3bbd2638cdb96a31b X-Msg-Generator: CA X-RootMTR: 20200915193459eucas1p19f5d1cbea87d7dc3bbd2638cdb96a31b X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20200915193459eucas1p19f5d1cbea87d7dc3bbd2638cdb96a31b References: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v1 4/6] app/test: collect return mbufs in distributor test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" During quit_workers function distributor's main core processes some packets to wake up pending worker cores so they can quit. As quit_workers acts also as a cleanup procedure for next test case it should also collect these packages returned by workers' handlers, so the cyclic buffer with returned packets in distributor remains empty. Fixes: c3eabff124e6 ("distributor: add unit tests") Cc: bruce.richardson@intel.com Fixes: c0de0eb82e40 ("distributor: switch over to new API") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- app/test/test_distributor.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index da13a9a3f..13c6397cc 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -599,6 +599,10 @@ quit_workers(struct worker_params *wp, struct rte_mempool *p) rte_distributor_process(d, NULL, 0); rte_distributor_flush(d); rte_eal_mp_wait_lcore(); + + while (rte_distributor_returned_pkts(d, bufs, RTE_MAX_LCORE)) + ; + quit = 0; worker_idx = 0; zero_idx = 0; From patchwork Tue Sep 15 19:34:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 77815 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4D74FA04C7; Tue, 15 Sep 2020 21:35:47 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 363F71C19C; Tue, 15 Sep 2020 21:35:09 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 253531C10D for ; Tue, 15 Sep 2020 21:35:01 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20200915193500euoutp027eb065a3bd62d7551cd03302bbb4303e~1DA7zjGrj1988219882euoutp02f for ; Tue, 15 Sep 2020 19:35:00 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20200915193500euoutp027eb065a3bd62d7551cd03302bbb4303e~1DA7zjGrj1988219882euoutp02f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1600198500; bh=tCOSBquxDc/4BM7ZzzVEA+FCBvHG6w6JIej8fSYIkFE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BmXWz5NgftDlGNkJDNuDZbcVcDPfb4IM8go7LaxI2lBbdG3JM1q8Rk3JxttKkyusa Nt5ZfcrOC4U9GRbuEPTxJS4Ks5EX1s10DojloEy/y4j2VRaFFdniqiiemAwzpigij3 jpbaLXg+GiZmnQR1/GNbXkOuryMETri4A03TB5dk= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20200915193500eucas1p27ae83690fe034915a6a4eb7d67050f3f~1DA7ev_W90572705727eucas1p2k; Tue, 15 Sep 2020 19:35:00 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id 8F.C6.06318.467116F5; Tue, 15 Sep 2020 20:35:00 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20200915193500eucas1p2b079e1dcfd2d54e01a5630609b82b370~1DA7PXrq-0363603636eucas1p2w; Tue, 15 Sep 2020 19:35:00 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20200915193500eusmtrp151490540a2d63c0f40f7796d4df25235~1DA7O5md81473114731eusmtrp1C; Tue, 15 Sep 2020 19:35:00 +0000 (GMT) X-AuditID: cbfec7f5-38bff700000018ae-2b-5f61176422ff Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 24.7C.06314.467116F5; Tue, 15 Sep 2020 20:35:00 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20200915193459eusmtip2ff480d370d7dc265cbc604bab15fcdd5~1DA6pAlYD1389713897eusmtip2O; Tue, 15 Sep 2020 19:34:59 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Tue, 15 Sep 2020 21:34:48 +0200 Message-Id: <20200915193449.13310-6-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrKIsWRmVeSWpSXmKPExsWy7djPc7op4onxBtN/KFncWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHMUl01Kak5mWWqRvl0CV8bxqxfY CxbxVtzadZi1gbGTu4uRk0NCwETi9YbLzF2MXBxCAisYJQ7dm8cMkhAS+MIo8auRDyLxmVFi /4RvLDAd/y/dYINILGeU2PCgmQWi4xOjxOtvEiA2m4CtxJGZX1lBbBGBMInm5r1ANRwczALO Ek++soGEhQW8JLYcnMAIEmYRUJU432EFEuYVcJX4fnsNM8QqeYnVGw6A2ZwCbhJXP84GO1RC 4DKbxKuPs6HucZH4fLMTqkFY4tXxLewQtozE6ck9LBAN2xglrv7+yQjh7GeUuN67AqrKWuLw v99sEMdpSqzfpQ8RdpSY9H0eO0hYQoBP4sZbQZAwM5A5adt0Zogwr0RHmxBEtZ7E056pjDBr /6x9AnWah8TFE5MZIUE1k1FiwfGlTBMY5WchLFvAyLiKUTy1tDg3PbXYOC+1XK84Mbe4NC9d Lzk/dxMjMPJP/zv+dQfjvj9JhxgFOBiVeHgX3E6IF2JNLCuuzD3EKMHBrCTC63T2dJwQb0pi ZVVqUX58UWlOavEhRmkOFiVxXuNFL2OFBNITS1KzU1MLUotgskwcnFINjH0FWx7ftJ7BarHU dbuzZqHGAob6BOW5xiHl+SLJDhlqbVbTbF7fUqu99FSweQVn58zAc3u9f4s23/YI/CL5+eK+ SW8XK58/Ya1V9MTnLzd3wdb+F5zVv1lWiHzJkVoTZc+93NV83xrj2a9q7k87IOj+S6XM7fmS uSucHp/vCVPrWHulZMPqg0osxRmJhlrMRcWJAOoEZV74AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrGLMWRmVeSWpSXmKPExsVy+t/xe7op4onxBquaBC1urLK36Jv0kcni 3aftTBbPetYxWvzr+MPuwOrxa8FSVo/Fe14yeRx8t4cpgDlKz6Yov7QkVSEjv7jEVina0MJI z9DSQs/IxFLP0Ng81srIVEnfziYlNSezLLVI3y5BL+P41QvsBYt4K27tOszawNjJ3cXIySEh YCLx/9INti5GLg4hgaWMEvvPNLB3MXIAJWQkPlwSgKgRlvhzrQuq5gOjxJuj09hAEmwCthJH Zn5lBakXEQiTOLHSHyTMLOAusWXxVGYQW1jAS2LLwQmMICUsAqoS5zusQMK8Aq4S32+vYYYY Ly+xesMBMJtTwE3i6sfZYLYQUM3ik+fZJzDyLWBkWMUoklpanJueW2yoV5yYW1yal66XnJ+7 iREYhtuO/dy8g/HSxuBDjAIcjEo8vAtuJ8QLsSaWFVfmHmKU4GBWEuF1Ons6Tog3JbGyKrUo P76oNCe1+BCjKdBNE5mlRJPzgTGSVxJvaGpobmFpaG5sbmxmoSTO2yFwMEZIID2xJDU7NbUg tQimj4mDU6qBUegH85Ll5amKhZNkVvSWd8movWK5tL07dVtfSmMUq4P5maKLoS8yuV0/vGI/ HCDZ0+Or6eWvVRAe1j4nSKb876PGv1I7n50XvvzScO50pg6ZiVHxKm8Ocy99L8zv6rVA8jjT xvw78SfO3RP1fhRmHP7t7mNpwfM9E6NnFrkem9i6JMW4vtJViaU4I9FQi7moOBEAG0LV4FkC AAA= X-CMS-MailID: 20200915193500eucas1p2b079e1dcfd2d54e01a5630609b82b370 X-Msg-Generator: CA X-RootMTR: 20200915193500eucas1p2b079e1dcfd2d54e01a5630609b82b370 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20200915193500eucas1p2b079e1dcfd2d54e01a5630609b82b370 References: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v1 5/6] distributor: fix missing handshake synchronization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" rte_distributor_return_pkt function which is run on worker cores must wait for distributor core to clear handshake on retptr64 before using those buffers. While the handshake is set distributor core controls buffers and any operations on worker side might overwrite buffers which are unread yet. Fixes: 775003ad2f96 ("distributor: add new burst-capable library") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- lib/librte_distributor/rte_distributor.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 1c047f065..89493c331 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -160,6 +160,7 @@ rte_distributor_return_pkt(struct rte_distributor *d, { struct rte_distributor_buffer *buf = &d->bufs[worker_id]; unsigned int i; + volatile int64_t *retptr64; if (unlikely(d->alg_type == RTE_DIST_ALG_SINGLE)) { if (num == 1) @@ -169,6 +170,19 @@ rte_distributor_return_pkt(struct rte_distributor *d, return -EINVAL; } + retptr64 = &(buf->retptr64[0]); + /* Spin while handshake bits are set (scheduler clears it). + * Sync with worker on GET_BUF flag. + */ + while (unlikely(__atomic_load_n(retptr64, __ATOMIC_ACQUIRE) + & RTE_DISTRIB_GET_BUF)) { + rte_pause(); + uint64_t t = rte_rdtsc()+100; + + while (rte_rdtsc() < t) + rte_pause(); + } + /* Sync with distributor to acquire retptrs */ __atomic_thread_fence(__ATOMIC_ACQUIRE); for (i = 0; i < RTE_DIST_BURST_SIZE; i++) From patchwork Tue Sep 15 19:34:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 77816 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A8D77A04C7; Tue, 15 Sep 2020 21:35:55 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 26AC11C1A2; Tue, 15 Sep 2020 21:35:10 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 2D8281C120 for ; Tue, 15 Sep 2020 21:35:02 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20200915193501euoutp022e4f6cd9bc898830fb4324bc782355f0~1DA8xyhAE1988219882euoutp02g for ; Tue, 15 Sep 2020 19:35:01 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20200915193501euoutp022e4f6cd9bc898830fb4324bc782355f0~1DA8xyhAE1988219882euoutp02g DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1600198502; bh=F/QcS1TsD+CKMSw/ff2Fe2SfzJqQzA9Zm090joEQmbE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VmGOAit+/7tj9cZfKaRr1ZLzM2zHodTZAbDQ1UoiKHfhXwr58XovYHcMmDPYUAgvR YLX1QHGx4/yEtAPgq5KztfdvOIxTrYLzNOO3FRbvUt+HWouBCBC9PVZ+blYA/vDZop ScOBnoBUrAx+ZWy2V6fJXak2Q1BHwA7xRC6KQq7M= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20200915193501eucas1p13038defc40f3004b76f6168bf0d96b61~1DA8bqpJ_0245102451eucas1p1h; Tue, 15 Sep 2020 19:35:01 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 8F.60.05997.567116F5; Tue, 15 Sep 2020 20:35:01 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20200915193501eucas1p2333f0b08077c06ba04b89ce192072f9a~1DA776dUy0565705657eucas1p2n; Tue, 15 Sep 2020 19:35:01 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20200915193501eusmtrp1fd87506751c2a1e0dabe666f5b8c6b65~1DA77V-h61473114731eusmtrp1F; Tue, 15 Sep 2020 19:35:01 +0000 (GMT) X-AuditID: cbfec7f4-677ff7000000176d-12-5f6117659f51 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 7A.6C.06017.567116F5; Tue, 15 Sep 2020 20:35:01 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20200915193500eusmtip29af20133654451b2b9841b0ac5b4938d~1DA7U1z042437224372eusmtip2y; Tue, 15 Sep 2020 19:35:00 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Tue, 15 Sep 2020 21:34:49 +0200 Message-Id: <20200915193449.13310-7-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGIsWRmVeSWpSXmKPExsWy7djP87qp4onxBi/uKFvcWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHMUl01Kak5mWWqRvl0CV8acjx3s BZMEK349vs7awDiVr4uRk0NCwERi1dR7TF2MXBxCAisYJR58/sIG4XxhlJgzey4TSJWQwGdG iVP7JGE6mt5vYoYoWs4ocfvGIUYI5xOjRMOnJ2AdbAK2EkdmfmUFsUUEwiSam/eydDFycDAL OEs8+coGEhYWsJaYuLYDrJxFQFXi9csmZpASXgFXiTXvoa6Tl1i94QAziM0p4CZx9eNssL0S ApfZJB4fOssKUeQise7wY0YIW1ji1fEt7BC2jMTpyT0sEA3bGCWu/v7JCOHsZ5S43rsCqspa 4vC/32wQx2lKrN+lDxF2lJi87hdYWEKAT+LGW0GQMDOQOWnbdGaIMK9ER5sQRLWexNOeqYww a/+sfcICYXtIvJrazgIJnpmMEkvuHGWfwCg/C2HZAkbGVYziqaXFuempxUZ5qeV6xYm5xaV5 6XrJ+bmbGIGxf/rf8S87GHf9STrEKMDBqMTDu+B2QrwQa2JZcWXuIUYJDmYlEV6ns6fjhHhT EiurUovy44tKc1KLDzFKc7AoifMaL3oZKySQnliSmp2aWpBaBJNl4uCUamC03ia323zJese5 pzhXzHb6MlnSfc2KDKtF19r/esZ+sdN7ZVX089+1XVUCFqVzlmmWl4ctDlgac21bfMb9Zyur ngotDy5aH/rjCaP9QpWt6/QZgn+IJPlnO/Aq7fYOOnnvDccyD2s3d72ktaLvFaWv7D+wNDUt 439FB3OlUuzFJfkOM4T1n5cosRRnJBpqMRcVJwIAXAEgyfkCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrGLMWRmVeSWpSXmKPExsVy+t/xe7qp4onxBhtnsljcWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHOUnk1RfmlJqkJGfnGJrVK0oYWR nqGlhZ6RiaWeobF5rJWRqZK+nU1Kak5mWWqRvl2CXsacjx3sBZMEK349vs7awDiVr4uRk0NC wESi6f0m5i5GLg4hgaWMEvfbdrJ2MXIAJWQkPlwSgKgRlvhzrYsNouYDo0TP/W42kASbgK3E kZlfwepFBMIkTqz0BwkzC7hLbFk8lRnEFhawlpi4toMJxGYRUJV4/bKJGaScV8BVYs17qBPk JVZvOABWzingJnH142wwWwioZPHJ8+wTGPkWMDKsYhRJLS3OTc8tNtIrTswtLs1L10vOz93E CAzDbcd+btnB2PUu+BCjAAejEg/vgtsJ8UKsiWXFlbmHGCU4mJVEeJ3Ono4T4k1JrKxKLcqP LyrNSS0+xGgKdNNEZinR5HxgjOSVxBuaGppbWBqaG5sbm1koifN2CByMERJITyxJzU5NLUgt gulj4uCUamCcoH9d49MbR+cQh2n3JFZOvWmQ7Lly9dnHxaLNwvs4rlgXXLZLK4uWF/23XNHj lF6X16qgnumTtYSYjW90Nd5c+nBr3OSC/5vYjlZf/xHZ1/T/Zy7bm/ILgkbN2hWFx9iW5Z19 eDpiht9Jw4NJ9//s/vBX+anX6cTwrt1/sv/vSiyvOTXnIz+TEktxRqKhFnNRcSIAPirjClkC AAA= X-CMS-MailID: 20200915193501eucas1p2333f0b08077c06ba04b89ce192072f9a X-Msg-Generator: CA X-RootMTR: 20200915193501eucas1p2333f0b08077c06ba04b89ce192072f9a X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20200915193501eucas1p2333f0b08077c06ba04b89ce192072f9a References: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v1 6/6] distributor: fix handshake deadlock X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Synchronization of data exchange between distributor and worker cores is based on 2 handshakes: retptr64 for returning mbufs from workers to distributor and bufptr64 for passing mbufs to workers. Without proper order of verifying those 2 handshakes a deadlock may occur. This can happen when worker core want to return back mbufs and waits for retptr handshake to be cleared and distributor core wait for bufptr to send mbufs to worker. This can happen as worker core first returns mbufs to distributor and later gets new mbufs, while distributor first release mbufs to worker and later handle returning packets. This patch fixes possibility of the deadlock by always taking care of returning packets first on the distributor side and handling packets while waiting to release new. Fixes: 775003ad2f96 ("distributor: add new burst-capable library") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- lib/librte_distributor/rte_distributor.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 89493c331..12b3db33c 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -321,12 +321,14 @@ release(struct rte_distributor *d, unsigned int wkr) struct rte_distributor_buffer *buf = &(d->bufs[wkr]); unsigned int i; + handle_returns(d, wkr); + /* Sync with worker on GET_BUF flag */ while (!(__atomic_load_n(&(d->bufs[wkr].bufptr64[0]), __ATOMIC_ACQUIRE) - & RTE_DISTRIB_GET_BUF)) + & RTE_DISTRIB_GET_BUF)) { + handle_returns(d, wkr); rte_pause(); - - handle_returns(d, wkr); + } buf->count = 0; @@ -376,6 +378,7 @@ rte_distributor_process(struct rte_distributor *d, /* Flush out all non-full cache-lines to workers. */ for (wid = 0 ; wid < d->num_workers; wid++) { /* Sync with worker on GET_BUF flag. */ + handle_returns(d, wid); if (__atomic_load_n(&(d->bufs[wid].bufptr64[0]), __ATOMIC_ACQUIRE) & RTE_DISTRIB_GET_BUF) { release(d, wid);