From patchwork Fri Oct 9 22:01:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80230 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E0A33A04BC; Sat, 10 Oct 2020 00:02:50 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BF2FE1D5EA; Sat, 10 Oct 2020 00:02:49 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id C457C1D5C8 for ; Sat, 10 Oct 2020 00:02:46 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220236euoutp02aae7367abbeeff6d6350ff5af1817247~8cgpyOhXK2454124541euoutp02C for ; Fri, 9 Oct 2020 22:02:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220236euoutp02aae7367abbeeff6d6350ff5af1817247~8cgpyOhXK2454124541euoutp02C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280956; bh=yH3M1Cb4L5UauMPb4vqNmQJysLFgOKEd5bHi5VjtoSc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ox8R0LYFjXuoyuV7eHZxDbL8d3/nRzOyFqTVnlRgFaYbAQ2WMWofDDJjs82ksx9t4 ShDyc3xvLRl1VLgP0bFmTpNDBHo032pkiuyKm8JDqkJ51tL0kbR4Qb4PNG+I4n7Kdq h1sFTXzTGSAZDf3sS+2qn1Pla8a9ROqN5gONSXFc= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20201009220231eucas1p228d0dc392f16bee5cd170cfa5ac90400~8cgk_ZZX41995819958eucas1p2S; Fri, 9 Oct 2020 22:02:31 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id 7D.E2.06318.7FDD08F5; Fri, 9 Oct 2020 23:02:31 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20201009220229eucas1p17ad627f31005ed506c5422b93ad6d112~8cgjlWdP10450404504eucas1p1y; Fri, 9 Oct 2020 22:02:29 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220229eusmtrp29284b30d5953b910aa235f33ff39f20a~8cgjk02Gq1642116421eusmtrp2W; Fri, 9 Oct 2020 22:02:29 +0000 (GMT) X-AuditID: cbfec7f5-38bff700000018ae-95-5f80ddf7b097 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 02.5F.06017.5FDD08F5; Fri, 9 Oct 2020 23:02:29 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220210eusmtip1850c0d5c6eb400f2bda759aed12f0c2b~8cgR6kEFs1334813348eusmtip1r; Fri, 9 Oct 2020 22:02:07 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:48 +0200 Message-Id: <20201009220202.20834-2-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrOIsWRmVeSWpSXmKPExsWy7djPc7rf7zbEG0z8KmpxY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRXDYpqTmZZalF+nYJXBkdD84x FpwWrXjwaRNzA+N6oS5GTg4JAROJxxN2snQxcnEICaxglLi26xIThPOFUeLLkRWsEM5nRok/ T5cAZTjAWu5OD4CIL2eU+DdpFzuE84lRYk7PSnaQuWwCthJHZn5lBbFFBMIkmpv3soA0Mws4 Szz5ygYSFhbwkTi26j0jiM0ioCqx5dsbFhCbV8BVYtGKDUwQ58lLrN5wgBnE5hRwk2jdtQzs VAmB62wS31csZoMocpG4+uAjK4QtLPHq+BZ2CFtG4v/O+UwQDdsYJa7+/skI4exnlLjeuwKq ylri8L/fbBDXaUqs36UPEXaUWDv1FNTHfBI33gqChJmBzEnbpjNDhHklOtqgwagn8bRnKiPM 2j9rn7BA2B4SresmQAPxKqPE+iPHmScwys9CWLaAkXEVo3hqaXFuemqxcV5quV5xYm5xaV66 XnJ+7iZGYPSf/nf86w7GfX+SDjEKcDAq8fA2JDfEC7EmlhVX5h5ilOBgVhLhdTp7Ok6INyWx siq1KD++qDQntfgQozQHi5I4r/Gil7FCAumJJanZqakFqUUwWSYOTqkGRqHFHS49TmfWtH80 70nriE9MjszLN9pVY9y93X5ysPKBefqWaXm5Ses/vWSS3++/z/34BM7Ow10HnUvzZFbaCmoY lnGvmX67zic1/+aDSR8O51eySNr80r3syRAkW/PwUPofsY//Zt/Vf+4vUeFpFxTwxW0Kl6Xb vuerDvySvbg0b0H/j3ITJZbijERDLeai4kQAx8kOfvoCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrOLMWRmVeSWpSXmKPExsVy+t/xu7pf7zbEG7T+MLO4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoPZui/NKSVIWM/OISW6VoQwsj PUNLCz0jE0s9Q2PzWCsjUyV9O5uU1JzMstQifbsEvYyOB+cYC06LVjz4tIm5gXG9UBcjB4eE gInE3ekBXYxcHEICSxklHl/7wAYRl5H4cEmgi5ETyBSW+HOtiw2i5gOjxPO3fUwgCTYBW4kj M7+ygtSLCIRJnFjpDxJmFnCX2LJ4KjOILSzgI3Fs1XtGEJtFQFViy7c3LCA2r4CrxKIVG5gg 5stLrN5wAKyeU8BNonXXMhaIXY2MElsOvGSewMi3gJFhFaNIamlxbnpusZFecWJucWleul5y fu4mRmAgbjv2c8sOxq53wYcYBTgYlXh4NRIb4oVYE8uKK3MPMUpwMCuJ8DqdPR0nxJuSWFmV WpQfX1Sak1p8iNEU6KqJzFKiyfnAKMkriTc0NTS3sDQ0NzY3NrNQEuftEDgYIySQnliSmp2a WpBaBNPHxMEp1cBoeZRtafQOb8NJ6Zn7/nIJLipkni4VFmMh+Mf+T5fAZLu+y42JP7+uWM96 6se+B9ML1vec3hCiolZy7WtP8fapyas61gtM4k92YPzwTv85t33YCsnoQ2aJdTofFj1d1Vi7 9tjSnNVJxkXLw0uilNtn3PxoIBHP9iF+cpYnY5/b3ubYXZMnK5xWYinOSDTUYi4qTgQAJSAw 01oCAAA= X-CMS-MailID: 20201009220229eucas1p17ad627f31005ed506c5422b93ad6d112 X-Msg-Generator: CA X-RootMTR: 20201009220229eucas1p17ad627f31005ed506c5422b93ad6d112 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220229eucas1p17ad627f31005ed506c5422b93ad6d112 References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 01/15] distributor: fix missing handshake synchronization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" rte_distributor_return_pkt function which is run on worker cores must wait for distributor core to clear handshake on retptr64 before using those buffers. While the handshake is set distributor core controls buffers and any operations on worker side might overwrite buffers which are unread yet. Same situation appears in the legacy single distributor. Function rte_distributor_return_pkt_single shouldn't modify the bufptr64 until handshake on it is cleared by distributor lcore. Fixes: 775003ad2f96 ("distributor: add new burst-capable library") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- lib/librte_distributor/rte_distributor.c | 14 ++++++++++++++ lib/librte_distributor/rte_distributor_single.c | 4 ++++ 2 files changed, 18 insertions(+) diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 1c047f065..89493c331 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -160,6 +160,7 @@ rte_distributor_return_pkt(struct rte_distributor *d, { struct rte_distributor_buffer *buf = &d->bufs[worker_id]; unsigned int i; + volatile int64_t *retptr64; if (unlikely(d->alg_type == RTE_DIST_ALG_SINGLE)) { if (num == 1) @@ -169,6 +170,19 @@ rte_distributor_return_pkt(struct rte_distributor *d, return -EINVAL; } + retptr64 = &(buf->retptr64[0]); + /* Spin while handshake bits are set (scheduler clears it). + * Sync with worker on GET_BUF flag. + */ + while (unlikely(__atomic_load_n(retptr64, __ATOMIC_ACQUIRE) + & RTE_DISTRIB_GET_BUF)) { + rte_pause(); + uint64_t t = rte_rdtsc()+100; + + while (rte_rdtsc() < t) + rte_pause(); + } + /* Sync with distributor to acquire retptrs */ __atomic_thread_fence(__ATOMIC_ACQUIRE); for (i = 0; i < RTE_DIST_BURST_SIZE; i++) diff --git a/lib/librte_distributor/rte_distributor_single.c b/lib/librte_distributor/rte_distributor_single.c index abaf7730c..f4725b1d0 100644 --- a/lib/librte_distributor/rte_distributor_single.c +++ b/lib/librte_distributor/rte_distributor_single.c @@ -74,6 +74,10 @@ rte_distributor_return_pkt_single(struct rte_distributor_single *d, union rte_distributor_buffer_single *buf = &d->bufs[worker_id]; uint64_t req = (((int64_t)(uintptr_t)oldpkt) << RTE_DISTRIB_FLAG_BITS) | RTE_DISTRIB_RETURN_BUF; + while (unlikely(__atomic_load_n(&buf->bufptr64, __ATOMIC_RELAXED) + & RTE_DISTRIB_FLAGS_MASK)) + rte_pause(); + /* Sync with distributor on RETURN_BUF flag. */ __atomic_store_n(&(buf->bufptr64), req, __ATOMIC_RELEASE); return 0; From patchwork Fri Oct 9 22:01:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80231 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3EEDBA04BC; Sat, 10 Oct 2020 00:03:13 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D413F1D5FC; Sat, 10 Oct 2020 00:02:51 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 5CBC81D5C8 for ; Sat, 10 Oct 2020 00:02:48 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220237euoutp02e8aa7bb4fb6aa372cfc004f0e232dd70~8cgqS4tli1685816858euoutp02m for ; Fri, 9 Oct 2020 22:02:37 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220237euoutp02e8aa7bb4fb6aa372cfc004f0e232dd70~8cgqS4tli1685816858euoutp02m DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280957; bh=vuwpOk5DYAg8nrtQAkN6Jx3ymyd0V4eE98pnCakaiTY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PWhhMG/J3EZSl1/5H8bhgh9ppxiXxn9Ug+fqOh6bI9gSZ7VygKEtPWsOIMZKNMN+v q1YwX4TJqtPgCMLnlEofG5LunVFrgi/isqYRAr3qKTwOQnKmAyDKWw6TfBVlxjvvgD xNAf4Me5lTuDmtzI9uyF6jJa/q0UCKW2Gb8LPeiU= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20201009220231eucas1p15ab66d078308bacac156bc2b8f0f3c90~8cglFagMQ1023910239eucas1p1b; Fri, 9 Oct 2020 22:02:31 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 33.88.05997.7FDD08F5; Fri, 9 Oct 2020 23:02:31 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20201009220231eucas1p217c48d880aaa7f15e4351f92eede01b6~8cgktOpNq1926719267eucas1p2h; Fri, 9 Oct 2020 22:02:31 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220231eusmtrp2bfc2d7a6b92eff2947f2c304e0352c9c~8cgkspy0B1642116421eusmtrp2X; Fri, 9 Oct 2020 22:02:31 +0000 (GMT) X-AuditID: cbfec7f4-677ff7000000176d-f1-5f80ddf746c9 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 72.E2.06314.7FDD08F5; Fri, 9 Oct 2020 23:02:31 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220230eusmtip1c8faceecd183e000f66feff7d0c691d0~8cgjrYEHl1990219902eusmtip1E; Fri, 9 Oct 2020 22:02:29 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:49 +0200 Message-Id: <20201009220202.20834-3-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrOIsWRmVeSWpSXmKPExsWy7djP87rf7zbEG+zoUbS4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoLpuU1JzMstQifbsEroy9U+Yy FhwRrLj88ihjA+Mhvi5GTg4JAROJ2Y37WboYuTiEBFYwSkxYeJ8ZwvnCKPHg6HdWCOczo8Tt CfeYYFqmr34L1bKcUeLVjB1QVZ8YJbouTGQDqWITsJU4MvMrK4gtIhAm0dy8F6iDg4NZwFni yVewEmGgkpMXZrGA2CwCqhITe/4ygti8Aq4SLX9PskIsk5dYveEAM4jNKeAm0bprGdhiCYHr bBLbXixnhChykfj99QI7hC0s8er4FihbRuL/zvlMEA3bGCWu/v7JCOHsZ5S43rsCqspa4vC/ 32wQ12lKrN+lD2JKCDhK3O+QgDD5JG68FQQpZgYyJ22bzgwR5pXoaBOCmKEn8bRnKiPM1j9r n7BA2B4SW85uYoIEz1VGiY1dl1kmMMrPQti1gJFxFaN4amlxbnpqsVFearlecWJucWleul5y fu4mRmD0n/53/MsOxl1/kg4xCnAwKvHwNiQ3xAuxJpYVV+YeYpTgYFYS4XU6ezpOiDclsbIq tSg/vqg0J7X4EKM0B4uSOK/xopexQgLpiSWp2ampBalFMFkmDk6pBsaoJYeDD8sLi1RlHk9g XcQYeC5L5WSIsNKUdTqJEabqHwXWvW80PO6yfPOF+RcEpnZN3uf2dom4493aYwYPezNfiHku n3V3jWCMcc2EerdlWQnFEy/tD2RXeae8TfHKLe4/Ae0JOffnpGX53Sxw9jh5ycDLcWrLz6Lr 3fdapzNd2/1u2vIX5SxKLMUZiYZazEXFiQA0+ww2+gIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrBLMWRmVeSWpSXmKPExsVy+t/xu7rf7zbEG+w8wWpxY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRejZF+aUlqQoZ+cUltkrRhhZG eoaWFnpGJpZ6hsbmsVZGpkr6djYpqTmZZalF+nYJehl7p8xlLDgiWHH55VHGBsZDfF2MnBwS AiYS01e/Zeli5OIQEljKKPHt8AL2LkYOoISMxIdLAhA1whJ/rnWxQdR8YJTo23+dHSTBJmAr cWTmV1aQehGBMIkTK/1BwswC7hJbFk9lBrGFgUpOXpjFAmKzCKhKTOz5ywhi8wq4SrT8PckK MV9eYvWGA2D1nAJuEq27lkHd08goseXAS+YJjHwLGBlWMYqklhbnpucWG+oVJ+YWl+al6yXn 525iBIbitmM/N+9gvLQx+BCjAAejEg+vRmJDvBBrYllxZe4hRgkOZiURXqezp+OEeFMSK6tS i/Lji0pzUosPMZoCXTWRWUo0OR8YJ3kl8YamhuYWlobmxubGZhZK4rwdAgdjhATSE0tSs1NT C1KLYPqYODilGhjTWda1Oc5TMm7/cCggZdLZpBrDqPXM9pdUpqZFCFtduf4ot8j5xYEdJXd+ /JqYvPfsWc57Nwzv37ol63Bm6rmFEw95dHZ5qyyYqutx+Qv3PxbBvHmhl7WrOlS0tiw18E9q XyJ1k7PIV1Rwid5OO9nG639ZGYyldm946LV4/nvxq7yHY6oWWS5TYinOSDTUYi4qTgQAPQxj rVsCAAA= X-CMS-MailID: 20201009220231eucas1p217c48d880aaa7f15e4351f92eede01b6 X-Msg-Generator: CA X-RootMTR: 20201009220231eucas1p217c48d880aaa7f15e4351f92eede01b6 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220231eucas1p217c48d880aaa7f15e4351f92eede01b6 References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 02/15] distributor: fix handshake deadlock X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Synchronization of data exchange between distributor and worker cores is based on 2 handshakes: retptr64 for returning mbufs from workers to distributor and bufptr64 for passing mbufs to workers. Without proper order of verifying those 2 handshakes a deadlock may occur. This can happen when worker core wants to return back mbufs and waits for retptr handshake to be cleared while distributor core waits for bufptr to send mbufs to worker. This can happen as worker core first returns mbufs to distributor and later gets new mbufs, while distributor first releases mbufs to worker and later handle returning packets. This patch fixes possibility of the deadlock by always taking care of returning packets first on the distributor side and handling packets while waiting to release new. Fixes: 775003ad2f96 ("distributor: add new burst-capable library") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- lib/librte_distributor/rte_distributor.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 89493c331..12b3db33c 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -321,12 +321,14 @@ release(struct rte_distributor *d, unsigned int wkr) struct rte_distributor_buffer *buf = &(d->bufs[wkr]); unsigned int i; + handle_returns(d, wkr); + /* Sync with worker on GET_BUF flag */ while (!(__atomic_load_n(&(d->bufs[wkr].bufptr64[0]), __ATOMIC_ACQUIRE) - & RTE_DISTRIB_GET_BUF)) + & RTE_DISTRIB_GET_BUF)) { + handle_returns(d, wkr); rte_pause(); - - handle_returns(d, wkr); + } buf->count = 0; @@ -376,6 +378,7 @@ rte_distributor_process(struct rte_distributor *d, /* Flush out all non-full cache-lines to workers. */ for (wid = 0 ; wid < d->num_workers; wid++) { /* Sync with worker on GET_BUF flag. */ + handle_returns(d, wid); if (__atomic_load_n(&(d->bufs[wid].bufptr64[0]), __ATOMIC_ACQUIRE) & RTE_DISTRIB_GET_BUF) { release(d, wid); From patchwork Fri Oct 9 22:01:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80232 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 18C7DA04BC; Sat, 10 Oct 2020 00:03:35 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DC3921D60E; Sat, 10 Oct 2020 00:02:54 +0200 (CEST) Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id 22C541D5F2 for ; Sat, 10 Oct 2020 00:02:50 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220238euoutp012780f4e22934dfcd6894b4754196758a~8cgrt9zVz2928229282euoutp01k for ; Fri, 9 Oct 2020 22:02:38 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20201009220238euoutp012780f4e22934dfcd6894b4754196758a~8cgrt9zVz2928229282euoutp01k DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280958; bh=NXpLZiftAddfs3hsi2CLAhUURHCIw4Q+iTczB/vGCL0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mnkvNtw9Ff0/PbnWvhUO/VY/qMEyOuUEa++EVjTao8NvN+VRDaL5rDDZucRorM6IC mZuVbseb6Xnwj8zqs6Xs//sG/j6BRyNWkpQECI5khJ/bgOlmRdlT24K/YtGF5w0qc5 06O3K0ZgYHGr7L7qaiSxqVyZd+MjZWv9+8RoXiLM= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20201009220233eucas1p117e7f4720be8459df0c9733ed1500e40~8cgmvnf7C1723217232eucas1p1W; Fri, 9 Oct 2020 22:02:33 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id DD.E2.06318.9FDD08F5; Fri, 9 Oct 2020 23:02:33 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20201009220232eucas1p201d3b81574b7ec42ff3fb18f4bbfcbea~8cglozoTP1921319213eucas1p2S; Fri, 9 Oct 2020 22:02:32 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220232eusmtrp2a7c04bf24cf2c11da0b7d36feb141452~8cgloKEQq1642116421eusmtrp2Z; Fri, 9 Oct 2020 22:02:32 +0000 (GMT) X-AuditID: cbfec7f5-371ff700000018ae-98-5f80ddf937dd Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id D2.E2.06314.8FDD08F5; Fri, 9 Oct 2020 23:02:32 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220231eusmtip1a3b4500e6408000efd622e71583ddac0~8cgk2bDSs1334813348eusmtip1s; Fri, 9 Oct 2020 22:02:31 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:50 +0200 Message-Id: <20201009220202.20834-4-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrCIsWRmVeSWpSXmKPExsWy7djP87o/7zbEG5xcJmhxY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRXDYpqTmZZalF+nYJXBk71gYX PFCp+PRhIWMD4znZLkZODgkBE4kLW06yg9hCAisYJY6fye1i5AKyvzBKzL54jhHC+cwosWJn J0sXIwdYx4LFiRDx5YwSO/5fZoJwPjFKNN5rYgUZxSZgK3Fk5lcwW0QgTKK5eS9YM7OAs8ST r2wgYWEBD4mOy5+YQcIsAqoS9xeJgIR5BVwlHtz4wgRxnLzE6g0HmEFsTgE3idZdy1hAVkkI XGaTOP15KhtEkYvEzdWT2CFsYYlXx7dA2TIS/3fOZ4Jo2MYocfX3T0YIZz+jxPXeFVBV1hKH //1mgzhOU2L9Ln2IsKPEuX332CEe5pO48VYQJMwMZE7aNp0ZIswr0dEmBFGtJ/G0ZyojzNo/ a5+wQNgeEpO+LWeHBM9VRon9zRNYJjDKz0JYtoCRcRWjeGppcW56arFxXmq5XnFibnFpXrpe cn7uJkZg3J/+d/zrDsZ9f5IOMQpwMCrx8DYkN8QLsSaWFVfmHmKU4GBWEuF1Ons6Tog3JbGy KrUoP76oNCe1+BCjNAeLkjiv8aKXsUIC6YklqdmpqQWpRTBZJg5OqQbGrMonWTvWST9hYbx7 3iBn7s+LSl2H+M0Fc4r2ztrvJMwc8vHjrE3nPptkPv3j+2rirwkX9/DNe2d7quFDg2X0Y73v qqxNd/ZcmZ9TL7xOc91HvS/dKafUbr7rC3jiZ257gvuRjc2ceWtFDyz8uvjPlYZjWiuL56hd fTrj5b5QrmXhJYUXdu39/liJpTgj0VCLuag4EQCLC1HD9wIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrKLMWRmVeSWpSXmKPExsVy+t/xu7o/7jbEGyw8zGJxY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRejZF+aUlqQoZ+cUltkrRhhZG eoaWFnpGJpZ6hsbmsVZGpkr6djYpqTmZZalF+nYJehk71gYXPFCp+PRhIWMD4znZLkYODgkB E4kFixO7GLk4hASWMkpMv/2SBSIuI/HhkkAXIyeQKSzx51oXG0TNB0aJj81T2UASbAK2Ekdm fmUFqRcRCJM4sdIfJMws4C6xZfFUZhBbWMBDouPyJ2aQEhYBVYn7i0RAwrwCrhIPbnxhghgv L7F6wwGwck4BN4nWXctYIFY1MkpsOfCSeQIj3wJGhlWMIqmlxbnpucWGesWJucWleel6yfm5 mxiBQbjt2M/NOxgvbQw+xCjAwajEw6uR2BAvxJpYVlyZe4hRgoNZSYTX6ezpOCHelMTKqtSi /Pii0pzU4kOMpkBHTWSWEk3OB0ZIXkm8oamhuYWlobmxubGZhZI4b4fAwRghgfTEktTs1NSC 1CKYPiYOTqkGRpENysGWMiJ3zHf3iE5Nfb9MiM/mTHeUzsSPa66se/20q+TEcX6zFx3+lwO3 fFvO4tBmzfY5uuE0t9fNtXP6nwYtmcgk8mJWfufvCrH7sz2/uF/hla/rqo9+XW397OpXx+AM m/gLNj15F5XmuC0VZTlTffZV9QpDQzb37/cazz3KVq42vu7TpMRSnJFoqMVcVJwIAFGOgiBY AgAA X-CMS-MailID: 20201009220232eucas1p201d3b81574b7ec42ff3fb18f4bbfcbea X-Msg-Generator: CA X-RootMTR: 20201009220232eucas1p201d3b81574b7ec42ff3fb18f4bbfcbea X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220232eucas1p201d3b81574b7ec42ff3fb18f4bbfcbea References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 03/15] distributor: do not use oldpkt when not needed X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" rte_distributor_request_pkt and rte_distributor_get_pkt dereferenced oldpkt parameter when in RTE_DIST_ALG_SINGLE even if number of returned buffers from worker to distributor was 0. This patch passes NULL to the legacy API when number of returned buffers is 0. This allows passing NULL as oldpkt parameter. Distributor tests are also updated passing NULL as oldpkt and 0 as number of returned packets, where packets are not returned. Fixes: 775003ad2f96 ("distributor: add new burst-capable library") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- app/test/test_distributor.c | 28 +++++++++--------------- lib/librte_distributor/rte_distributor.c | 4 ++-- 2 files changed, 12 insertions(+), 20 deletions(-) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index ba1f81cf8..52230d250 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -62,13 +62,10 @@ handle_work(void *arg) struct rte_mbuf *buf[8] __rte_cache_aligned; struct worker_params *wp = arg; struct rte_distributor *db = wp->dist; - unsigned int count = 0, num = 0; + unsigned int count = 0, num; unsigned int id = __atomic_fetch_add(&worker_idx, 1, __ATOMIC_RELAXED); - int i; - for (i = 0; i < 8; i++) - buf[i] = NULL; - num = rte_distributor_get_pkt(db, id, buf, buf, num); + num = rte_distributor_get_pkt(db, id, buf, NULL, 0); while (!quit) { __atomic_fetch_add(&worker_stats[id].handled_packets, num, __ATOMIC_RELAXED); @@ -272,19 +269,16 @@ handle_work_with_free_mbufs(void *arg) struct rte_distributor *d = wp->dist; unsigned int count = 0; unsigned int i; - unsigned int num = 0; + unsigned int num; unsigned int id = __atomic_fetch_add(&worker_idx, 1, __ATOMIC_RELAXED); - for (i = 0; i < 8; i++) - buf[i] = NULL; - num = rte_distributor_get_pkt(d, id, buf, buf, num); + num = rte_distributor_get_pkt(d, id, buf, NULL, 0); while (!quit) { worker_stats[id].handled_packets += num; count += num; for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); - num = rte_distributor_get_pkt(d, - id, buf, buf, num); + num = rte_distributor_get_pkt(d, id, buf, NULL, 0); } worker_stats[id].handled_packets += num; count += num; @@ -342,14 +336,14 @@ handle_work_for_shutdown_test(void *arg) struct worker_params *wp = arg; struct rte_distributor *d = wp->dist; unsigned int count = 0; - unsigned int num = 0; + unsigned int num; unsigned int total = 0; unsigned int i; unsigned int returned = 0; const unsigned int id = __atomic_fetch_add(&worker_idx, 1, __ATOMIC_RELAXED); - num = rte_distributor_get_pkt(d, id, buf, buf, num); + num = rte_distributor_get_pkt(d, id, buf, NULL, 0); /* wait for quit single globally, or for worker zero, wait * for zero_quit */ @@ -358,8 +352,7 @@ handle_work_for_shutdown_test(void *arg) count += num; for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); - num = rte_distributor_get_pkt(d, - id, buf, buf, num); + num = rte_distributor_get_pkt(d, id, buf, NULL, 0); total += num; } worker_stats[id].handled_packets += num; @@ -373,14 +366,13 @@ handle_work_for_shutdown_test(void *arg) while (zero_quit) usleep(100); - num = rte_distributor_get_pkt(d, - id, buf, buf, num); + num = rte_distributor_get_pkt(d, id, buf, NULL, 0); while (!quit) { worker_stats[id].handled_packets += num; count += num; rte_pktmbuf_free(pkt); - num = rte_distributor_get_pkt(d, id, buf, buf, num); + num = rte_distributor_get_pkt(d, id, buf, NULL, 0); } returned = rte_distributor_return_pkt(d, id, buf, num); diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 12b3db33c..b720abe03 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -42,7 +42,7 @@ rte_distributor_request_pkt(struct rte_distributor *d, if (unlikely(d->alg_type == RTE_DIST_ALG_SINGLE)) { rte_distributor_request_pkt_single(d->d_single, - worker_id, oldpkt[0]); + worker_id, count ? oldpkt[0] : NULL); return; } @@ -134,7 +134,7 @@ rte_distributor_get_pkt(struct rte_distributor *d, if (unlikely(d->alg_type == RTE_DIST_ALG_SINGLE)) { if (return_count <= 1) { pkts[0] = rte_distributor_get_pkt_single(d->d_single, - worker_id, oldpkt[0]); + worker_id, return_count ? oldpkt[0] : NULL); return (pkts[0]) ? 1 : 0; } else return -EINVAL; From patchwork Fri Oct 9 22:01:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80233 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3241CA04BC; Sat, 10 Oct 2020 00:04:01 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 47C5A1D62E; Sat, 10 Oct 2020 00:03:00 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id DF6881D5F3 for ; Sat, 10 Oct 2020 00:02:57 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220241euoutp027cc25e7eaa0ae0f68be2f916f7728615~8cguiSbHb1684516845euoutp02r for ; Fri, 9 Oct 2020 22:02:41 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220241euoutp027cc25e7eaa0ae0f68be2f916f7728615~8cguiSbHb1684516845euoutp02r DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280961; bh=AqUBlf8gPTXwJQZGL7gAA/vvwSQFfIKWCLO6QqixeJI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Fdu2N1qDHjvMgKRgEbUdUbC46BIbNRBxrG0M7Ddua6sAAY5/sh/+MYjhUkrgYC8sy u3SNjN5/o/lIecNs/NImuLTJc3Ny4hFx/eTKTl7+N/SRACwGfab/C9uStzX1Knvppt xvQRCto0ZlAbdM2ZZb6PnCUpFB0au/m/Wq/vnz5Q= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20201009220234eucas1p138b0dc40e4004b4a9ea6205f60792aca~8cgnsUJLs0993409934eucas1p1W; Fri, 9 Oct 2020 22:02:34 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 93.88.05997.AFDD08F5; Fri, 9 Oct 2020 23:02:34 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20201009220233eucas1p285b4d01402c0c8bcfd018673afeb05eb~8cgmqBlfZ1628316283eucas1p2K; Fri, 9 Oct 2020 22:02:33 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220233eusmtrp278c0a0fbe8bf8112577f0d522086f3b9~8cgmpe2iQ1642116421eusmtrp2a; Fri, 9 Oct 2020 22:02:33 +0000 (GMT) X-AuditID: cbfec7f4-677ff7000000176d-f5-5f80ddfa0734 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id D2.5F.06017.9FDD08F5; Fri, 9 Oct 2020 23:02:33 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220232eusmtip1aa0b547a59cee487d8b0e77ebdfa1362~8cglsNKW21334813348eusmtip1t; Fri, 9 Oct 2020 22:02:32 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:51 +0200 Message-Id: <20201009220202.20834-5-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGIsWRmVeSWpSXmKPExsWy7djPc7q/7jbEGxxpEra4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoLpuU1JzMstQifbsErozr69ML TuRXXP1+lK2B8WZEFyMnh4SAicTGnrXsXYxcHEICKxglbm9uZoJwvjBK9N1ZBJX5zCjx9Ncv 5i5GDrCW35MDIOLLGSW+nGlhhnA+MUoc3DmdCWQum4CtxJGZX1lBbBGBMInm5r0sIM3MAs4S T76ygYSFBbwl3i1rBCthEVCVmLZhKzOIzSvgKrFj1zF2iPPkJVZvOAAW5xRwk2jdtYwFZJeE wGU2iX8Tp7NBFLlIzF7ykhHCFpZ4dXwLVLOMxP+d85kgGrYxSlz9/ZMRwtnPKHG9dwVUlbXE 4X+/2SCu05RYv0sfIuwo0b9hARPEx3wSN94KgoSZgcxJ26ZDA4JXoqNNCKJaT+Jpz1RGmLV/ 1j5hgbA9JKYcOQwN0auMEov6VjNOYJSfhbBsASPjKkbx1NLi3PTUYqO81HK94sTc4tK8dL3k /NxNjMDYP/3v+JcdjLv+JB1iFOBgVOLhbUhuiBdiTSwrrsw9xCjBwawkwut09nScEG9KYmVV alF+fFFpTmrxIUZpDhYlcV7jRS9jhQTSE0tSs1NTC1KLYLJMHJxSDYwFn+9tUlKuFDD4ZWTI H55yKitFlqlvzbfsQt3GJhMxlf3qpjvbmHIDeBYaBgf3OO4+v+fs89NSj9e/sAl4bFKZuG21 gtLN9mUmnTelQyL499dNWR/y6vxcjsV8ZlXe79dP1j25bNf+DUePOESwC//9Iupi9Mhcp/Xh 25uhsoqHDrb9MegKmqjEUpyRaKjFXFScCABAeU1s+QIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrKLMWRmVeSWpSXmKPExsVy+t/xu7o/7zbEGyz6y2NxY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRejZF+aUlqQoZ+cUltkrRhhZG eoaWFnpGJpZ6hsbmsVZGpkr6djYpqTmZZalF+nYJehnX16cXnMivuPr9KFsD482ILkYODgkB E4nfkwO6GLk4hASWMkpsufqQFSIuI/HhkkAXIyeQKSzx51oXG4gtJPCBUeLRTSMQm03AVuLI zK9g5SICYRInVvqDhJkF3CW2LJ7KDGILC3hLvFvWyApiswioSkzbsBUszivgKrFj1zF2iPHy Eqs3HACLcwq4SbTuWsYCcU4j0DkHXjJPYORbwMiwilEktbQ4Nz232EivODG3uDQvXS85P3cT IzAItx37uWUHY9e74EOMAhyMSjy8GokN8UKsiWXFlbmHGCU4mJVEeJ3Ono4T4k1JrKxKLcqP LyrNSS0+xGgKdNVEZinR5HxghOSVxBuaGppbWBqaG5sbm1koifN2CByMERJITyxJzU5NLUgt gulj4uCUamDUS/nY9b6+v/v25Nmt++NEPPdpXH79k3/vp299eWYV+f2Hpf9IaKzM/WlhOUvK rnIO44xbrTtMzfca3GyNWynx5N2VKyfYfjlsLk3YvsAycNN+hQ3L+3P+73WweutzoHGP350n sWdC0++Htmv5cxTP/rrw15V5V3v6+3NPv0qeW1s7Q6acqWq6EktxRqKhFnNRcSIAPkzQnFgC AAA= X-CMS-MailID: 20201009220233eucas1p285b4d01402c0c8bcfd018673afeb05eb X-Msg-Generator: CA X-RootMTR: 20201009220233eucas1p285b4d01402c0c8bcfd018673afeb05eb X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220233eucas1p285b4d01402c0c8bcfd018673afeb05eb References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 04/15] distributor: handle worker shutdown in burst mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The burst version of distributor implementation was missing proper handling of worker shutdown. A worker processing packets received from distributor can call rte_distributor_return_pkt() function informing distributor that it want no more packets. Further calls to rte_distributor_request_pkt() or rte_distributor_get_pkt() however should inform distributor that new packets are requested again. Lack of the proper implementation has caused that even after worker informed about returning last packets, new packets were still sent from distributor causing deadlocks as no one could get them on worker side. This patch adds handling shutdown of the worker in following way: 1) It fixes usage of RTE_DISTRIB_VALID_BUF handshake flag. This flag was formerly unused in burst implementation and now it is used for marking valid packets in retptr64 replacing invalid use of RTE_DISTRIB_RETURN_BUF flag. 2) Uses RTE_DISTRIB_RETURN_BUF as a worker to distributor handshake in retptr64 to indicate that worker has shutdown. 3) Worker that shuts down blocks also bufptr for itself with RTE_DISTRIB_RETURN_BUF flag allowing distributor to retrieve any in flight packets. 4) When distributor receives information about shutdown of a worker, it: marks worker as not active; retrieves any in flight and backlog packets and process them to different workers; unlocks bufptr64 by clearing RTE_DISTRIB_RETURN_BUF flag and allowing use in the future if worker requests any new packages. 5) Do not allow to: send or add to backlog any packets for not active workers. Such workers are also ignored if matched. 6) Adjust calls to handle_returns() and tags matching procedure to react for possible activation deactivation of workers. Fixes: 775003ad2f96 ("distributor: add new burst-capable library") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- lib/librte_distributor/distributor_private.h | 3 + lib/librte_distributor/rte_distributor.c | 175 +++++++++++++++---- 2 files changed, 146 insertions(+), 32 deletions(-) diff --git a/lib/librte_distributor/distributor_private.h b/lib/librte_distributor/distributor_private.h index 489aef2ac..689fe3e18 100644 --- a/lib/librte_distributor/distributor_private.h +++ b/lib/librte_distributor/distributor_private.h @@ -155,6 +155,9 @@ struct rte_distributor { enum rte_distributor_match_function dist_match_fn; struct rte_distributor_single *d_single; + + uint8_t active[RTE_DISTRIB_MAX_WORKERS]; + uint8_t activesum; }; void diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index b720abe03..115443fc0 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -51,7 +51,7 @@ rte_distributor_request_pkt(struct rte_distributor *d, * Sync with worker on GET_BUF flag. */ while (unlikely(__atomic_load_n(retptr64, __ATOMIC_ACQUIRE) - & RTE_DISTRIB_GET_BUF)) { + & (RTE_DISTRIB_GET_BUF | RTE_DISTRIB_RETURN_BUF))) { rte_pause(); uint64_t t = rte_rdtsc()+100; @@ -67,11 +67,11 @@ rte_distributor_request_pkt(struct rte_distributor *d, for (i = count; i < RTE_DIST_BURST_SIZE; i++) buf->retptr64[i] = 0; - /* Set Return bit for each packet returned */ + /* Set VALID_BUF bit for each packet returned */ for (i = count; i-- > 0; ) buf->retptr64[i] = (((int64_t)(uintptr_t)(oldpkt[i])) << - RTE_DISTRIB_FLAG_BITS) | RTE_DISTRIB_RETURN_BUF; + RTE_DISTRIB_FLAG_BITS) | RTE_DISTRIB_VALID_BUF; /* * Finally, set the GET_BUF to signal to distributor that cache @@ -97,11 +97,13 @@ rte_distributor_poll_pkt(struct rte_distributor *d, return (pkts[0]) ? 1 : 0; } - /* If bit is set, return + /* If any of below bits is set, return. + * GET_BUF is set when distributor hasn't sent any packets yet + * RETURN_BUF is set when distributor must retrieve in-flight packets * Sync with distributor to acquire bufptrs */ if (__atomic_load_n(&(buf->bufptr64[0]), __ATOMIC_ACQUIRE) - & RTE_DISTRIB_GET_BUF) + & (RTE_DISTRIB_GET_BUF | RTE_DISTRIB_RETURN_BUF)) return -1; /* since bufptr64 is signed, this should be an arithmetic shift */ @@ -113,7 +115,7 @@ rte_distributor_poll_pkt(struct rte_distributor *d, } /* - * so now we've got the contents of the cacheline into an array of + * so now we've got the contents of the cacheline into an array of * mbuf pointers, so toggle the bit so scheduler can start working * on the next cacheline while we're working. * Sync with distributor on GET_BUF flag. Release bufptrs. @@ -175,7 +177,7 @@ rte_distributor_return_pkt(struct rte_distributor *d, * Sync with worker on GET_BUF flag. */ while (unlikely(__atomic_load_n(retptr64, __ATOMIC_ACQUIRE) - & RTE_DISTRIB_GET_BUF)) { + & (RTE_DISTRIB_GET_BUF | RTE_DISTRIB_RETURN_BUF))) { rte_pause(); uint64_t t = rte_rdtsc()+100; @@ -187,17 +189,25 @@ rte_distributor_return_pkt(struct rte_distributor *d, __atomic_thread_fence(__ATOMIC_ACQUIRE); for (i = 0; i < RTE_DIST_BURST_SIZE; i++) /* Switch off the return bit first */ - buf->retptr64[i] &= ~RTE_DISTRIB_RETURN_BUF; + buf->retptr64[i] = 0; for (i = num; i-- > 0; ) buf->retptr64[i] = (((int64_t)(uintptr_t)oldpkt[i]) << - RTE_DISTRIB_FLAG_BITS) | RTE_DISTRIB_RETURN_BUF; + RTE_DISTRIB_FLAG_BITS) | RTE_DISTRIB_VALID_BUF; + + /* Use RETURN_BUF on bufptr64 to notify distributor that + * we won't read any mbufs from there even if GET_BUF is set. + * This allows distributor to retrieve in-flight already sent packets. + */ + __atomic_or_fetch(&(buf->bufptr64[0]), RTE_DISTRIB_RETURN_BUF, + __ATOMIC_ACQ_REL); - /* set the GET_BUF but even if we got no returns. - * Sync with distributor on GET_BUF flag. Release retptrs. + /* set the RETURN_BUF on retptr64 even if we got no returns. + * Sync with distributor on RETURN_BUF flag. Release retptrs. + * Notify distributor that we don't request more packets any more. */ __atomic_store_n(&(buf->retptr64[0]), - buf->retptr64[0] | RTE_DISTRIB_GET_BUF, __ATOMIC_RELEASE); + buf->retptr64[0] | RTE_DISTRIB_RETURN_BUF, __ATOMIC_RELEASE); return 0; } @@ -267,6 +277,59 @@ find_match_scalar(struct rte_distributor *d, */ } +/* + * When worker called rte_distributor_return_pkt() + * and passed RTE_DISTRIB_RETURN_BUF handshake through retptr64, + * distributor must retrieve both inflight and backlog packets assigned + * to the worker and reprocess them to another worker. + */ +static void +handle_worker_shutdown(struct rte_distributor *d, unsigned int wkr) +{ + struct rte_distributor_buffer *buf = &(d->bufs[wkr]); + /* double BURST size for storing both inflights and backlog */ + struct rte_mbuf *pkts[RTE_DIST_BURST_SIZE * 2]; + unsigned int pkts_count = 0; + unsigned int i; + + /* If GET_BUF is cleared there are in-flight packets sent + * to worker which does not require new packets. + * They must be retrieved and assigned to another worker. + */ + if (!(__atomic_load_n(&(buf->bufptr64[0]), __ATOMIC_ACQUIRE) + & RTE_DISTRIB_GET_BUF)) + for (i = 0; i < RTE_DIST_BURST_SIZE; i++) + if (buf->bufptr64[i] & RTE_DISTRIB_VALID_BUF) + pkts[pkts_count++] = (void *)((uintptr_t) + (buf->bufptr64[i] + >> RTE_DISTRIB_FLAG_BITS)); + + /* Make following operations on handshake flags on bufptr64: + * - set GET_BUF to indicate that distributor can overwrite buffer + * with new packets if worker will make a new request. + * - clear RETURN_BUF to unlock reads on worker side. + */ + __atomic_store_n(&(buf->bufptr64[0]), RTE_DISTRIB_GET_BUF, + __ATOMIC_RELEASE); + + /* Collect backlog packets from worker */ + for (i = 0; i < d->backlog[wkr].count; i++) + pkts[pkts_count++] = (void *)((uintptr_t) + (d->backlog[wkr].pkts[i] >> RTE_DISTRIB_FLAG_BITS)); + + d->backlog[wkr].count = 0; + + /* Clear both inflight and backlog tags */ + for (i = 0; i < RTE_DIST_BURST_SIZE; i++) { + d->in_flight_tags[wkr][i] = 0; + d->backlog[wkr].tags[i] = 0; + } + + /* Recursive call */ + if (pkts_count > 0) + rte_distributor_process(d, pkts, pkts_count); +} + /* * When the handshake bits indicate that there are packets coming @@ -285,19 +348,33 @@ handle_returns(struct rte_distributor *d, unsigned int wkr) /* Sync on GET_BUF flag. Acquire retptrs. */ if (__atomic_load_n(&(buf->retptr64[0]), __ATOMIC_ACQUIRE) - & RTE_DISTRIB_GET_BUF) { + & (RTE_DISTRIB_GET_BUF | RTE_DISTRIB_RETURN_BUF)) { for (i = 0; i < RTE_DIST_BURST_SIZE; i++) { - if (buf->retptr64[i] & RTE_DISTRIB_RETURN_BUF) { + if (buf->retptr64[i] & RTE_DISTRIB_VALID_BUF) { oldbuf = ((uintptr_t)(buf->retptr64[i] >> RTE_DISTRIB_FLAG_BITS)); /* store returns in a circular buffer */ store_return(oldbuf, d, &ret_start, &ret_count); count++; - buf->retptr64[i] &= ~RTE_DISTRIB_RETURN_BUF; + buf->retptr64[i] &= ~RTE_DISTRIB_VALID_BUF; } } d->returns.start = ret_start; d->returns.count = ret_count; + + /* If worker requested packets with GET_BUF, set it to active + * otherwise (RETURN_BUF), set it to not active. + */ + d->activesum -= d->active[wkr]; + d->active[wkr] = !!(buf->retptr64[0] & RTE_DISTRIB_GET_BUF); + d->activesum += d->active[wkr]; + + /* If worker returned packets without requesting new ones, + * handle all in-flights and backlog packets assigned to it. + */ + if (unlikely(buf->retptr64[0] & RTE_DISTRIB_RETURN_BUF)) + handle_worker_shutdown(d, wkr); + /* Clear for the worker to populate with more returns. * Sync with distributor on GET_BUF flag. Release retptrs. */ @@ -322,11 +399,15 @@ release(struct rte_distributor *d, unsigned int wkr) unsigned int i; handle_returns(d, wkr); + if (unlikely(!d->active[wkr])) + return 0; /* Sync with worker on GET_BUF flag */ while (!(__atomic_load_n(&(d->bufs[wkr].bufptr64[0]), __ATOMIC_ACQUIRE) & RTE_DISTRIB_GET_BUF)) { handle_returns(d, wkr); + if (unlikely(!d->active[wkr])) + return 0; rte_pause(); } @@ -366,7 +447,7 @@ rte_distributor_process(struct rte_distributor *d, int64_t next_value = 0; uint16_t new_tag = 0; uint16_t flows[RTE_DIST_BURST_SIZE] __rte_cache_aligned; - unsigned int i, j, w, wid; + unsigned int i, j, w, wid, matching_required; if (d->alg_type == RTE_DIST_ALG_SINGLE) { /* Call the old API */ @@ -374,11 +455,13 @@ rte_distributor_process(struct rte_distributor *d, mbufs, num_mbufs); } + for (wid = 0 ; wid < d->num_workers; wid++) + handle_returns(d, wid); + if (unlikely(num_mbufs == 0)) { /* Flush out all non-full cache-lines to workers. */ for (wid = 0 ; wid < d->num_workers; wid++) { /* Sync with worker on GET_BUF flag. */ - handle_returns(d, wid); if (__atomic_load_n(&(d->bufs[wid].bufptr64[0]), __ATOMIC_ACQUIRE) & RTE_DISTRIB_GET_BUF) { release(d, wid); @@ -388,6 +471,9 @@ rte_distributor_process(struct rte_distributor *d, return 0; } + if (unlikely(!d->activesum)) + return 0; + while (next_idx < num_mbufs) { uint16_t matches[RTE_DIST_BURST_SIZE]; unsigned int pkts; @@ -412,22 +498,30 @@ rte_distributor_process(struct rte_distributor *d, for (; i < RTE_DIST_BURST_SIZE; i++) flows[i] = 0; - switch (d->dist_match_fn) { - case RTE_DIST_MATCH_VECTOR: - find_match_vec(d, &flows[0], &matches[0]); - break; - default: - find_match_scalar(d, &flows[0], &matches[0]); - } + matching_required = 1; + for (j = 0; j < pkts; j++) { + if (unlikely(!d->activesum)) + return next_idx; + + if (unlikely(matching_required)) { + switch (d->dist_match_fn) { + case RTE_DIST_MATCH_VECTOR: + find_match_vec(d, &flows[0], + &matches[0]); + break; + default: + find_match_scalar(d, &flows[0], + &matches[0]); + } + matching_required = 0; + } /* * Matches array now contain the intended worker ID (+1) of * the incoming packets. Any zeroes need to be assigned * workers. */ - for (j = 0; j < pkts; j++) { - next_mb = mbufs[next_idx++]; next_value = (((int64_t)(uintptr_t)next_mb) << RTE_DISTRIB_FLAG_BITS); @@ -447,12 +541,18 @@ rte_distributor_process(struct rte_distributor *d, */ /* matches[j] = 0; */ - if (matches[j]) { + if (matches[j] && d->active[matches[j]-1]) { struct rte_distributor_backlog *bl = &d->backlog[matches[j]-1]; if (unlikely(bl->count == RTE_DIST_BURST_SIZE)) { release(d, matches[j]-1); + if (!d->active[matches[j]-1]) { + j--; + next_idx--; + matching_required = 1; + continue; + } } /* Add to worker that already has flow */ @@ -462,11 +562,21 @@ rte_distributor_process(struct rte_distributor *d, bl->pkts[idx] = next_value; } else { - struct rte_distributor_backlog *bl = - &d->backlog[wkr]; + struct rte_distributor_backlog *bl; + + while (unlikely(!d->active[wkr])) + wkr = (wkr + 1) % d->num_workers; + bl = &d->backlog[wkr]; + if (unlikely(bl->count == RTE_DIST_BURST_SIZE)) { release(d, wkr); + if (!d->active[wkr]) { + j--; + next_idx--; + matching_required = 1; + continue; + } } /* Add to current worker worker */ @@ -485,9 +595,7 @@ rte_distributor_process(struct rte_distributor *d, matches[w] = wkr+1; } } - wkr++; - if (wkr >= d->num_workers) - wkr = 0; + wkr = (wkr + 1) % d->num_workers; } /* Flush out all non-full cache-lines to workers. */ @@ -663,6 +771,9 @@ rte_distributor_create(const char *name, for (i = 0 ; i < num_workers ; i++) d->backlog[i].tags = &d->in_flight_tags[i][RTE_DIST_BURST_SIZE]; + memset(d->active, 0, sizeof(d->active)); + d->activesum = 0; + dist_burst_list = RTE_TAILQ_CAST(rte_dist_burst_tailq.head, rte_dist_burst_list); From patchwork Fri Oct 9 22:01:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80235 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7911DA04BC; Sat, 10 Oct 2020 00:04:51 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DAD221D65E; Sat, 10 Oct 2020 00:03:04 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 0C6791D5D8 for ; Sat, 10 Oct 2020 00:02:59 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220243euoutp02a87bd2455ead2f3b5a764c39d263c27e~8cgwDvmwG2500425004euoutp02B for ; Fri, 9 Oct 2020 22:02:43 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220243euoutp02a87bd2455ead2f3b5a764c39d263c27e~8cgwDvmwG2500425004euoutp02B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280963; bh=LemR1YjnE+c23RXwEQzxMBn4x6DVQHDmQWboTCBU/MI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AT824QkuVUIg0cN39bePqggsUi1Mx8F//b7ViKY9CYBiNvnqadUI0wHb4EJkBXPAX qqm2DAODbp+v54xLWIx4nSRPwn6FNjYVyEXSgUNXHnxIsRWm0MXCiRGDcfhNAOk3fG ts5pWYqjDoGUtIq833842SlPM51dFkk7t4gosp70= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20201009220235eucas1p2afe71e16cfd3ef4f3d7e8e6594750dc4~8cgpKgXEO1920419204eucas1p2T; Fri, 9 Oct 2020 22:02:35 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id 3E.E2.06318.BFDD08F5; Fri, 9 Oct 2020 23:02:35 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20201009220235eucas1p17ded8b5bb42f2fef159a5715ef6fbca7~8cgozEMe71072710727eucas1p1i; Fri, 9 Oct 2020 22:02:35 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220235eusmtrp2ade60c4a998e0dedbb4e95c7727db4aa~8cgoykv6p1642116421eusmtrp2c; Fri, 9 Oct 2020 22:02:35 +0000 (GMT) X-AuditID: cbfec7f5-38bff700000018ae-9d-5f80ddfb647a Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 33.5F.06017.BFDD08F5; Fri, 9 Oct 2020 23:02:35 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220233eusmtip152f68bcceaf3e7c2c4d5bba1c67d8df0~8cgm2nqZO1990219902eusmtip1G; Fri, 9 Oct 2020 22:02:33 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:52 +0200 Message-Id: <20201009220202.20834-6-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBIsWRmVeSWpSXmKPExsWy7djPc7q/7zbEG9z6rGdxY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRXDYpqTmZZalF+nYJXBlPnx1l KTgrVXFo/SbGBsbtol2MnBwSAiYS53ecYuli5OIQEljBKNH58AEThPOFUeLg3OuMEM5nRolb 85+wwbT0b90JVbWcUWLR20/MEM4nRomb7c8YQarYBGwljsz8ygpiiwiESTQ37wVawsHBLOAs 8eQr2CBhAXeJeXdugpWzCKhKTOmaxgxi8wq4SmxtPMUOsUxeYvWGA2BxTgE3idZdy8BulRC4 zCbxavd2VogiF4l7P39DXScs8er4FqhmGYn/O+czQTRsY5S4+vsnI4Szn1Hieu8KqCpricP/ QLpBrtOUWL9LHyLsKLHh7jMmkLCEAJ/EjbeCIGFmIHPStunMEGFeiY42IYhqPYmnPVMZYdb+ WfuEBcL2kFi/ZDM7JHyuMkpM6bvBOIFRfhbCsgWMjKsYxVNLi3PTU4uN81LL9YoTc4tL89L1 kvNzNzEC4//0v+NfdzDu+5N0iFGAg1GJh7chuSFeiDWxrLgy9xCjBAezkgiv09nTcUK8KYmV ValF+fFFpTmpxYcYpTlYlMR5jRe9jBUSSE8sSc1OTS1ILYLJMnFwSjUwMteGBS9kYPwqr2S7 XpDp3Ne09qRbL0tSn2zelese6xAclHXsB0v8isfRejcr18hOvm4kmMQz071wR/u0mJWrK2uv HJq9hfXC6y23t3W6KTrkXHmw+OUUKQXhpZLss9021h/Xu6y26N/+yn+i7WzXS3R+p0wMj5aR fuWre35ZZXG/mKtWrP8WJZbijERDLeai4kQA91g9gvsCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrBLMWRmVeSWpSXmKPExsVy+t/xu7q/7zbEG8y5IG9xY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRejZF+aUlqQoZ+cUltkrRhhZG eoaWFnpGJpZ6hsbmsVZGpkr6djYpqTmZZalF+nYJehlPnx1lKTgrVXFo/SbGBsbtol2MnBwS AiYS/Vt3MnUxcnEICSxllLjat4K5i5EDKCEj8eGSAESNsMSfa11sEDUfGCWWvf3BDJJgE7CV ODLzKytIvYhAmMSJlf4gYWYBd4kti6eClQgD2fPu3GQEsVkEVCWmdE0Di/MKuEpsbTzFDjFf XmL1hgNgcU4BN4nWXctYIHY1MkpsOfCSeQIj3wJGhlWMIqmlxbnpucVGesWJucWleel6yfm5 mxiBobjt2M8tOxi73gUfYhTgYFTi4dVIbIgXYk0sK67MPcQowcGsJMLrdPZ0nBBvSmJlVWpR fnxRaU5q8SFGU6CrJjJLiSbnA+MkryTe0NTQ3MLS0NzY3NjMQkmct0PgYIyQQHpiSWp2ampB ahFMHxMHp1QDo5cUv/+HnxWHnocEOfcmCazffZjLrZY/wi5ukmRXjV5D3bw9JptUTx1Zkdcm wLmOfcKLe2yxzr6+th0592S+/3Nf8vL8iSf+W28HZcySaTNt7apm2nHzTYHbVr/0uhVFshIM GrM7fdxvPj+s7cLsy+p2olfzx5zJfoVT/M2WSyxfUrxq6+cXSizFGYmGWsxFxYkAFSUdbFsC AAA= X-CMS-MailID: 20201009220235eucas1p17ded8b5bb42f2fef159a5715ef6fbca7 X-Msg-Generator: CA X-RootMTR: 20201009220235eucas1p17ded8b5bb42f2fef159a5715ef6fbca7 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220235eucas1p17ded8b5bb42f2fef159a5715ef6fbca7 References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 05/15] test/distributor: fix shutdown of busy worker X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The sanity test with worker shutdown delegates all bufs to be processed by a single lcore worker, then it freezes one of the lcore workers and continues to send more bufs. The freezed core shuts down first by calling rte_distributor_return_pkt(). The test intention is to verify if packets assigned to the shut down lcore will be reassigned to another worker. However the shutdown core was not always the one, that was processing packets. The lcore processing mbufs might be different every time test is launched. This is caused by keeping the value of wkr static variable in rte_distributor_process() function between running test cases. Test freezed always lcore with 0 id. The patch stores the id of worker that is processing the data in zero_idx global atomic variable. This way the freezed lcore is always the proper one. Fixes: c3eabff124e6 ("distributor: add unit tests") Cc: bruce.richardson@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Tested-by: David Hunt --- app/test/test_distributor.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index 52230d250..6cd7a2edd 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -28,6 +28,7 @@ struct worker_params worker_params; static volatile int quit; /**< general quit variable for all threads */ static volatile int zero_quit; /**< var for when we just want thr0 to quit*/ static volatile unsigned worker_idx; +static volatile unsigned zero_idx; struct worker_stats { volatile unsigned handled_packets; @@ -340,26 +341,43 @@ handle_work_for_shutdown_test(void *arg) unsigned int total = 0; unsigned int i; unsigned int returned = 0; + unsigned int zero_id = 0; + unsigned int zero_unset; const unsigned int id = __atomic_fetch_add(&worker_idx, 1, __ATOMIC_RELAXED); num = rte_distributor_get_pkt(d, id, buf, NULL, 0); + if (num > 0) { + zero_unset = RTE_MAX_LCORE; + __atomic_compare_exchange_n(&zero_idx, &zero_unset, id, + false, __ATOMIC_ACQ_REL, __ATOMIC_ACQUIRE); + } + zero_id = __atomic_load_n(&zero_idx, __ATOMIC_ACQUIRE); + /* wait for quit single globally, or for worker zero, wait * for zero_quit */ - while (!quit && !(id == 0 && zero_quit)) { + while (!quit && !(id == zero_id && zero_quit)) { worker_stats[id].handled_packets += num; count += num; for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, id, buf, NULL, 0); + + if (num > 0) { + zero_unset = RTE_MAX_LCORE; + __atomic_compare_exchange_n(&zero_idx, &zero_unset, id, + false, __ATOMIC_ACQ_REL, __ATOMIC_ACQUIRE); + } + zero_id = __atomic_load_n(&zero_idx, __ATOMIC_ACQUIRE); + total += num; } worker_stats[id].handled_packets += num; count += num; returned = rte_distributor_return_pkt(d, id, buf, num); - if (id == 0) { + if (id == zero_id) { /* for worker zero, allow it to restart to pick up last packet * when all workers are shutting down. */ @@ -578,6 +596,7 @@ quit_workers(struct worker_params *wp, struct rte_mempool *p) rte_eal_mp_wait_lcore(); quit = 0; worker_idx = 0; + zero_idx = RTE_MAX_LCORE; } static int From patchwork Fri Oct 9 22:01:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80234 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A09F2A04BC; Sat, 10 Oct 2020 00:04:18 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id ED22F1D634; Sat, 10 Oct 2020 00:03:01 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id E95C01D62F for ; Sat, 10 Oct 2020 00:02:58 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220243euoutp021d44f467527fdd80601053369ee112fa~8cgv-k9U81684516845euoutp02s for ; Fri, 9 Oct 2020 22:02:43 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220243euoutp021d44f467527fdd80601053369ee112fa~8cgv-k9U81684516845euoutp02s DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280963; bh=Ac7zUxbaUtkPUHeaPly+rcwUzr6TKwAmXEJAVMzD+lE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rn8bNyvX39iuov3zXGACyi0ZDtBbe3EeUQ1deeCcBYTUgl+/WCrwNB3kUfc6MtgJ5 5U340nLFV5vR90YFuB1ooPiDiCuPW1nljdth0TLio2M7zwm7GppWRXqNR5eW2MgQxS n1SvIGJ6kii5po4uTJDjNheDnWmgSB3ArCHr8gC0= Received: from eusmges1new.samsung.com (unknown [203.254.199.242]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20201009220237eucas1p1e87988b9a9423b238e5a50ed66f804eb~8cgqIwZ2g0993409934eucas1p1Z; Fri, 9 Oct 2020 22:02:37 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges1new.samsung.com (EUCPMTA) with SMTP id EE.77.06456.CFDD08F5; Fri, 9 Oct 2020 23:02:36 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20201009220236eucas1p192e34b3bbf00681ec90de296abd1a6b5~8cgp0paro1070410704eucas1p1X; Fri, 9 Oct 2020 22:02:36 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220236eusmtrp2e846ecb2be6b0c14836e3cd6d5a12621~8cgp0JXBb1642116421eusmtrp2e; Fri, 9 Oct 2020 22:02:36 +0000 (GMT) X-AuditID: cbfec7f2-809ff70000001938-06-5f80ddfc6234 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 93.5F.06017.CFDD08F5; Fri, 9 Oct 2020 23:02:36 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220235eusmtip12df4f3d8b0749d38d0a8b56d634319d3~8cgo_KVhq1990219902eusmtip1H; Fri, 9 Oct 2020 22:02:35 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:53 +0200 Message-Id: <20201009220202.20834-7-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBIsWRmVeSWpSXmKPExsWy7djPc7p/7jbEG/yfZWVxY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRXDYpqTmZZalF+nYJXBmXO88y FjzUrejdtZOxgXGSahcjJ4eEgInEwp6bjF2MXBxCAisYJV71nWKGcL4wSuw+uw/K+cwo8ezB dSaYlp6ZfewQieWMEh+Wt0P1f2KUuHjoPytIFZuArcSRmV/BbBGBMInm5r0sXYwcHMwCzhJP vrKBhIUFPCU2vnjNDmKzCKhK3Dz4BizOK+AqcX7VJlaIZfISqzccYAaxOQXcJFp3LWMB2SUh cJ1N4vWmZVBFLhIXu1dD2cISr45vYYewZST+75zPBNGwjVHi6u+fjBDOfkaJ670roKqsJQ7/ +80GcZ2mxPpd+hBhR4m5t6Yzg4QlBPgkbrwVBAkzA5mTtsGEeSU62oQgqvUknvZMZYRZ+2ft ExYI20Ni/40HbJDwucoosejpdMYJjPKzEJYtYGRcxSieWlqcm55abJiXWq5XnJhbXJqXrpec n7uJERj/p/8d/7SD8eulpEOMAhyMSjy8DckN8UKsiWXFlbmHGCU4mJVEeJ3Ono4T4k1JrKxK LcqPLyrNSS0+xCjNwaIkzmu86GWskEB6YklqdmpqQWoRTJaJg1OqgXG7UpfE33fC0i8kXd4Z 9MwuNrl0ZeKWwk3Xd8dZfiqKECkryTiatvXs/BuBmUaTPdwV/SdwJZ5/46zpr7N5a0LRxxDp OUvVMyquv0/+9/kzX+C0Wh7fApcIxgfbj2jez7RSvnZMhivnRJPqrCuzfyx02cDe+/XKieJH tocUHodIGTtd+53O46LEUpyRaKjFXFScCAD+uX48+wIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrBLMWRmVeSWpSXmKPExsVy+t/xu7p/7jbEGyzeqmxxY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRejZF+aUlqQoZ+cUltkrRhhZG eoaWFnpGJpZ6hsbmsVZGpkr6djYpqTmZZalF+nYJehmXO88yFjzUrejdtZOxgXGSahcjJ4eE gIlEz8w+9i5GLg4hgaWMEgvOTGDsYuQASshIfLgkAFEjLPHnWhcbRM0HRon2TatZQRJsArYS R2Z+ZQWpFxEIkzix0h8kzCzgLrFl8VRmEFtYwFNi44vX7CA2i4CqxM2Db9hAbF4BV4nzqzax QsyXl1i94QBYPaeAm0TrrmUsELsaGSW2HHjJPIGRbwEjwypGkdTS4tz03GIjveLE3OLSvHS9 5PzcTYzAUNx27OeWHYxd74IPMQpwMCrx8GokNsQLsSaWFVfmHmKU4GBWEuF1Ons6Tog3JbGy KrUoP76oNCe1+BCjKdBVE5mlRJPzgXGSVxJvaGpobmFpaG5sbmxmoSTO2yFwMEZIID2xJDU7 NbUgtQimj4mDU6qBcVrHlIVXrxdv8KicuL7Mz8Z044/euYfPLpN3PdgU8Knqxy0t+8KWpmP5 mZ7up2V9Uy4Z2IeVLzqV52vJ5zfht5VKXoHoO9Grz884CF169lif78l10dtaJ0yulXju3zFV 9sSO7Gl93jq9Nimi7b1fvI2yfi953z7vQyyz3eUZ5a87QzpUu0snK7EUZyQaajEXFScCAEdP Eq1bAgAA X-CMS-MailID: 20201009220236eucas1p192e34b3bbf00681ec90de296abd1a6b5 X-Msg-Generator: CA X-RootMTR: 20201009220236eucas1p192e34b3bbf00681ec90de296abd1a6b5 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220236eucas1p192e34b3bbf00681ec90de296abd1a6b5 References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 06/15] test/distributor: synchronize lcores statistics X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Statistics of handled packets are cleared and read on main lcore, while they are increased in workers handlers on different lcores. Without synchronization occasionally showed invalid values. This patch uses atomic acquire/release mechanisms to synchronize. Fixes: c3eabff124e6 ("distributor: add unit tests") Cc: bruce.richardson@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- app/test/test_distributor.c | 43 +++++++++++++++++++++++++------------ 1 file changed, 29 insertions(+), 14 deletions(-) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index 6cd7a2edd..838459392 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -43,7 +43,8 @@ total_packet_count(void) { unsigned i, count = 0; for (i = 0; i < worker_idx; i++) - count += worker_stats[i].handled_packets; + count += __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE); return count; } @@ -51,7 +52,10 @@ total_packet_count(void) static inline void clear_packet_count(void) { - memset(&worker_stats, 0, sizeof(worker_stats)); + unsigned int i; + for (i = 0; i < RTE_MAX_LCORE; i++) + __atomic_store_n(&worker_stats[i].handled_packets, 0, + __ATOMIC_RELEASE); } /* this is the basic worker function for sanity test @@ -69,13 +73,13 @@ handle_work(void *arg) num = rte_distributor_get_pkt(db, id, buf, NULL, 0); while (!quit) { __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_RELAXED); + __ATOMIC_ACQ_REL); count += num; num = rte_distributor_get_pkt(db, id, buf, buf, num); } __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_RELAXED); + __ATOMIC_ACQ_REL); count += num; rte_distributor_return_pkt(db, id, buf, num); return 0; @@ -131,7 +135,8 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); printf("Sanity test with all zero hashes done.\n"); /* pick two flows and check they go correctly */ @@ -156,7 +161,9 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n( + &worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); printf("Sanity test with two hash values done\n"); } @@ -182,7 +189,8 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); printf("Sanity test with non-zero hashes done\n"); rte_mempool_put_bulk(p, (void *)bufs, BURST); @@ -275,14 +283,16 @@ handle_work_with_free_mbufs(void *arg) num = rte_distributor_get_pkt(d, id, buf, NULL, 0); while (!quit) { - worker_stats[id].handled_packets += num; count += num; + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, id, buf, NULL, 0); } - worker_stats[id].handled_packets += num; count += num; + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); rte_distributor_return_pkt(d, id, buf, num); return 0; } @@ -358,8 +368,9 @@ handle_work_for_shutdown_test(void *arg) /* wait for quit single globally, or for worker zero, wait * for zero_quit */ while (!quit && !(id == zero_id && zero_quit)) { - worker_stats[id].handled_packets += num; count += num; + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, id, buf, NULL, 0); @@ -373,10 +384,11 @@ handle_work_for_shutdown_test(void *arg) total += num; } - worker_stats[id].handled_packets += num; count += num; returned = rte_distributor_return_pkt(d, id, buf, num); + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); if (id == zero_id) { /* for worker zero, allow it to restart to pick up last packet * when all workers are shutting down. @@ -387,7 +399,8 @@ handle_work_for_shutdown_test(void *arg) num = rte_distributor_get_pkt(d, id, buf, NULL, 0); while (!quit) { - worker_stats[id].handled_packets += num; + __atomic_fetch_add(&worker_stats[id].handled_packets, + num, __ATOMIC_ACQ_REL); count += num; rte_pktmbuf_free(pkt); num = rte_distributor_get_pkt(d, id, buf, NULL, 0); @@ -454,7 +467,8 @@ sanity_test_with_worker_shutdown(struct worker_params *wp, for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); if (total_packet_count() != BURST * 2) { printf("Line %d: Error, not all packets flushed. " @@ -507,7 +521,8 @@ test_flush_with_worker_shutdown(struct worker_params *wp, zero_quit = 0; for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); if (total_packet_count() != BURST) { printf("Line %d: Error, not all packets flushed. " From patchwork Fri Oct 9 22:01:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80236 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1CB7FA04BC; Sat, 10 Oct 2020 00:05:10 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 46D4F1D625; Sat, 10 Oct 2020 00:03:08 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 111441D651 for ; Sat, 10 Oct 2020 00:03:04 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220244euoutp02464ddbf23fcbff49927d1091ccb477ee~8cgxg4mYM1684516845euoutp02u for ; Fri, 9 Oct 2020 22:02:44 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220244euoutp02464ddbf23fcbff49927d1091ccb477ee~8cgxg4mYM1684516845euoutp02u DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280964; bh=Ijucr6xPwukpZ96boOK/+8uaapTvTE3KxkYfXLm/HJY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sOdO1pe7Ek6tdQBoO1qHvVi9G47UqoTJojNDOxjTnQcr+O6bglVqAyQW7/1mpBeml ynQ1SAwJmKmeEJWFxIPzg+uh+yyLuxYKgi88tA8XjbmxigvckK3rwwuMfYeaRomxne rozJ289LTIlhnCRX+O8v9QDjOrBHYsS3m2UDZfBw= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20201009220239eucas1p1d278a00f9834d426c8287460e249a6ad~8cgshJWT20450404504eucas1p10; Fri, 9 Oct 2020 22:02:39 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id 9E.E2.06318.FFDD08F5; Fri, 9 Oct 2020 23:02:39 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20201009220238eucas1p2e86c0026064774e5b494c16c7fd384ec~8cgrUWBrL1922919229eucas1p2U; Fri, 9 Oct 2020 22:02:38 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220238eusmtrp262c786c28a818128d4a77c2df4959307~8cgrTx8-G1642116421eusmtrp2f; Fri, 9 Oct 2020 22:02:38 +0000 (GMT) X-AuditID: cbfec7f5-371ff700000018ae-a0-5f80ddffed8b Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 83.E2.06314.EFDD08F5; Fri, 9 Oct 2020 23:02:38 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220236eusmtip130620801bc7427f396306dbb816af2f6~8cgp7b6Ej1334813348eusmtip1x; Fri, 9 Oct 2020 22:02:36 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:54 +0200 Message-Id: <20201009220202.20834-8-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrEIsWRmVeSWpSXmKPExsWy7djPc7r/7zbEGyy9LWtxY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRXDYpqTmZZalF+nYJXBkXl+xl LZjEXXH+zBy2BsbNnF2MnBwSAiYSn6/PYu1i5OIQEljBKDHl5yIo5wujxPa951hAqoQEPjNK vD8R2sXIAdbRsS4PomY5o0T34z5miJpPjBKz5zGB2GwCthJHZn5lBbFFBMIkmpv3soD0Mgs4 Szz5ygYSFhbwknjW2AMWZhFQlVj01RskzCvgKnF531Z2iNvkJVZvOAA2nVPATaJ11zIWiPhl NolNxwMhbBeJ3tNvoOqFJV4d3wJly0j83zmfCeRMCYFtjBJXf/9khHD2M0pc710BVWUtcfjf bzaI2zQl1u/Sh3jRUWLxFgsIk0/ixltBkGJmIHPStunMEGFeiY42IYgZehJPe6Yywmz9s/YJ 1JUeEu0bPrNDAuoqo0TX7suMExjlZyHsWsDIuIpRPLW0ODc9tdg4L7Vcrzgxt7g0L10vOT93 EyMw5k//O/51B+O+P0mHGAU4GJV4eBuSG+KFWBPLiitzDzFKcDArifA6nT0dJ8SbklhZlVqU H19UmpNafIhRmoNFSZzXeNHLWCGB9MSS1OzU1ILUIpgsEwenVAPjmS17q8X909e0TvXzNT3g EKZYnjvxzeNa/uR/mg5GW8r8Rbrv5LekzmxcbybofX6LjLuZ7UI2a5lpyeGacr6fTn3/5DW1 rm99ToVWx4Jbyz4yXHxnfDZw30Nt2b8OXVbmf2bdbvXYyfbk36rVug3SZ34F12ZNFV/uanw1 WePOesZ9b2TsMyYrsRRnJBpqMRcVJwIAk7l9ZvUCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrGLMWRmVeSWpSXmKPExsVy+t/xu7r/7jbEG1y4x21xY5W9Rd+kj0wW 7z5tZ7J41rOO0eJfxx92B1aPXwuWsnos3vOSyePguz1MAcxRejZF+aUlqQoZ+cUltkrRhhZG eoaWFnpGJpZ6hsbmsVZGpkr6djYpqTmZZalF+nYJehkXl+xlLZjEXXH+zBy2BsbNnF2MHBwS AiYSHevyuhi5OIQEljJK7Lh9mgUiLiPx4ZJAFyMnkCks8edaFxtEzQdGiUd/mlhAEmwCthJH Zn5lBakXEQiTOLHSHyTMLOAusWXxVGYQW1jAS+JZYw/YSBYBVYlFX71BwrwCrhKX921lhxgv L7F6wwGwck4BN4nWXctYIFY1MkpsOfCSeQIj3wJGhlWMIqmlxbnpucWGesWJucWleel6yfm5 mxiBYbjt2M/NOxgvbQw+xCjAwajEw6uR2BAvxJpYVlyZe4hRgoNZSYTX6ezpOCHelMTKqtSi /Pii0pzU4kOMpkBHTWSWEk3OB8ZIXkm8oamhuYWlobmxubGZhZI4b4fAwRghgfTEktTs1NSC 1CKYPiYOTqkGxmjWk3oF1WH9MT4ZDm2Won1VudE3dLfd6v80Ua2TfeIBIZ5ehbVyxv2ROyZP Frz8+66+6JMEtpLeCzadx3Omu9lOOqmQPrPw/e7n0y47Bwtdyzr7Qob9xRb2D8/meltHpV8R 5135v2P/y1dzz/aYMtzyF+zdcK/06uETqZeEr9wQaQ3btX+vhRJLcUaioRZzUXEiAJoNUqlZ AgAA X-CMS-MailID: 20201009220238eucas1p2e86c0026064774e5b494c16c7fd384ec X-Msg-Generator: CA X-RootMTR: 20201009220238eucas1p2e86c0026064774e5b494c16c7fd384ec X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220238eucas1p2e86c0026064774e5b494c16c7fd384ec References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 07/15] distributor: fix return pkt calls in single mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In the single legacy version of the distributor synchronization requires continues exchange of buffers between distributor and workers. Empty buffers are sent if only handshake synchronization is required. However calls to the rte_distributor_return_pkt() with 0 buffers in single mode were ignored and not passed to the legacy algorithm implementation causing lack of synchronization. This patch fixes this issue by passing NULL as buffer which is a valid way of sending just synchronization handshakes in single mode. Fixes: 775003ad2f96 ("distributor: add new burst-capable library") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- lib/librte_distributor/rte_distributor.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 115443fc0..9fd7dcab7 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -168,6 +168,9 @@ rte_distributor_return_pkt(struct rte_distributor *d, if (num == 1) return rte_distributor_return_pkt_single(d->d_single, worker_id, oldpkt[0]); + else if (num == 0) + return rte_distributor_return_pkt_single(d->d_single, + worker_id, NULL); else return -EINVAL; } From patchwork Fri Oct 9 22:01:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80238 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 648E1A04BC; Sat, 10 Oct 2020 00:05:45 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 781CC1D666; Sat, 10 Oct 2020 00:03:13 +0200 (CEST) Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id 74C821D618 for ; Sat, 10 Oct 2020 00:03:04 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220253euoutp014a3fd5899781ee793530f82450e5c5eb~8cg5Tym_w2276922769euoutp01D for ; Fri, 9 Oct 2020 22:02:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20201009220253euoutp014a3fd5899781ee793530f82450e5c5eb~8cg5Tym_w2276922769euoutp01D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280973; bh=XHBRFJdpy0VQOXDZHZktkwJ397h3dgSUtlAUFqe+eH4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A/FnxLYctm/owAeQX2scjU7lZFFhzRrbYRr9VSuWEMG4mHbevVV6e71Lh6cdtBOlD NvoMvgeYOqx63GOX+tWIMSxSpV/T7w3Ax8wYHsD96BNJO7aTcSL4fUYv75t1jWGdMc sDvH4rVdnwwp+SCVCRbpR54mcaT9+FZkG/tBHlPI= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20201009220247eucas1p20d0a52f47db83ff7e9678188fe3df7b3~8cg0FHg2y1921319213eucas1p2V; Fri, 9 Oct 2020 22:02:47 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 14.88.05997.70ED08F5; Fri, 9 Oct 2020 23:02:47 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20201009220246eucas1p1283b16f1f54c572b5952ca9334d667da~8cgzJIvdh1723217232eucas1p1b; Fri, 9 Oct 2020 22:02:46 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220246eusmtrp2be9e2334d175b3148dd2c2f0288e8c25~8cgzIoCdb1642116421eusmtrp2g; Fri, 9 Oct 2020 22:02:46 +0000 (GMT) X-AuditID: cbfec7f4-677ff7000000176d-03-5f80de07606f Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id E3.E2.06314.60ED08F5; Fri, 9 Oct 2020 23:02:46 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220245eusmtip1cc58a4dbb2adbb93a8c5ea97fdeeeacf~8cgyTP6p51990219902eusmtip1I; Fri, 9 Oct 2020 22:02:45 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:55 +0200 Message-Id: <20201009220202.20834-9-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGIsWRmVeSWpSXmKPExsWy7djP87rs9xriDWZ8VbC4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoLpuU1JzMstQifbsEroxTOx+y FswNqtj8dCt7A+NLxy5GDg4JAROJk5Ntuhi5OIQEVjBK9Kz/yQThfGGUuNj7lRXC+cwo8f3T JrYuRk6wjgezN4DZQgLLGSUu/4yHKPrEKHF/8R0WkASbgK3EkZkg3ZwcIgJhEs3Ne1lA1jEL OEs8+QrWKwxUcvDkZTCbRUBV4sqTTrByXgFXifsr70HtkpdYveEAM4jNKeAm0bprGQvILgmB y2wSK88uYoIocpHYt24VK4QtLPHq+BZ2CFtG4v/O+UwQDdsYJa7+/skI4exnlLjeuwKqylri 8L/fbBDXaUqs36UPEXaUePvoETMkjPgkbrwVBAkzA5mTtk2HCvNKdLQJQVTrSTztmcoIs/bP 2icsELaHxOEpm9kg4XOVUeLs8l8sExjlZyEsW8DIuIpRPLW0ODc9tdgoL7Vcrzgxt7g0L10v OT93EyMw9k//O/5lB+OuP0mHGAU4GJV4eBuSG+KFWBPLiitzDzFKcDArifA6nT0dJ8SbklhZ lVqUH19UmpNafIhRmoNFSZzXeNHLWCGB9MSS1OzU1ILUIpgsEwenVANj2S5LwbOzrKYvE6pf d9CgLdbqxhpD7QUrqvbOep9utOJ7xOF1d//K+ie9ULTd4VSz2kpnz70erg0Lp1vlOT/JZHXP WJZtJRF+3bZAsMYopfVR2SN+paX3FimzLfi29scr3uby2bZ+01sOSt9aZ/fokvrhtv8r72/K +PfOj+PsPC7VJTv9M4s2KbEUZyQaajEXFScCAPpOqT75AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrBLMWRmVeSWpSXmKPExsVy+t/xu7ps9xriDRb1q1ncWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHOUnk1RfmlJqkJGfnGJrVK0oYWR nqGlhZ6RiaWeobF5rJWRqZK+nU1Kak5mWWqRvl2CXsapnQ9ZC+YGVWx+upW9gfGlYxcjJ4eE gInEg9kb2LoYuTiEBJYySvR9PMTcxcgBlJCR+HBJAKJGWOLPtS6omg+MEq8PzGMFSbAJ2Eoc mfmVFaReRCBM4sRKf5Aws4C7xJbFU5lBbGGgkoMnL7OB2CwCqhJXnnSCtfIKuErcX3mPDWK+ vMTqDQfA6jkF3CRady1jgdjVyCix5cBL5gmMfAsYGVYxiqSWFuem5xYb6hUn5haX5qXrJefn bmIEhuK2Yz8372C8tDH4EKMAB6MSD69GYkO8EGtiWXFl7iFGCQ5mJRFep7On44R4UxIrq1KL 8uOLSnNSiw8xmgJdNZFZSjQ5HxgneSXxhqaG5haWhubG5sZmFkrivB0CB2OEBNITS1KzU1ML Uotg+pg4OKUaGN07Mle5HSrWeFs577zKB4HdsX7CRqFekS4+ZvxVqxNWZPgZrHgnt/7ytYs6 gfZz+K8x/u/zftyp8m/qe+crIkGdeXfk1VofaCr8LWfhX+r44LT17n9FLrZuR46JHJGYW3yh /12Giqxx16WYn57zftq5/zuwntl9kQjvsTMvvmY9mHxT+3r0cyWW4oxEQy3mouJEAMcgFe5b AgAA X-CMS-MailID: 20201009220246eucas1p1283b16f1f54c572b5952ca9334d667da X-Msg-Generator: CA X-RootMTR: 20201009220246eucas1p1283b16f1f54c572b5952ca9334d667da X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220246eucas1p1283b16f1f54c572b5952ca9334d667da References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 08/15] test/distributor: fix freeing mbufs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Sanity tests with mbuf alloc and shutdown tests assume that mbufs passed to worker cores are freed in handlers. Such packets should not be returned to the distributor's main core. The only packets that should be returned are the packets send after completion of the tests in quit_workers function. This patch stops returning mbufs to distributor's core. In case of shutdown tests it is impossible to determine how worker and distributor threads would synchronize. Packets used by tests should be freed and packets used during quit_workers() shouldn't. That's why returning mbufs to mempool is moved to test procedure run on distributor thread from worker threads. Additionally this patch cleans up unused variables. Fixes: c0de0eb82e40 ("distributor: switch over to new API") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- app/test/test_distributor.c | 96 ++++++++++++++++++------------------- 1 file changed, 47 insertions(+), 49 deletions(-) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index 838459392..06e01ff9d 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -44,7 +44,7 @@ total_packet_count(void) unsigned i, count = 0; for (i = 0; i < worker_idx; i++) count += __atomic_load_n(&worker_stats[i].handled_packets, - __ATOMIC_ACQUIRE); + __ATOMIC_RELAXED); return count; } @@ -55,7 +55,7 @@ clear_packet_count(void) unsigned int i; for (i = 0; i < RTE_MAX_LCORE; i++) __atomic_store_n(&worker_stats[i].handled_packets, 0, - __ATOMIC_RELEASE); + __ATOMIC_RELAXED); } /* this is the basic worker function for sanity test @@ -67,20 +67,18 @@ handle_work(void *arg) struct rte_mbuf *buf[8] __rte_cache_aligned; struct worker_params *wp = arg; struct rte_distributor *db = wp->dist; - unsigned int count = 0, num; + unsigned int num; unsigned int id = __atomic_fetch_add(&worker_idx, 1, __ATOMIC_RELAXED); num = rte_distributor_get_pkt(db, id, buf, NULL, 0); while (!quit) { __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_ACQ_REL); - count += num; + __ATOMIC_RELAXED); num = rte_distributor_get_pkt(db, id, buf, buf, num); } __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_ACQ_REL); - count += num; + __ATOMIC_RELAXED); rte_distributor_return_pkt(db, id, buf, num); return 0; } @@ -136,7 +134,7 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, __atomic_load_n(&worker_stats[i].handled_packets, - __ATOMIC_ACQUIRE)); + __ATOMIC_RELAXED)); printf("Sanity test with all zero hashes done.\n"); /* pick two flows and check they go correctly */ @@ -163,7 +161,7 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) printf("Worker %u handled %u packets\n", i, __atomic_load_n( &worker_stats[i].handled_packets, - __ATOMIC_ACQUIRE)); + __ATOMIC_RELAXED)); printf("Sanity test with two hash values done\n"); } @@ -190,7 +188,7 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, __atomic_load_n(&worker_stats[i].handled_packets, - __ATOMIC_ACQUIRE)); + __ATOMIC_RELAXED)); printf("Sanity test with non-zero hashes done\n"); rte_mempool_put_bulk(p, (void *)bufs, BURST); @@ -276,23 +274,20 @@ handle_work_with_free_mbufs(void *arg) struct rte_mbuf *buf[8] __rte_cache_aligned; struct worker_params *wp = arg; struct rte_distributor *d = wp->dist; - unsigned int count = 0; unsigned int i; unsigned int num; unsigned int id = __atomic_fetch_add(&worker_idx, 1, __ATOMIC_RELAXED); num = rte_distributor_get_pkt(d, id, buf, NULL, 0); while (!quit) { - count += num; __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_ACQ_REL); + __ATOMIC_RELAXED); for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, id, buf, NULL, 0); } - count += num; __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_ACQ_REL); + __ATOMIC_RELAXED); rte_distributor_return_pkt(d, id, buf, num); return 0; } @@ -318,7 +313,6 @@ sanity_test_with_mbuf_alloc(struct worker_params *wp, struct rte_mempool *p) rte_distributor_process(d, NULL, 0); for (j = 0; j < BURST; j++) { bufs[j]->hash.usr = (i+j) << 1; - rte_mbuf_refcnt_set(bufs[j], 1); } rte_distributor_process(d, bufs, BURST); @@ -342,15 +336,10 @@ sanity_test_with_mbuf_alloc(struct worker_params *wp, struct rte_mempool *p) static int handle_work_for_shutdown_test(void *arg) { - struct rte_mbuf *pkt = NULL; struct rte_mbuf *buf[8] __rte_cache_aligned; struct worker_params *wp = arg; struct rte_distributor *d = wp->dist; - unsigned int count = 0; unsigned int num; - unsigned int total = 0; - unsigned int i; - unsigned int returned = 0; unsigned int zero_id = 0; unsigned int zero_unset; const unsigned int id = __atomic_fetch_add(&worker_idx, 1, @@ -368,11 +357,8 @@ handle_work_for_shutdown_test(void *arg) /* wait for quit single globally, or for worker zero, wait * for zero_quit */ while (!quit && !(id == zero_id && zero_quit)) { - count += num; __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_ACQ_REL); - for (i = 0; i < num; i++) - rte_pktmbuf_free(buf[i]); + __ATOMIC_RELAXED); num = rte_distributor_get_pkt(d, id, buf, NULL, 0); if (num > 0) { @@ -381,15 +367,12 @@ handle_work_for_shutdown_test(void *arg) false, __ATOMIC_ACQ_REL, __ATOMIC_ACQUIRE); } zero_id = __atomic_load_n(&zero_idx, __ATOMIC_ACQUIRE); - - total += num; } - count += num; - returned = rte_distributor_return_pkt(d, id, buf, num); - __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_ACQ_REL); + __ATOMIC_RELAXED); if (id == zero_id) { + rte_distributor_return_pkt(d, id, NULL, 0); + /* for worker zero, allow it to restart to pick up last packet * when all workers are shutting down. */ @@ -400,15 +383,11 @@ handle_work_for_shutdown_test(void *arg) while (!quit) { __atomic_fetch_add(&worker_stats[id].handled_packets, - num, __ATOMIC_ACQ_REL); - count += num; - rte_pktmbuf_free(pkt); + num, __ATOMIC_RELAXED); num = rte_distributor_get_pkt(d, id, buf, NULL, 0); } - returned = rte_distributor_return_pkt(d, - id, buf, num); - printf("Num returned = %d\n", returned); } + rte_distributor_return_pkt(d, id, buf, num); return 0; } @@ -424,7 +403,9 @@ sanity_test_with_worker_shutdown(struct worker_params *wp, { struct rte_distributor *d = wp->dist; struct rte_mbuf *bufs[BURST]; - unsigned i; + struct rte_mbuf *bufs2[BURST]; + unsigned int i; + unsigned int failed = 0; printf("=== Sanity test of worker shutdown ===\n"); @@ -450,16 +431,17 @@ sanity_test_with_worker_shutdown(struct worker_params *wp, */ /* get more buffers to queue up, again setting them to the same flow */ - if (rte_mempool_get_bulk(p, (void *)bufs, BURST) != 0) { + if (rte_mempool_get_bulk(p, (void *)bufs2, BURST) != 0) { printf("line %d: Error getting mbufs from pool\n", __LINE__); + rte_mempool_put_bulk(p, (void *)bufs, BURST); return -1; } for (i = 0; i < BURST; i++) - bufs[i]->hash.usr = 1; + bufs2[i]->hash.usr = 1; /* get worker zero to quit */ zero_quit = 1; - rte_distributor_process(d, bufs, BURST); + rte_distributor_process(d, bufs2, BURST); /* flush the distributor */ rte_distributor_flush(d); @@ -468,15 +450,21 @@ sanity_test_with_worker_shutdown(struct worker_params *wp, for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, __atomic_load_n(&worker_stats[i].handled_packets, - __ATOMIC_ACQUIRE)); + __ATOMIC_RELAXED)); if (total_packet_count() != BURST * 2) { printf("Line %d: Error, not all packets flushed. " "Expected %u, got %u\n", __LINE__, BURST * 2, total_packet_count()); - return -1; + failed = 1; } + rte_mempool_put_bulk(p, (void *)bufs, BURST); + rte_mempool_put_bulk(p, (void *)bufs2, BURST); + + if (failed) + return -1; + printf("Sanity test with worker shutdown passed\n\n"); return 0; } @@ -490,7 +478,8 @@ test_flush_with_worker_shutdown(struct worker_params *wp, { struct rte_distributor *d = wp->dist; struct rte_mbuf *bufs[BURST]; - unsigned i; + unsigned int i; + unsigned int failed = 0; printf("=== Test flush fn with worker shutdown (%s) ===\n", wp->name); @@ -522,15 +511,20 @@ test_flush_with_worker_shutdown(struct worker_params *wp, for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, __atomic_load_n(&worker_stats[i].handled_packets, - __ATOMIC_ACQUIRE)); + __ATOMIC_RELAXED)); if (total_packet_count() != BURST) { printf("Line %d: Error, not all packets flushed. " "Expected %u, got %u\n", __LINE__, BURST, total_packet_count()); - return -1; + failed = 1; } + rte_mempool_put_bulk(p, (void *)bufs, BURST); + + if (failed) + return -1; + printf("Flush test with worker shutdown passed\n\n"); return 0; } @@ -596,7 +590,10 @@ quit_workers(struct worker_params *wp, struct rte_mempool *p) const unsigned num_workers = rte_lcore_count() - 1; unsigned i; struct rte_mbuf *bufs[RTE_MAX_LCORE]; - rte_mempool_get_bulk(p, (void *)bufs, num_workers); + if (rte_mempool_get_bulk(p, (void *)bufs, num_workers) != 0) { + printf("line %d: Error getting mbufs from pool\n", __LINE__); + return; + } zero_quit = 0; quit = 1; @@ -604,11 +601,12 @@ quit_workers(struct worker_params *wp, struct rte_mempool *p) bufs[i]->hash.usr = i << 1; rte_distributor_process(d, bufs, num_workers); - rte_mempool_put_bulk(p, (void *)bufs, num_workers); - rte_distributor_process(d, NULL, 0); rte_distributor_flush(d); rte_eal_mp_wait_lcore(); + + rte_mempool_put_bulk(p, (void *)bufs, num_workers); + quit = 0; worker_idx = 0; zero_idx = RTE_MAX_LCORE; From patchwork Fri Oct 9 22:01:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80237 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D4360A04BC; Sat, 10 Oct 2020 00:05:24 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CBEEB1D670; Sat, 10 Oct 2020 00:03:11 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id C70911D65B for ; Sat, 10 Oct 2020 00:03:04 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220254euoutp021385b0e811f065024805c52a6e8ab5e1~8cg6daOu61684516845euoutp02v for ; Fri, 9 Oct 2020 22:02:54 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220254euoutp021385b0e811f065024805c52a6e8ab5e1~8cg6daOu61684516845euoutp02v DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280974; bh=faTZS3uOROPHkE9n9BxhsWOgeJZpgrOtfleCwr1/fg8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N2ROe+JdUIAN+Wgsc/hlp5XnXLXyhXCK2RWxJ6Lfhkp3q9akw9DTBS8NJ+Rad9ReI iVG+I9tIvnTYK8n23w1AWBoOUEhP/fJ8Rkaatu3hvoCAsDrSAm7XM5M77kBSqamOdF lbpsq0vD8gDmTBsm90mTmdRKPKWZpsoWSwpxroc4= Received: from eusmges1new.samsung.com (unknown [203.254.199.242]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20201009220248eucas1p25090e0b04a0e234a90740fcfe7dbe9ae~8cg0wuy-q1926719267eucas1p2j; Fri, 9 Oct 2020 22:02:48 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges1new.samsung.com (EUCPMTA) with SMTP id 4F.77.06456.80ED08F5; Fri, 9 Oct 2020 23:02:48 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20201009220247eucas1p1a783663e586127cbfd406a61e13c40eb~8cgz_eH3S0993409934eucas1p1c; Fri, 9 Oct 2020 22:02:47 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220247eusmtrp240f9f8f5f3986dbf8691be90abd645bd~8cgz97xtZ1642116421eusmtrp2h; Fri, 9 Oct 2020 22:02:47 +0000 (GMT) X-AuditID: cbfec7f2-809ff70000001938-12-5f80de08b372 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id F3.5F.06017.70ED08F5; Fri, 9 Oct 2020 23:02:47 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220246eusmtip1a78229ccacaaf34e4a3b32888d1863f1~8cgzRoikl1334813348eusmtip1y; Fri, 9 Oct 2020 22:02:46 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:56 +0200 Message-Id: <20201009220202.20834-10-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrOIsWRmVeSWpSXmKPExsWy7djPc7oc9xriDR6/FLa4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoLpuU1JzMstQifbsErowvt5uY Cl7wVLw7fJ6pgfEVVxcjJ4eEgInEtzVHmbsYuTiEBFYwSlxe/oIFwvnCKPF75URGkCohgc+M Eu//18J0TNvYzAZRtJxR4tiG7awQzidGiRufJ4N1sAnYShyZ+ZUVxBYRCJNobt4LNJaDg1nA WeLJVzaQsLCAg8SeaevBbBYBVYm/B7vBynkF3CQ+bdzJBrFMXmL1hgPMIDYnULx11zKw6yQE LrNJbFx2lRWiyEViSsNLKFtY4tXxLewQtozE/53zmSAatjFKXP39kxHC2c8ocb13BVSVtcTh f7/ZIK7TlFi/Sx8i7Chx/d8ZsLCEAJ/EjbeCIGFmIHPStunMEGFeiY42IYhqPYmnPVMZYdb+ WfuEBcL2kNg7fzo0sK4ySixef4VpAqP8LIRlCxgZVzGKp5YW56anFhvmpZbrFSfmFpfmpesl 5+duYgRG/+l/xz/tYPx6KekQowAHoxIPb0NyQ7wQa2JZcWXuIUYJDmYlEV6ns6fjhHhTEiur Uovy44tKc1KLDzFKc7AoifMaL3oZKySQnliSmp2aWpBaBJNl4uCUamBsWBUWYfddVGA249Nr wksmnTlzVkw1XOjnZ5sL/q9SDm6U7NQM3/Fo9UeOxbdj7EqXl1z9+cLZtCL/RNqxbfs3h3W+ j06f/t/Tiu+70lPvuXqy1xIvzzH9LGv2onZqldZe68+yX2+sv5givfrlNMUzp1bOWH6/1GiL 04agU7br3J4/908W/sR1UYmlOCPRUIu5qDgRAFKsMe36AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrOLMWRmVeSWpSXmKPExsVy+t/xu7rs9xriDRaukLG4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoPZui/NKSVIWM/OISW6VoQwsj PUNLCz0jE0s9Q2PzWCsjUyV9O5uU1JzMstQifbsEvYwvt5uYCl7wVLw7fJ6pgfEVVxcjJ4eE gInEtI3NbCC2kMBSRom3rw27GDmA4jISHy4JQJQIS/y51gVUwgVU8oFRYt/q5awgCTYBW4kj M7+ygtSLCIRJnFjpDxJmFnCX2LJ4KjOILSzgILFn2nqw8SwCqhJ/D3aDtfIKuEl82riTDWK+ vMTqDQfA6jmB4q27lrFA7GpklNhy4CXzBEa+BYwMqxhFUkuLc9Nzi430ihNzi0vz0vWS83M3 MQIDcduxn1t2MHa9Cz7EKMDBqMTDq5HYEC/EmlhWXJl7iFGCg1lJhNfp7Ok4Id6UxMqq1KL8 +KLSnNTiQ4ymQFdNZJYSTc4HRkleSbyhqaG5haWhubG5sZmFkjhvh8DBGCGB9MSS1OzU1ILU Ipg+Jg5OqQbGWHO/b4plgitKCn7zbou7rrnVNyJ6b5qE4vnkqhq3xVzFlx61KD7ZPW3d1wkq oRMYnSbZ3ze7bDkvS2hpikf9x2+dXSLx0xb2rXOszN5pyF4wkenyqUDd6JOMPvOnzFKav+Kp x/KJXls8H+7jsWPVXhJapahupPN2YozliUSRyqU2D36c7ClQYinOSDTUYi4qTgQAO7lH5loC AAA= X-CMS-MailID: 20201009220247eucas1p1a783663e586127cbfd406a61e13c40eb X-Msg-Generator: CA X-RootMTR: 20201009220247eucas1p1a783663e586127cbfd406a61e13c40eb X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220247eucas1p1a783663e586127cbfd406a61e13c40eb References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 09/15] test/distributor: collect return mbufs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" During quit_workers function distributor's main core processes some packets to wake up pending worker cores so they can quit. As quit_workers acts also as a cleanup procedure for next test case it should also collect these packages returned by workers' handlers, so the cyclic buffer with returned packets in distributor remains empty. Fixes: c3eabff124e6 ("distributor: add unit tests") Cc: bruce.richardson@intel.com Fixes: c0de0eb82e40 ("distributor: switch over to new API") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- app/test/test_distributor.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index 06e01ff9d..ed03040d1 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -590,6 +590,7 @@ quit_workers(struct worker_params *wp, struct rte_mempool *p) const unsigned num_workers = rte_lcore_count() - 1; unsigned i; struct rte_mbuf *bufs[RTE_MAX_LCORE]; + struct rte_mbuf *returns[RTE_MAX_LCORE]; if (rte_mempool_get_bulk(p, (void *)bufs, num_workers) != 0) { printf("line %d: Error getting mbufs from pool\n", __LINE__); return; @@ -605,6 +606,10 @@ quit_workers(struct worker_params *wp, struct rte_mempool *p) rte_distributor_flush(d); rte_eal_mp_wait_lcore(); + while (rte_distributor_returned_pkts(d, returns, RTE_MAX_LCORE)) + ; + + rte_distributor_clear_returns(d); rte_mempool_put_bulk(p, (void *)bufs, num_workers); quit = 0; From patchwork Fri Oct 9 22:01:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80240 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A0BEDA04BC; Sat, 10 Oct 2020 00:06:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 32F531D694; Sat, 10 Oct 2020 00:03:17 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 09A9A1D666 for ; Sat, 10 Oct 2020 00:03:09 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220254euoutp0291bfdd748baaf3d18cadc1e34e2efd07~8cg6mz5FV2454124541euoutp02J for ; Fri, 9 Oct 2020 22:02:54 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220254euoutp0291bfdd748baaf3d18cadc1e34e2efd07~8cg6mz5FV2454124541euoutp02J DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280974; bh=fynvMAAU6OLk1/+zR+LOvZdNO64pUieNLjkvDKcsip8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JHg0PxhVtdVH6OYQg5ixu/BGxgJSxtiCsZBK2tm3P3OkCgNN1fb/dC27l+bsN8c6J 5wM+l8QGFFFPx+PEwLfSQJVifVOoCPESC1uzfzJ3fGosNlaOvjjlDDVogDGwnHZtEx WDg7gv/q7r9WP9U+tlTg26Uf4yyTwffI6s69jcls= Received: from eusmges1new.samsung.com (unknown [203.254.199.242]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20201009220248eucas1p136ad470b30de992daf84753fe3e59b8f~8cg08lYR01072710727eucas1p1m; Fri, 9 Oct 2020 22:02:48 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges1new.samsung.com (EUCPMTA) with SMTP id AF.77.06456.80ED08F5; Fri, 9 Oct 2020 23:02:48 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20201009220248eucas1p156346857c1aab2340ccd7549abdce966~8cg0mrw5A1023910239eucas1p1e; Fri, 9 Oct 2020 22:02:48 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220248eusmtrp270636e8194bb792b3f9371403779a750~8cg0mNXMl1642116421eusmtrp2i; Fri, 9 Oct 2020 22:02:48 +0000 (GMT) X-AuditID: cbfec7f2-809ff70000001938-14-5f80de08d91e Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 44.E2.06314.80ED08F5; Fri, 9 Oct 2020 23:02:48 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220247eusmtip14dfa75672817af520c9810563a17ccb0~8cg0BZEq51990219902eusmtip1J; Fri, 9 Oct 2020 22:02:47 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:57 +0200 Message-Id: <20201009220202.20834-11-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrKIsWRmVeSWpSXmKPExsWy7djP87oc9xriDf4+lbO4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoLpuU1JzMstQifbsEroxbl0oK PkhVXHzWxNzAeE+0i5GTQ0LARKJv2SW2LkYuDiGBFYwSG17MYIdwvjBK9E05zArhfGaU6O/e wArTsmhyPxNEYjmjxNOmE1AtnxgldnafZgSpYhOwlTgy8ytYh4hAmERz816WLkYODmYBZ4kn X9lAwsICHhJrOxezgYRZBFQlJv6oBQnzCrhJXDoLcgXILnmJ1RsOMIPYnEDx1l3LWCDi19kk ervyIGwXic1PTrBB2MISr45vgeqVkfi/cz7YnRIC2xglrv7+yQjh7GeUuN67AqrKWuLwv99s ELdpSqzfpQ8RdpRYsWkDI0hYQoBP4sZbQZAwM5A5adt0Zogwr0RHmxBEtZ7E056pjDBr/6x9 AnWmh8Saf7egoXuVUeLAwSbmCYzysxCWLWBkXMUonlpanJueWmyYl1quV5yYW1yal66XnJ+7 iREY+af/Hf+0g/HrpaRDjAIcjEo8vA3JDfFCrIllxZW5hxglOJiVRHidzp6OE+JNSaysSi3K jy8qzUktPsQozcGiJM5rvOhlrJBAemJJanZqakFqEUyWiYNTqoFxpvzrjCzV4IjV3tNVt89l /5sbovtto2zJ06uWDIfkNpyfuEOFa64Dx6u3Zmk1POc6+Oojf3XJm53T/rttbSDzRrZ7kZmF Wl0ZU6+bXs9SsDmvUMe/tm3hzfXPPTMjgt79P3Ms02H5tTSHzVFhkfwWXsalf7vPuvQ+entt /tMu5VSJu1zH71xXYinOSDTUYi4qTgQA/i3XkfgCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrGLMWRmVeSWpSXmKPExsVy+t/xu7oc9xriDZYs4bK4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoPZui/NKSVIWM/OISW6VoQwsj PUNLCz0jE0s9Q2PzWCsjUyV9O5uU1JzMstQifbsEvYxbl0oKPkhVXHzWxNzAeE+0i5GTQ0LA RGLR5H6mLkYuDiGBpYwSFxf2MXYxcgAlZCQ+XBKAqBGW+HOtiw2i5gOjxLHvWxlBEmwCthJH Zn5lBakXEQiTOLHSHyTMLOAusWXxVGYQW1jAQ2Jt52I2kBIWAVWJiT9qQcK8Am4Sl87OYIcY Ly+xesMBsHJOoHjrrmUsEKsaGSW2HHjJPIGRbwEjwypGkdTS4tz03GJDveLE3OLSvHS95Pzc TYzAMNx27OfmHYyXNgYfYhTgYFTi4dVIbIgXYk0sK67MPcQowcGsJMLrdPZ0nBBvSmJlVWpR fnxRaU5q8SFGU6CjJjJLiSbnA2MkryTe0NTQ3MLS0NzY3NjMQkmct0PgYIyQQHpiSWp2ampB ahFMHxMHp1QDo8Jr1SP2875KXpJ6KHXK6n7i7xe83/NyVv1PeKjgKmt2dtes6Tt5Ys4fOlUz YWue9Z+9mTcfT+I+HbBYrnzBpl83E2W9I3vkq6T/Hrp9UFljl8Ld7CuXwqsK5n1ZolgXJZO6 3/D//uj4o0f27azZEiAh77ipQeaV+oSD2dxs/W5fTQ5kbq+3FVZiKc5INNRiLipOBADLVWIK WQIAAA== X-CMS-MailID: 20201009220248eucas1p156346857c1aab2340ccd7549abdce966 X-Msg-Generator: CA X-RootMTR: 20201009220248eucas1p156346857c1aab2340ccd7549abdce966 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220248eucas1p156346857c1aab2340ccd7549abdce966 References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 10/15] distributor: align API documentation with code X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" After introducing burst API there were some artefacts in the API documentation from legacy single API. Also the rte_distributor_poll_pkt() function return values mismatched the implementation. Fixes: c0de0eb82e40 ("distributor: switch over to new API") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- lib/librte_distributor/rte_distributor.h | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/lib/librte_distributor/rte_distributor.h b/lib/librte_distributor/rte_distributor.h index 327c0c4ab..a073e6461 100644 --- a/lib/librte_distributor/rte_distributor.h +++ b/lib/librte_distributor/rte_distributor.h @@ -155,7 +155,7 @@ rte_distributor_clear_returns(struct rte_distributor *d); * @param pkts * The mbufs pointer array to be filled in (up to 8 packets) * @param oldpkt - * The previous packet, if any, being processed by the worker + * The previous packets, if any, being processed by the worker * @param retcount * The number of packets being returned * @@ -187,15 +187,15 @@ rte_distributor_return_pkt(struct rte_distributor *d, /** * API called by a worker to request a new packet to process. - * Any previous packet given to the worker is assumed to have completed + * Any previous packets given to the worker are assumed to have completed * processing, and may be optionally returned to the distributor via * the oldpkt parameter. - * Unlike rte_distributor_get_pkt_burst(), this function does not wait for a - * new packet to be provided by the distributor. + * Unlike rte_distributor_get_pkt(), this function does not wait for + * new packets to be provided by the distributor. * - * NOTE: after calling this function, rte_distributor_poll_pkt_burst() should - * be used to poll for the packet requested. The rte_distributor_get_pkt_burst() - * API should *not* be used to try and retrieve the new packet. + * NOTE: after calling this function, rte_distributor_poll_pkt() should + * be used to poll for the packets requested. The rte_distributor_get_pkt() + * API should *not* be used to try and retrieve the new packets. * * @param d * The distributor instance to be used @@ -213,9 +213,9 @@ rte_distributor_request_pkt(struct rte_distributor *d, unsigned int count); /** - * API called by a worker to check for a new packet that was previously + * API called by a worker to check for new packets that were previously * requested by a call to rte_distributor_request_pkt(). It does not wait - * for the new packet to be available, but returns NULL if the request has + * for the new packets to be available, but returns if the request has * not yet been fulfilled by the distributor. * * @param d @@ -227,8 +227,9 @@ rte_distributor_request_pkt(struct rte_distributor *d, * The array of mbufs being given to the worker * * @return - * The number of packets being given to the worker thread, zero if no - * packet is yet available. + * The number of packets being given to the worker thread, + * -1 if no packets are yet available (burst API - RTE_DIST_ALG_BURST) + * 0 if no packets are yet available (legacy single API - RTE_DIST_ALG_SINGLE) */ int rte_distributor_poll_pkt(struct rte_distributor *d, From patchwork Fri Oct 9 22:01:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80239 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 46B46A04BC; Sat, 10 Oct 2020 00:06:13 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 929551D687; Sat, 10 Oct 2020 00:03:15 +0200 (CEST) Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id 8A4761D664 for ; Sat, 10 Oct 2020 00:03:08 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220258euoutp0116ed2b148e5d8bdc8fb044f562dec225~8cg_Dcl222277122771euoutp01E for ; Fri, 9 Oct 2020 22:02:58 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20201009220258euoutp0116ed2b148e5d8bdc8fb044f562dec225~8cg_Dcl222277122771euoutp01E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280978; bh=9P4E0B5kieQ8/tBzQ8nyDsGgZm6tUvk7hTzdKz8POqg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sjbZPTY0IYDSUpN/znyy/DE6KFMsYi9kTIPrfRHwnb3BbBXisyL6XXVo+7efEiCXw DilxPPOOL9B3XtvMU6C1QbxvufAjOibDDHuCqkzI4nXQL6VdpPm80Lv+LuT0IPQdqG nuIpViHQ1Nx9IsG9zsS7Y4w/kCna+PhooDz0r8qE= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20201009220252eucas1p18c131871fbdb533a883d2b01fbf3e6fe~8cg4PW1og1070410704eucas1p1Z; Fri, 9 Oct 2020 22:02:52 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id 3F.E2.06318.C0ED08F5; Fri, 9 Oct 2020 23:02:52 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20201009220250eucas1p18587737171d82a9bde52c767ee8ed24b~8cg3I_pGB0993409934eucas1p1d; Fri, 9 Oct 2020 22:02:50 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220250eusmtrp248f73080e8ca700c2302efa00b61d26a~8cg3IfCY81642116421eusmtrp2j; Fri, 9 Oct 2020 22:02:50 +0000 (GMT) X-AuditID: cbfec7f5-38bff700000018ae-af-5f80de0cdbea Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 54.5F.06017.A0ED08F5; Fri, 9 Oct 2020 23:02:50 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220249eusmtip111213ace1658ffabcc4181737eb1cdb6~8cg1bharQ1334813348eusmtip1z; Fri, 9 Oct 2020 22:02:48 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:58 +0200 Message-Id: <20201009220202.20834-12-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBIsWRmVeSWpSXmKPExsWy7djPc7o89xriDWb2slrcWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHMUl01Kak5mWWqRvl0CV8b7zSsY C7pEK46dPMjcwLhGsIuRk0NCwERif9MW1i5GLg4hgRWMErd3XWGDcL4wSpzc950JwvkM5Nx7 zgTTcmp2IyNEYjmjxKNHV1kgnE+MEmdv3mEDqWITsJU4MvMrK4gtIhAm0dy8F6iIg4NZwFni yVewEmEBL4mmnoPMIDaLgKrEzSl7wcp5BdwkPu9dyQyxTF5i9YYDYDYnULx11zKwXRICl9kk HnX2sUEUuUjMmDGTFcIWlnh1fAs7hC0j8X/nfCaIhm2MEld//2SEcPYzSlzvXQFVZS1x+N9v NojrNCXW79KHCDtKrLj/hAkkLCHAJ3HjLTjAmIHMSdumM0OEeSU62oQgqvUknvZMZYRZ+2ft ExYI20Piyrop7JDwucoo8eXlHOYJjPKzEJYtYGRcxSieWlqcm55abJyXWq5XnJhbXJqXrpec n7uJERj/p/8d/7qDcd+fpEOMAhyMSjy8DckN8UKsiWXFlbmHGCU4mJVEeJ3Ono4T4k1JrKxK LcqPLyrNSS0+xCjNwaIkzmu86GWskEB6YklqdmpqQWoRTJaJg1OqgdFYNCDuwkwGvcLnm/NT 5QuspkRdSFVYtXCBH9Nr4a3zY16fFo6XNeo4LPhLzWd5IZdXkviUO9eCzNf9Yv7ik8hjtmiP QmUgx4e8i/asyZt7HvvOXXKQ7cnJF9wOaTkWL1cyLcoUXHEpPLBG9EnctbKjflN2TpAK/SSS uffC8sydp+2q8h6YeCixFGckGmoxFxUnAgCe4RDl+wIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrBLMWRmVeSWpSXmKPExsVy+t/xu7pc9xriDf7eN7G4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoPZui/NKSVIWM/OISW6VoQwsj PUNLCz0jE0s9Q2PzWCsjUyV9O5uU1JzMstQifbsEvYz3m1cwFnSJVhw7eZC5gXGNYBcjJ4eE gInEqdmNjF2MXBxCAksZJd5ca2XuYuQASshIfLgkAFEjLPHnWhcbRM0HRokfp64zgSTYBGwl jsz8ygpSLyIQJnFipT9ImFnAXWLL4qnMILawgJdEU89BMJtFQFXi5pS9rCA2r4CbxOe9K5kh 5stLrN5wAMzmBIq37lrGArGrkVFiy4GXzBMY+RYwMqxiFEktLc5Nzy020itOzC0uzUvXS87P 3cQIDMVtx35u2cHY9S74EKMAB6MSD69GYkO8EGtiWXFl7iFGCQ5mJRFep7On44R4UxIrq1KL 8uOLSnNSiw8xmgJdNZFZSjQ5HxgneSXxhqaG5haWhubG5sZmFkrivB0CB2OEBNITS1KzU1ML Uotg+pg4OKUaGJMDzyr1Sxa8LBCxj11oPK+A2951zxZmZ02RfRqtIedSJZwUysQ/LnZ7ankn Uqnd8fTRxRcNjwd9s/jaLRfz8NmUJxPr+Bc+U7de8/DK0aS8n9J73Gq8fqi17nexu1isnuUf dWpCb1NQvVjfvIbOiXLGz7pW/ubO/GV/XWiX8GMTx+pXhbyMSizFGYmGWsxFxYkAPMcebVsC AAA= X-CMS-MailID: 20201009220250eucas1p18587737171d82a9bde52c767ee8ed24b X-Msg-Generator: CA X-RootMTR: 20201009220250eucas1p18587737171d82a9bde52c767ee8ed24b X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220250eucas1p18587737171d82a9bde52c767ee8ed24b References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 11/15] test/distributor: replace delays with spin locks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Instead of making delays in test code and waiting for worker hopefully to reach proper states, synchronize worker shutdown test cases with spin lock on atomic variable. Fixes: c0de0eb82e40 ("distributor: switch over to new API") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- app/test/test_distributor.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index ed03040d1..e8dd75078 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -27,6 +27,7 @@ struct worker_params worker_params; /* statics - all zero-initialized by default */ static volatile int quit; /**< general quit variable for all threads */ static volatile int zero_quit; /**< var for when we just want thr0 to quit*/ +static volatile int zero_sleep; /**< thr0 has quit basic loop and is sleeping*/ static volatile unsigned worker_idx; static volatile unsigned zero_idx; @@ -376,8 +377,10 @@ handle_work_for_shutdown_test(void *arg) /* for worker zero, allow it to restart to pick up last packet * when all workers are shutting down. */ + __atomic_store_n(&zero_sleep, 1, __ATOMIC_RELEASE); while (zero_quit) usleep(100); + __atomic_store_n(&zero_sleep, 0, __ATOMIC_RELEASE); num = rte_distributor_get_pkt(d, id, buf, NULL, 0); @@ -445,7 +448,12 @@ sanity_test_with_worker_shutdown(struct worker_params *wp, /* flush the distributor */ rte_distributor_flush(d); - rte_delay_us(10000); + while (!__atomic_load_n(&zero_sleep, __ATOMIC_ACQUIRE)) + rte_distributor_flush(d); + + zero_quit = 0; + while (__atomic_load_n(&zero_sleep, __ATOMIC_ACQUIRE)) + rte_delay_us(100); for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, @@ -505,9 +513,14 @@ test_flush_with_worker_shutdown(struct worker_params *wp, /* flush the distributor */ rte_distributor_flush(d); - rte_delay_us(10000); + while (!__atomic_load_n(&zero_sleep, __ATOMIC_ACQUIRE)) + rte_distributor_flush(d); zero_quit = 0; + + while (__atomic_load_n(&zero_sleep, __ATOMIC_ACQUIRE)) + rte_delay_us(100); + for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, __atomic_load_n(&worker_stats[i].handled_packets, @@ -615,6 +628,8 @@ quit_workers(struct worker_params *wp, struct rte_mempool *p) quit = 0; worker_idx = 0; zero_idx = RTE_MAX_LCORE; + zero_quit = 0; + zero_sleep = 0; } static int From patchwork Fri Oct 9 22:01:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80241 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 61589A04BC; Sat, 10 Oct 2020 00:06:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 48ABA1D6AA; Sat, 10 Oct 2020 00:03:19 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id C995B1D66D for ; Sat, 10 Oct 2020 00:03:09 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220259euoutp02f6079d062efdca8e51f18f64d9831957~8cg-OSU_-2500425004euoutp02F for ; Fri, 9 Oct 2020 22:02:59 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220259euoutp02f6079d062efdca8e51f18f64d9831957~8cg-OSU_-2500425004euoutp02F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280979; bh=1Px3vlBp24aI4tqkdpolnyQ52wyBpb+Nn7D22MHq1OM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P6vbOMwg4Y5aSUgVY4fGzMfDftE8wYNZHM8toKo5SdguoosfUf3lriOrclSRH76Ic 3uwPiFYCSUMintIt6jZg5SzoU24zVj6AS0sU4mivESeSszGaUlR/BOl0P3fCb4IBKu zTEAVMf8sxJdDrmY+PWqeo71O10T4/QsmpzEQeTo= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20201009220253eucas1p2201e6b8e48adf9c28731069640bafc44~8cg5qwIMI1628316283eucas1p2L; Fri, 9 Oct 2020 22:02:53 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 74.88.05997.D0ED08F5; Fri, 9 Oct 2020 23:02:53 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20201009220253eucas1p14078ab159186d2c26e787b3b2ed68062~8cg5MLI7p1072710727eucas1p1o; Fri, 9 Oct 2020 22:02:53 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220253eusmtrp22cce8f06f25b70cbb7d8c49f5b8e259d~8cg5LpjSh1642116421eusmtrp2k; Fri, 9 Oct 2020 22:02:53 +0000 (GMT) X-AuditID: cbfec7f4-65dff7000000176d-0a-5f80de0d8e6e Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id B4.5F.06017.D0ED08F5; Fri, 9 Oct 2020 23:02:53 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220252eusmtip1e33b15bf3bede05e0242bc829324b4cb~8cg4j7CSJ1990219902eusmtip1K; Fri, 9 Oct 2020 22:02:51 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:01:59 +0200 Message-Id: <20201009220202.20834-13-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrKIsWRmVeSWpSXmKPExsWy7djP87q89xriDV6dVrO4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoLpuU1JzMstQifbsEroztq2cw Fqzgqthy9zxbA+Mqji5GTg4JAROJm9cbmEBsIYEVjBLPJhV2MXIB2V8YJebvfcUKkfjMKHG7 txSm4UbrY6iG5YwSz68lQjR8YpTY2N0ClmATsJU4MvMrWLOIQJhEc/Neli5GDg5mAWeJJ1/Z QMLCAlYSf39sYgGxWQRUJc4umwVWzivgJtH6fysrxC55idUbDjCD2Jwg8V3LWEB2SQhcZ5OY eX8/G0SRi8S0vefYIWxhiVfHt0DZMhL/d85ngmjYxihx9fdPRghnP6PE9d4VUFXWEof//WaD uE5TYv0ufYiwo0TDj6esIGEJAT6JG28FQcLMQOakbdOZIcK8Eh1tQhDVehJPe6Yywqz9s/YJ C4TtIXGv+yEzJHyuMko0dS5hm8AoPwth2QJGxlWM4qmlxbnpqcVGeanlesWJucWleel6yfm5 mxiBkX/63/EvOxh3/Uk6xCjAwajEw9uQ3BAvxJpYVlyZe4hRgoNZSYTX6ezpOCHelMTKqtSi /Pii0pzU4kOM0hwsSuK8xotexgoJpCeWpGanphakFsFkmTg4pRoY5Xg2PL996Kx/zmJH1xjv oNTj9d2nsmfUfeJLE14e3nk132xiaeRt9l5uIR1unTntarzMycnmp3962YhzMWo+DkiY7305 4BnbKwGGlbZbqqb1dnbLdrswqH0R+Ta3dN/EwsT8uZlWkRle7bI8ii4PHx0Lfigzx5PvXOLt Ax2pJ26x/Tp7rF2JpTgj0VCLuag4EQAVA35k+AIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrOLMWRmVeSWpSXmKPExsVy+t/xu7q89xriDU7+YLa4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoPZui/NKSVIWM/OISW6VoQwsj PUNLCz0jE0s9Q2PzWCsjUyV9O5uU1JzMstQifbsEvYztq2cwFqzgqthy9zxbA+Mqji5GTg4J AROJG62PmboYuTiEBJYySrw7f4u5i5EDKCEj8eGSAESNsMSfa11sILaQwAdGiWeztUBsNgFb iSMzv7KClIsIhEmcWOkPEmYWcJfYsngqM4gtLGAl8ffHJhYQm0VAVeLsslmsIDavgJtE6/+t rBDj5SVWbzgAVs8JEt+1jAXinEZGiS0HXjJPYORbwMiwilEktbQ4Nz232EivODG3uDQvXS85 P3cTIzAQtx37uWUHY9e74EOMAhyMSjy8GokN8UKsiWXFlbmHGCU4mJVEeJ3Ono4T4k1JrKxK LcqPLyrNSS0+xGgKdNVEZinR5HxglOSVxBuaGppbWBqaG5sbm1koifN2CByMERJITyxJzU5N LUgtgulj4uCUamBMP3aNtcDXplJo2rHDsuuUF8Yc6bm//Gj9n6NrJ9Q/3Xrn3yyL9mL73N+n jON0kl7IebXsubxPVPuU1/WUiJ5/k5kSVb7JBkYf13GYE//Oq2yBK+9VD6OnQcVVnvkSN5O9 vVh+6HY0V0e2c83R3M+lP9/hks+BdWHrT+9ZHMGepio1b7Hq+39KLMUZiYZazEXFiQC6U9NZ WgIAAA== X-CMS-MailID: 20201009220253eucas1p14078ab159186d2c26e787b3b2ed68062 X-Msg-Generator: CA X-RootMTR: 20201009220253eucas1p14078ab159186d2c26e787b3b2ed68062 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220253eucas1p14078ab159186d2c26e787b3b2ed68062 References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 12/15] distributor: fix scalar matching X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Fix improper indexes while comparing tags. In the find_match_scalar() function: * j iterates over flow tags of following packets; * w iterates over backlog or in flight tags positions. Fixes: 775003ad2f96 ("distributor: add new burst-capable library") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- lib/librte_distributor/rte_distributor.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 9fd7dcab7..4bd23a990 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -261,13 +261,13 @@ find_match_scalar(struct rte_distributor *d, for (j = 0; j < RTE_DIST_BURST_SIZE ; j++) for (w = 0; w < RTE_DIST_BURST_SIZE; w++) - if (d->in_flight_tags[i][j] == data_ptr[w]) { + if (d->in_flight_tags[i][w] == data_ptr[j]) { output_ptr[j] = i+1; break; } for (j = 0; j < RTE_DIST_BURST_SIZE; j++) for (w = 0; w < RTE_DIST_BURST_SIZE; w++) - if (bl->tags[j] == data_ptr[w]) { + if (bl->tags[w] == data_ptr[j]) { output_ptr[j] = i+1; break; } From patchwork Fri Oct 9 22:02:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80242 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9AAEFA04BC; Sat, 10 Oct 2020 00:07:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3A7D41D6AF; Sat, 10 Oct 2020 00:03:21 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 72F001D66A for ; Sat, 10 Oct 2020 00:03:10 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220300euoutp020c1642af047a1bde88bd6d3ebbae504d~8cg-2V29X1684516845euoutp02y for ; Fri, 9 Oct 2020 22:03:00 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220300euoutp020c1642af047a1bde88bd6d3ebbae504d~8cg-2V29X1684516845euoutp02y DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280980; bh=H0SESRE59ofq6tEaPxg1A+VKjAU63AAfipdXXlkRUGc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ApxczXocZd1G/d6Ebr3BCY655eYg/r6UsUdf+IIv/s9rboh0v35LQMNyMdQDvfNX7 /jCk9p/tazuKdbEtQITiyRrW4d22xTsBLVB73aiX6Eg1G3vGKzJWpzUpQVmptEKSlD m/qSaJnk5I7zWPGil+VgeLR/HY3+3wmezVJG/EPM= Received: from eusmges1new.samsung.com (unknown [203.254.199.242]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20201009220254eucas1p1a85ddeb69360f58f057dcf40791e31c8~8cg6oLy1j1071010710eucas1p1f; Fri, 9 Oct 2020 22:02:54 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges1new.samsung.com (EUCPMTA) with SMTP id 00.87.06456.E0ED08F5; Fri, 9 Oct 2020 23:02:54 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20201009220253eucas1p2c0e27c3a495cb9603102b2cbf8a8f706~8cg5vnIMl1920419204eucas1p2W; Fri, 9 Oct 2020 22:02:53 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220253eusmtrp2d2846ef4ae5c5a300f850b59de7b2fb0~8cg5vCM7F2402824028eusmtrp2B; Fri, 9 Oct 2020 22:02:53 +0000 (GMT) X-AuditID: cbfec7f2-7efff70000001938-1b-5f80de0eb5fd Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id A4.E2.06314.D0ED08F5; Fri, 9 Oct 2020 23:02:53 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220253eusmtip14fbeb38d36f65331065bdf48c4a0bc98~8cg5PbH0R1334813348eusmtip10; Fri, 9 Oct 2020 22:02:53 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com Date: Sat, 10 Oct 2020 00:02:00 +0200 Message-Id: <20201009220202.20834-14-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrHIsWRmVeSWpSXmKPExsWy7djP87p89xriDZY907Tom/SRyeLdp+1M Fs961jE6MHv8WrCU1WPxnpdMHgff7WEKYI7isklJzcksSy3St0vgynj34hBbwWGtilVbTrM1 MG5V7GLk5JAQMJH42HeDpYuRi0NIYAWjxJn9F6CcL4wSP66+ZoVwPjNKNK+byATTMvlkLyNE YjmjxOF/L9hBEkICnxgl3jezgdhsArYSR2Z+ZQWxRQRUJf7t/wTWzCxgJPGyeyJzFyMHh7CA p8SPX2BnsACVNPzeDFbCK+Am8ez4L3aIXfISqzccYAaxOYHirbuWgV0nIXCETWJeSyvUQS4S Zx+chmoQlnh1fAuULSPxf+d8JoiGbYwSV3//ZIRw9jNKXO9dAVVlDfTBbzaQi5gFNCXW79KH CDtKnFm7A+xQCQE+iRtvBSHu55OYtG06VJhXoqNNCKJaT+Jpz1RGmLV/1j5hgbA9JPonvYaG 6FVGiWWvp7BOYJSfhbBsASPjKkbx1NLi3PTUYsO81HK94sTc4tK8dL3k/NxNjMBYP/3v+Kcd jF8vJR1iFOBgVOLhbUhuiBdiTSwrrsw9xCjBwawkwut09nScEG9KYmVValF+fFFpTmrxIUZp DhYlcV7jRS9jhQTSE0tSs1NTC1KLYLJMHJxSDYzVW0LPHS35PN/jvczq2/syu/+KLPr4P1pI fllSw66C62tb75neDDm2pNbxZIHh16cP9n693PopzcRaborF62ebf89borv6bdmbf7a1K7Xu Hzrl03Jixnz9JMc53wUYei88U3b4Iziz6r3GAoMjf4J8J25duMo69WDMnPvTNmzZz3Tyh9vf u2GCq5RYijMSDbWYi4oTAeSwmW/xAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrPLMWRmVeSWpSXmKPExsVy+t/xu7q89xriDQ51qVv0TfrIZPHu03Ym i2c96xgdmD1+LVjK6rF4z0smj4Pv9jAFMEfp2RTll5akKmTkF5fYKkUbWhjpGVpa6BmZWOoZ GpvHWhmZKunb2aSk5mSWpRbp2yXoZbx7cYit4LBWxaotp9kaGLcqdjFyckgImEhMPtnL2MXI xSEksJRRYsGZVcxdjBxACRmJD5cEIGqEJf5c62KDqPnAKPHn1msWkASbgK3EkZlfWUFsEQFV iX/7PzGB2MxAQ2/Pa2IDmSMs4Cnx4xfYLhagkobfm8FKeAXcJJ4d/8UOMV9eYvWGA8wgNidQ vHXXMhaIXY2MElsOvGSewMi3gJFhFaNIamlxbnpusaFecWJucWleul5yfu4mRmDgbTv2c/MO xksbgw8xCnAwKvHwaiQ2xAuxJpYVV+YeYpTgYFYS4XU6ezpOiDclsbIqtSg/vqg0J7X4EKMp 0FUTmaVEk/OBUZFXEm9oamhuYWlobmxubGahJM7bIXAwRkggPbEkNTs1tSC1CKaPiYNTqoFx Vlfmuuufj9srV8R4LbyVu4x965HLeu4vIyQ9bkz6c3zrf8MLSwN1L7nu4mk/vOjoTZ7/l9mO lm15/+n0/JdBC9t1t3aLfA74ftrKrVhuVZ3IoptFMZerv8VlxD46Z9jouf/yCvaKyE+xVcpa 818Y2e9c4nKFvSbZTGeWa5vGf8toyUfvgln9lFiKMxINtZiLihMBVXDKhVICAAA= X-CMS-MailID: 20201009220253eucas1p2c0e27c3a495cb9603102b2cbf8a8f706 X-Msg-Generator: CA X-RootMTR: 20201009220253eucas1p2c0e27c3a495cb9603102b2cbf8a8f706 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220253eucas1p2c0e27c3a495cb9603102b2cbf8a8f706 References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 13/15] test/distributor: add test with packets marking X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" All of the former tests analyzed only statistics of packets processed by all workers. The new test verifies also if packets are processed on workers as expected. Every packets processed by the worker is marked and analyzed after it is returned to distributor. This test allows finding issues in matching algorithms. Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- app/test/test_distributor.c | 141 ++++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index e8dd75078..4fc10b3cc 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -542,6 +542,141 @@ test_flush_with_worker_shutdown(struct worker_params *wp, return 0; } +static int +handle_and_mark_work(void *arg) +{ + struct rte_mbuf *buf[8] __rte_cache_aligned; + struct worker_params *wp = arg; + struct rte_distributor *db = wp->dist; + unsigned int num, i; + unsigned int id = __atomic_fetch_add(&worker_idx, 1, __ATOMIC_RELAXED); + num = rte_distributor_get_pkt(db, id, buf, NULL, 0); + while (!quit) { + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_RELAXED); + for (i = 0; i < num; i++) + buf[i]->udata64 += id + 1; + num = rte_distributor_get_pkt(db, id, + buf, buf, num); + } + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_RELAXED); + rte_distributor_return_pkt(db, id, buf, num); + return 0; +} + +/* sanity_mark_test sends packets to workers which mark them. + * Every packet has also encoded sequence number. + * The returned packets are sorted and verified if they were handled + * by proper workers. + */ +static int +sanity_mark_test(struct worker_params *wp, struct rte_mempool *p) +{ + const unsigned int buf_count = 24; + const unsigned int burst = 8; + const unsigned int shift = 12; + const unsigned int seq_shift = 10; + + struct rte_distributor *db = wp->dist; + struct rte_mbuf *bufs[buf_count]; + struct rte_mbuf *returns[buf_count]; + unsigned int i, count, id; + unsigned int sorted[buf_count], seq; + unsigned int failed = 0; + + printf("=== Marked packets test ===\n"); + clear_packet_count(); + if (rte_mempool_get_bulk(p, (void *)bufs, buf_count) != 0) { + printf("line %d: Error getting mbufs from pool\n", __LINE__); + return -1; + } + + /* bufs' hashes will be like these below, but shifted left. + * The shifting is for avoiding collisions with backlogs + * and in-flight tags left by previous tests. + * [1, 1, 1, 1, 1, 1, 1, 1 + * 1, 1, 1, 1, 2, 2, 2, 2 + * 2, 2, 2, 2, 1, 1, 1, 1] + */ + for (i = 0; i < burst; i++) { + bufs[0 * burst + i]->hash.usr = 1 << shift; + bufs[1 * burst + i]->hash.usr = ((i < burst / 2) ? 1 : 2) + << shift; + bufs[2 * burst + i]->hash.usr = ((i < burst / 2) ? 2 : 1) + << shift; + } + /* Assign a sequence number to each packet. The sequence is shifted, + * so that lower bits of the udate64 will hold mark from worker. + */ + for (i = 0; i < buf_count; i++) + bufs[i]->udata64 = i << seq_shift; + + count = 0; + for (i = 0; i < buf_count/burst; i++) { + rte_distributor_process(db, &bufs[i * burst], burst); + count += rte_distributor_returned_pkts(db, &returns[count], + buf_count - count); + } + + do { + rte_distributor_flush(db); + count += rte_distributor_returned_pkts(db, &returns[count], + buf_count - count); + } while (count < buf_count); + + for (i = 0; i < rte_lcore_count() - 1; i++) + printf("Worker %u handled %u packets\n", i, + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_RELAXED)); + + /* Sort returned packets by sent order (sequence numbers). */ + for (i = 0; i < buf_count; i++) { + seq = returns[i]->udata64 >> seq_shift; + id = returns[i]->udata64 - (seq << seq_shift); + sorted[seq] = id; + } + + /* Verify that packets [0-11] and [20-23] were processed + * by the same worker + */ + for (i = 1; i < 12; i++) { + if (sorted[i] != sorted[0]) { + printf("Packet number %u processed by worker %u," + " but should be processes by worker %u\n", + i, sorted[i], sorted[0]); + failed = 1; + } + } + for (i = 20; i < 24; i++) { + if (sorted[i] != sorted[0]) { + printf("Packet number %u processed by worker %u," + " but should be processes by worker %u\n", + i, sorted[i], sorted[0]); + failed = 1; + } + } + /* And verify that packets [12-19] were processed + * by the another worker + */ + for (i = 13; i < 20; i++) { + if (sorted[i] != sorted[12]) { + printf("Packet number %u processed by worker %u," + " but should be processes by worker %u\n", + i, sorted[i], sorted[12]); + failed = 1; + } + } + + rte_mempool_put_bulk(p, (void *)bufs, buf_count); + + if (failed) + return -1; + + printf("Marked packets test passed\n"); + return 0; +} + static int test_error_distributor_create_name(void) { @@ -726,6 +861,12 @@ test_distributor(void) goto err; quit_workers(&worker_params, p); + rte_eal_mp_remote_launch(handle_and_mark_work, + &worker_params, SKIP_MASTER); + if (sanity_mark_test(&worker_params, p) < 0) + goto err; + quit_workers(&worker_params, p); + } else { printf("Too few cores to run worker shutdown test\n"); } From patchwork Fri Oct 9 22:02:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80243 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1E90CA04BC; Sat, 10 Oct 2020 00:07:35 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D528E1D6B7; Sat, 10 Oct 2020 00:03:22 +0200 (CEST) Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id 7B1901D621 for ; Sat, 10 Oct 2020 00:03:11 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220300euoutp010ddd3bf1a59e8024b1bc4e6293cc1f6b~8cg-203HE2274222742euoutp01J for ; Fri, 9 Oct 2020 22:03:00 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20201009220300euoutp010ddd3bf1a59e8024b1bc4e6293cc1f6b~8cg-203HE2274222742euoutp01J DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280980; bh=v4SHqhfr0CgQSyQySuuoqmQz3Fqbg8VL6UkTHS7IUM8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gqzY0e5xCZlNV3yRk9pn0dIhT1lAnL/xxjtuZhrtwU1fj1l3nN/guCjxFcpT4PieB dJj9qoxgBIOJHzTpV2jntVz4dAK9ns1mrYohebAdRBxcECInCREPKiFEJQFbQoSaLQ Y65UT3Iem20CwsVbkwM2LxDAGHikq8k3LI9JJIWA= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20201009220254eucas1p23088f6944dac82ccbd8e8a99c0861686~8cg6rNjJ81919219192eucas1p2X; Fri, 9 Oct 2020 22:02:54 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id 9F.E2.06318.E0ED08F5; Fri, 9 Oct 2020 23:02:54 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20201009220254eucas1p187bad9a066f00ee4c05ec6ca7fb4decd~8cg6TuLFH0869608696eucas1p1e; Fri, 9 Oct 2020 22:02:54 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220254eusmtrp2d676157a36dfbba61d71992d371f9437~8cg6TMyfe2402824028eusmtrp2C; Fri, 9 Oct 2020 22:02:54 +0000 (GMT) X-AuditID: cbfec7f5-38bff700000018ae-b3-5f80de0ea37f Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 15.5F.06017.E0ED08F5; Fri, 9 Oct 2020 23:02:54 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220253eusmtip16605958d3be8e27c1c49e660b89cea1a~8cg5xP1em1990219902eusmtip1L; Fri, 9 Oct 2020 22:02:53 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:02:01 +0200 Message-Id: <20201009220202.20834-15-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGIsWRmVeSWpSXmKPExsWy7djP87p89xriDSZM1LK4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoLpuU1JzMstQifbsEroxpT3qY Cm4JVZy94NnAuJ2/i5GTQ0LAROLfu0usXYxcHEICKxglNjbeZQRJCAl8YZSYdtEKIvGZUaJp 4QZGmI5L97cwQySWM0q8eXIdyvnEKLF91XY2kCo2AVuJIzO/soLYIgJhEs3Ne1m6GDk4mAWc JZ58BSsRFnCVWPNgK9hQFgFViQmNx9hBbF4BN4ndbUvZIZbJS6zecIAZxOYEirfuWsYCsktC 4DKbxJmPv9ggilwkJtydwwxhC0u8Or4FqllG4v/O+UwQDdsYJa7+/skI4exnlLjeuwKqylri 8L/fbBDXaUqs36UPEXaUWLl9FjNIWEKAT+LGW0GQMDOQOWnbdKgwr0RHmxBEtZ7E056pjDBr /6x9wgJhe0i03m5ih4TPVUaJFV82s05glJ+FsGwBI+MqRvHU0uLc9NRi47zUcr3ixNzi0rx0 veT83E2MwNg//e/41x2M+/4kHWIU4GBU4uFtSG6IF2JNLCuuzD3EKMHBrCTC63T2dJwQb0pi ZVVqUX58UWlOavEhRmkOFiVxXuNFL2OFBNITS1KzU1MLUotgskwcnFINjBnn9fj9Z8efvPNz e1Pf9B8hTJGFJl9cf99b8jGhSlRY6PSWnRKCihNEvkkHfbzaLjjr2ezsiVwnXNpPf1yi/fXz 9V8Crxi3PQjNZGHIDbkr9tHx/Lt8TnnBO4qNTpn2aXyLMtPZ/afftWbSuPu34KF1zU9zxgx1 gXu365znWF6oePbW3t54mRJLcUaioRZzUXEiAEO0PSL5AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrOLMWRmVeSWpSXmKPExsVy+t/xu7p89xriDU6vEbW4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoPZui/NKSVIWM/OISW6VoQwsj PUNLCz0jE0s9Q2PzWCsjUyV9O5uU1JzMstQifbsEvYxpT3qYCm4JVZy94NnAuJ2/i5GTQ0LA ROLS/S3MXYxcHEICSxklPm9bydLFyAGUkJH4cEkAokZY4s+1LjaImg+MEjNn9LOBJNgEbCWO zPzKClIvIhAmcWKlP0iYWcBdYsviqcwgtrCAq8SaB1sZQWwWAVWJCY3H2EFsXgE3id1tS9kh 5stLrN5wAKyeEyjeumsZC8SuRkaJLQdeMk9g5FvAyLCKUSS1tDg3PbfYSK84Mbe4NC9dLzk/ dxMjMBC3Hfu5ZQdj17vgQ4wCHIxKPLwaiQ3xQqyJZcWVuYcYJTiYlUR4nc6ejhPiTUmsrEot yo8vKs1JLT7EaAp01URmKdHkfGCU5JXEG5oamltYGpobmxubWSiJ83YIHIwREkhPLEnNTk0t SC2C6WPi4JRqYFxxO2bTPy9B3z/HOiWK2S2P5N7jP16zunv/jmvVsWnvfHTXSsUmXWvITnF8 +9fytvnCJJ6w73s4whPVbyg3Gn+tz3oY0i1XzLvEY3FsceoJjltC8RktTL4S9ZUZ+ryOL/2e ++glbY+2FfW538gQ9t15hl7Y2e6keYriGSeKymVTq3YFRu1XYinOSDTUYi4qTgQA/p4gDFoC AAA= X-CMS-MailID: 20201009220254eucas1p187bad9a066f00ee4c05ec6ca7fb4decd X-Msg-Generator: CA X-RootMTR: 20201009220254eucas1p187bad9a066f00ee4c05ec6ca7fb4decd X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220254eucas1p187bad9a066f00ee4c05ec6ca7fb4decd References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 14/15] distributor: fix flushing in flight packets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" rte_distributor_flush() is using total_outstanding() function to calculate if it should still wait for processing packets. However in burst mode only backlog packets were counted. This patch fixes that issue by counting also in flight packets. There are also sum fixes to properly keep count of in flight packets for each worker in bufs[].count. Fixes: 775003ad2f96 ("distributor: add new burst-capable library") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- lib/librte_distributor/rte_distributor.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 4bd23a990..2478de3b7 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -467,6 +467,7 @@ rte_distributor_process(struct rte_distributor *d, /* Sync with worker on GET_BUF flag. */ if (__atomic_load_n(&(d->bufs[wid].bufptr64[0]), __ATOMIC_ACQUIRE) & RTE_DISTRIB_GET_BUF) { + d->bufs[wid].count = 0; release(d, wid); handle_returns(d, wid); } @@ -481,11 +482,6 @@ rte_distributor_process(struct rte_distributor *d, uint16_t matches[RTE_DIST_BURST_SIZE]; unsigned int pkts; - /* Sync with worker on GET_BUF flag. */ - if (__atomic_load_n(&(d->bufs[wkr].bufptr64[0]), - __ATOMIC_ACQUIRE) & RTE_DISTRIB_GET_BUF) - d->bufs[wkr].count = 0; - if ((num_mbufs - next_idx) < RTE_DIST_BURST_SIZE) pkts = num_mbufs - next_idx; else @@ -605,8 +601,10 @@ rte_distributor_process(struct rte_distributor *d, for (wid = 0 ; wid < d->num_workers; wid++) /* Sync with worker on GET_BUF flag. */ if ((__atomic_load_n(&(d->bufs[wid].bufptr64[0]), - __ATOMIC_ACQUIRE) & RTE_DISTRIB_GET_BUF)) + __ATOMIC_ACQUIRE) & RTE_DISTRIB_GET_BUF)) { + d->bufs[wid].count = 0; release(d, wid); + } return num_mbufs; } @@ -649,7 +647,7 @@ total_outstanding(const struct rte_distributor *d) unsigned int wkr, total_outstanding = 0; for (wkr = 0; wkr < d->num_workers; wkr++) - total_outstanding += d->backlog[wkr].count; + total_outstanding += d->backlog[wkr].count + d->bufs[wkr].count; return total_outstanding; } From patchwork Fri Oct 9 22:02:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 80244 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1AAF5A04BC; Sat, 10 Oct 2020 00:07:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6ACCF1D6BB; Sat, 10 Oct 2020 00:03:24 +0200 (CEST) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 9C5831D666 for ; Sat, 10 Oct 2020 00:03:11 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20201009220301euoutp027d23806e939eafe058ee9177a5c483de~8chA75nIa1685816858euoutp02u for ; Fri, 9 Oct 2020 22:03:01 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20201009220301euoutp027d23806e939eafe058ee9177a5c483de~8chA75nIa1685816858euoutp02u DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1602280981; bh=OZZGi7agIbDWlz1FwjrRd3zGTVW3Tz6nh33nggujM50=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aowvoolaK+91jLQg0a7G+EjSgHL6K9KPtLwWE1t01SbLJgzIjg6QqZFuJ86hfOY/Y zOB1IiLlDIVSw7xMyqD27lO8H8L5g2h25dSXfiJQFKqQNj5Jl4KNGX2SDWwsfkXLdS 46U4EWHeI/gtqCboRsaMLU+lIFu8xVH0Rw267E1Q= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20201009220256eucas1p1853958cb8baf14239240bece909dc4cd~8cg7-w7e11072710727eucas1p1p; Fri, 9 Oct 2020 22:02:56 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id D4.88.05997.01ED08F5; Fri, 9 Oct 2020 23:02:56 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20201009220255eucas1p1e7a286684291e586ebb22cb0a2117e50~8cg7JljiN0993409934eucas1p1e; Fri, 9 Oct 2020 22:02:55 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20201009220255eusmtrp24c4cd6f868a26364e0f12719325ea3f5~8cg7JEZYH2402824028eusmtrp2D; Fri, 9 Oct 2020 22:02:55 +0000 (GMT) X-AuditID: cbfec7f4-677ff7000000176d-0d-5f80de10ddaf Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id E4.E2.06314.F0ED08F5; Fri, 9 Oct 2020 23:02:55 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20201009220254eusmtip1210c1f595e584cd4e2357b5edd01cbf3~8cg6cUmc01334813348eusmtip11; Fri, 9 Oct 2020 22:02:54 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Sat, 10 Oct 2020 00:02:02 +0200 Message-Id: <20201009220202.20834-16-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201009220202.20834-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrKIsWRmVeSWpSXmKPExsWy7djPc7oC9xriDQ5P4bS4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoLpuU1JzMstQifbsEroz9L7ay FRxhr5i8bzF7A+NOti5GDg4JAROJnsVpXYxcHEICKxgldkw+ywThfAFypqyHcj4DOZ/fsHYx coJ1nFuzlw0isZxRYvLHo6wQzidGib7bE1hAqtgEbCWOzPwK1iEiECbR3LyXBWQfs4CzxJOv bCBhYQEniUPdfWA2i4CqxLXVB8BO4hVwk5g1qQBil7zE6g0HmEFsTqBw665lLCCrJATOsEk0 7HrMBFHkItH87BsjhC0s8er4FnYIW0bi/875TBAN2xglrv7+yQjh7GeUuN67AqrKWuLwv99s EMdpSqzfpQ8RdpSYvHYxOySM+CRuvBUECTMDmZO2TWeGCPNKdLQJQVTrSTztmcoIs/bP2ics ECUeEtt/q0BC5yqjxIzJD1knMMrPQti1gJFxFaN4amlxbnpqsVFearlecWJucWleul5yfu4m RmDkn/53/MsOxl1/kg4xCnAwKvHwNiQ3xAuxJpYVV+YeYpTgYFYS4XU6ezpOiDclsbIqtSg/ vqg0J7X4EKM0B4uSOK/xopexQgLpiSWp2ampBalFMFkmDk6pBsbNXo0eSSmTV0/y/qrI2uN5 ZOsh8dSgDKfdU+VKz8XubzjCKimWLXxgaXTJoQjxurovu86/O/RqGnOmDv+LfSdDO0/dWLPn 7Zddtx4eC5u+LWjzkTb/r7UXYpPb/h4rFOYOeJ3V//6ust05saUrGbknzXa07T3T+0qxyn4N 24f+rJPnm6cKiDkqsRRnJBpqMRcVJwIAf8YKDvgCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrOLMWRmVeSWpSXmKPExsVy+t/xu7r89xriDX7u47W4screom/SRyaL d5+2M1k861nHaPGv4w+7A6vHrwVLWT0W73nJ5HHw3R6mAOYoPZui/NKSVIWM/OISW6VoQwsj PUNLCz0jE0s9Q2PzWCsjUyV9O5uU1JzMstQifbsEvYz9L7ayFRxhr5i8bzF7A+NOti5GTg4J AROJc2v2AtlcHEICSxkl7sxfA+RwACVkJD5cEoCoEZb4c60LquYDo8TsHytZQBJsArYSR2Z+ ZQWpFxEIkzix0h8kzCzgLrFl8VRmEFtYwEniUHcf2C4WAVWJa6sPgI3nFXCTmDWpAGK8vMTq DQfAyjmBwq27lrFArGpklNhy4CXzBEa+BYwMqxhFUkuLc9Nziw31ihNzi0vz0vWS83M3MQID cduxn5t3MF7aGHyIUYCDUYmHVyOxIV6INbGsuDL3EKMEB7OSCK/T2dNxQrwpiZVVqUX58UWl OanFhxhNgY6ayCwlmpwPjJK8knhDU0NzC0tDc2NzYzMLJXHeDoGDMUIC6YklqdmpqQWpRTB9 TBycUg2MU1RiJ/z41vooI+32Wk2L189kQiRd3D0MmcyvxnTrb7bbWHn3j84k6yNvQ8yDAm5O mHqdW/B6u2T7nvvfXsx5FmR4LP+E1wOJ8CX5L471OxTM7ow/yhUx6fACQ3WpPfHMdbuU5jpf fnlY8glHpFJ8zyyDGzfmn9kkYzk3ZuX9uetnz2kz+8KhpcRSnJFoqMVcVJwIAIHe5iNaAgAA X-CMS-MailID: 20201009220255eucas1p1e7a286684291e586ebb22cb0a2117e50 X-Msg-Generator: CA X-RootMTR: 20201009220255eucas1p1e7a286684291e586ebb22cb0a2117e50 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20201009220255eucas1p1e7a286684291e586ebb22cb0a2117e50 References: <20201008052323.11547-1-l.wojciechow@partner.samsung.com> <20201009220202.20834-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v6 15/15] distributor: fix clearing returns buffer X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The patch clears distributors returns buffer in clear_returns() by setting start and count to 0. Fixes: 775003ad2f96 ("distributor: add new burst-capable library") Cc: david.hunt@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- lib/librte_distributor/rte_distributor.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 2478de3b7..57240304a 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -704,6 +704,8 @@ rte_distributor_clear_returns(struct rte_distributor *d) /* Sync with worker. Release retptrs. */ __atomic_store_n(&(d->bufs[wkr].retptr64[0]), 0, __ATOMIC_RELEASE); + + d->returns.start = d->returns.count = 0; } /* creates a distributor instance */