From patchwork Thu Jan 14 07:08:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 86538 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 82704A0A02; Thu, 14 Jan 2021 08:09:59 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A511F140F3A; Thu, 14 Jan 2021 08:09:07 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 48788140F35; Thu, 14 Jan 2021 08:09:06 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CB23B1FB; Wed, 13 Jan 2021 23:09:05 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.127]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 257C63F719; Wed, 13 Jan 2021 23:09:02 -0800 (PST) From: Feifei Wang To: Jerin Jacob , Harry van Haaren Cc: dev@dpdk.org, Honnappa.Nagarahalli@arm.com, nd@arm.com, Feifei Wang , stable@dpdk.org, Ruifeng Wang Date: Thu, 14 Jan 2021 15:08:26 +0800 Message-Id: <20210114070830.707550-2-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210114070830.707550-1-feifei.wang2@arm.com> References: <20210114070830.707550-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v1 1/5] app/eventdev: fix SMP barrier bugs for perf test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch fixes RTE SMP barrier bugs for the perf test of eventdev. For the "perf_process_last_stage" function, wmb after storing processed_pkts should be moved before it. This is because the worker lcore should ensure it has really finished data processing, e.g. event stored into buffers, before the shared variables "w->processed_pkts"are stored. For the "perf_process_last_stage_latency", on the one hand, the wmb should be moved before storing into "w->processed_pkts". The reason is the same as above. But on the other hand, for "w->latency", wmb is unnecessary due to data dependency. Fixes: 2369f73329f8 ("app/testeventdev: add perf queue worker functions") Cc: jerinj@marvell.com Cc: stable@dpdk.org Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: Pavan Nikhilesh --- app/test-eventdev/test_perf_common.h | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/app/test-eventdev/test_perf_common.h b/app/test-eventdev/test_perf_common.h index ff9705df8..e7233e5a5 100644 --- a/app/test-eventdev/test_perf_common.h +++ b/app/test-eventdev/test_perf_common.h @@ -97,8 +97,13 @@ perf_process_last_stage(struct rte_mempool *const pool, void *bufs[], int const buf_sz, uint8_t count) { bufs[count++] = ev->event_ptr; - w->processed_pkts++; + + /* wmb here ensures event_prt is stored before + * updating the number of processed packets + * for worker lcores + */ rte_smp_wmb(); + w->processed_pkts++; if (unlikely(count == buf_sz)) { count = 0; @@ -116,6 +121,12 @@ perf_process_last_stage_latency(struct rte_mempool *const pool, struct perf_elt *const m = ev->event_ptr; bufs[count++] = ev->event_ptr; + + /* wmb here ensures event_prt is stored before + * updating the number of processed packets + * for worker lcores + */ + rte_smp_wmb(); w->processed_pkts++; if (unlikely(count == buf_sz)) { @@ -127,7 +138,6 @@ perf_process_last_stage_latency(struct rte_mempool *const pool, } w->latency += latency; - rte_smp_wmb(); return count; } From patchwork Thu Jan 14 07:08:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 86540 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 37509A0A02; Thu, 14 Jan 2021 08:10:24 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7C0E6140F4A; Thu, 14 Jan 2021 08:09:12 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 1657F140F46 for ; Thu, 14 Jan 2021 08:09:09 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 90EAA1FB; Wed, 13 Jan 2021 23:09:08 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.127]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6D6223F719; Wed, 13 Jan 2021 23:09:06 -0800 (PST) From: Feifei Wang To: Jerin Jacob Cc: dev@dpdk.org, Honnappa.Nagarahalli@arm.com, nd@arm.com, Feifei Wang , Ruifeng Wang Date: Thu, 14 Jan 2021 15:08:27 +0800 Message-Id: <20210114070830.707550-3-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210114070830.707550-1-feifei.wang2@arm.com> References: <20210114070830.707550-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v1 2/5] app/eventdev: remove unnecessary barriers for perf test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" For "processed_pkts" and "total_latency" functions, no operations should keep the order that being executed before loading "worker[i].processed_pkts". Thus rmb is unnecessary before loading. For "perf_launch_lcores" function, wmb after that the main lcore updates the variable "t->done", which represents the end of the test signal, is unnecessary. Because after the main lcore updates this siginal variable, it will jump out of the launch function loop, and wait other lcores stop or return error in the main function(evt_main.c). During this time, there is no important storing operation and thus no need for wmb. Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: Pavan Nikhilesh --- app/test-eventdev/test_perf_common.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/app/test-eventdev/test_perf_common.c b/app/test-eventdev/test_perf_common.c index 955edb752..34cded373 100644 --- a/app/test-eventdev/test_perf_common.c +++ b/app/test-eventdev/test_perf_common.c @@ -224,7 +224,6 @@ processed_pkts(struct test_perf *t) uint8_t i; uint64_t total = 0; - rte_smp_rmb(); for (i = 0; i < t->nb_workers; i++) total += t->worker[i].processed_pkts; @@ -237,7 +236,6 @@ total_latency(struct test_perf *t) uint8_t i; uint64_t total = 0; - rte_smp_rmb(); for (i = 0; i < t->nb_workers; i++) total += t->worker[i].latency; @@ -327,7 +325,6 @@ perf_launch_lcores(struct evt_test *test, struct evt_options *opt, opt->prod_type == EVT_PROD_TYPE_EVENT_TIMER_ADPTR) { t->done = true; - rte_smp_wmb(); break; } } @@ -341,7 +338,6 @@ perf_launch_lcores(struct evt_test *test, struct evt_options *opt, rte_event_dev_dump(opt->dev_id, stdout); evt_err("No schedules for seconds, deadlock"); t->done = true; - rte_smp_wmb(); break; } dead_lock_remaining = remaining; From patchwork Thu Jan 14 07:08:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 86541 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 48925A0A02; Thu, 14 Jan 2021 08:10:34 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B8873140F52; Thu, 14 Jan 2021 08:09:13 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 141C9140F38 for ; Thu, 14 Jan 2021 08:09:12 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9752A1FB; Wed, 13 Jan 2021 23:09:11 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.127]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 328263F719; Wed, 13 Jan 2021 23:09:08 -0800 (PST) From: Feifei Wang To: Jerin Jacob Cc: dev@dpdk.org, Honnappa.Nagarahalli@arm.com, nd@arm.com, Feifei Wang , Phil Yang , Ruifeng Wang Date: Thu, 14 Jan 2021 15:08:28 +0800 Message-Id: <20210114070830.707550-4-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210114070830.707550-1-feifei.wang2@arm.com> References: <20210114070830.707550-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v1 3/5] app/eventdev: replace wmb with thread fence for perf test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Simply replace rte_smp barrier with atomic threand fence. Signed-off-by: Phil Yang Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: Pavan Nikhilesh --- app/test-eventdev/test_perf_common.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/app/test-eventdev/test_perf_common.h b/app/test-eventdev/test_perf_common.h index e7233e5a5..9785dc3e2 100644 --- a/app/test-eventdev/test_perf_common.h +++ b/app/test-eventdev/test_perf_common.h @@ -98,11 +98,11 @@ perf_process_last_stage(struct rte_mempool *const pool, { bufs[count++] = ev->event_ptr; - /* wmb here ensures event_prt is stored before - * updating the number of processed packets - * for worker lcores + /* release fence here ensures event_prt is + * stored before updating the number of + * processed packets for worker lcores */ - rte_smp_wmb(); + rte_atomic_thread_fence(__ATOMIC_RELEASE); w->processed_pkts++; if (unlikely(count == buf_sz)) { @@ -122,11 +122,11 @@ perf_process_last_stage_latency(struct rte_mempool *const pool, bufs[count++] = ev->event_ptr; - /* wmb here ensures event_prt is stored before - * updating the number of processed packets - * for worker lcores + /* release fence here ensures event_prt is + * stored before updating the number of + * processed packets for worker lcores */ - rte_smp_wmb(); + rte_atomic_thread_fence(__ATOMIC_RELEASE); w->processed_pkts++; if (unlikely(count == buf_sz)) { From patchwork Thu Jan 14 07:08:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 86542 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8E92DA0A02; Thu, 14 Jan 2021 08:10:44 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DD836140F57; Thu, 14 Jan 2021 08:09:15 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id DDFF6140F56 for ; Thu, 14 Jan 2021 08:09:14 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 64CF71FB; Wed, 13 Jan 2021 23:09:14 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.127]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 37EEB3F719; Wed, 13 Jan 2021 23:09:11 -0800 (PST) From: Feifei Wang To: Jerin Jacob Cc: dev@dpdk.org, Honnappa.Nagarahalli@arm.com, nd@arm.com, Feifei Wang , Ruifeng Wang Date: Thu, 14 Jan 2021 15:08:29 +0800 Message-Id: <20210114070830.707550-5-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210114070830.707550-1-feifei.wang2@arm.com> References: <20210114070830.707550-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v1 4/5] app/eventdev: remove unnecessary barriers for pipeline test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" For "processed_pkts" function, no operations should keep the order that being executed before loading "worker[i].processed_pkts". Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: Pavan Nikhilesh --- app/test-eventdev/test_pipeline_common.c | 1 - 1 file changed, 1 deletion(-) diff --git a/app/test-eventdev/test_pipeline_common.c b/app/test-eventdev/test_pipeline_common.c index c67be48e9..b47d76743 100644 --- a/app/test-eventdev/test_pipeline_common.c +++ b/app/test-eventdev/test_pipeline_common.c @@ -44,7 +44,6 @@ processed_pkts(struct test_pipeline *t) uint8_t i; uint64_t total = 0; - rte_smp_rmb(); for (i = 0; i < t->nb_workers; i++) total += t->worker[i].processed_pkts; From patchwork Thu Jan 14 07:08:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 86543 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 530C1A0A02; Thu, 14 Jan 2021 08:10:56 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1EC6D140F41; Thu, 14 Jan 2021 08:09:19 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 9F544140EB0 for ; Thu, 14 Jan 2021 08:09:17 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 321451FB; Wed, 13 Jan 2021 23:09:17 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.127]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 05CA03F719; Wed, 13 Jan 2021 23:09:14 -0800 (PST) From: Feifei Wang To: Jerin Jacob Cc: dev@dpdk.org, Honnappa.Nagarahalli@arm.com, nd@arm.com, Feifei Wang , Ruifeng Wang Date: Thu, 14 Jan 2021 15:08:30 +0800 Message-Id: <20210114070830.707550-6-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210114070830.707550-1-feifei.wang2@arm.com> References: <20210114070830.707550-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v1 5/5] app/eventdev: remove unnecessary barriers for order test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" For the wmb in order_process_stage_1 and order_process_stage_invalid in the order test, they can be removed. This is because when the test results are wrong, the worker core writes 'true' to t->err. Then other worker cores, producer cores and the main core will load the 'error' index and stop testing. So, for the worker cores, no other storing operation needs to be guaranteed after this when errors happen. Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: Pavan Nikhilesh --- app/test-eventdev/test_order_common.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/app/test-eventdev/test_order_common.h b/app/test-eventdev/test_order_common.h index 5ef840493..cd9d6009e 100644 --- a/app/test-eventdev/test_order_common.h +++ b/app/test-eventdev/test_order_common.h @@ -104,7 +104,6 @@ order_process_stage_1(struct test_order *const t, flow, *order_mbuf_seqn(t, ev->mbuf), expected_flow_seq[flow]); t->err = true; - rte_smp_wmb(); } /* * Events from an atomic flow of an event queue can be scheduled only to @@ -123,7 +122,6 @@ order_process_stage_invalid(struct test_order *const t, { evt_err("invalid queue %d", ev->queue_id); t->err = true; - rte_smp_wmb(); } #define ORDER_WORKER_INIT\