From patchwork Mon Apr 26 16:33:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislaw Kardach X-Patchwork-Id: 92185 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 28EF4A0548; Mon, 26 Apr 2021 18:33:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9C1D241110; Mon, 26 Apr 2021 18:33:52 +0200 (CEST) Received: from mail-lj1-f180.google.com (mail-lj1-f180.google.com [209.85.208.180]) by mails.dpdk.org (Postfix) with ESMTP id C2F5C41104 for ; Mon, 26 Apr 2021 18:33:51 +0200 (CEST) Received: by mail-lj1-f180.google.com with SMTP id b38so23692674ljf.5 for ; Mon, 26 Apr 2021 09:33:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=LeqeWJNw9pB2LQwGRgR99CfhuH62rRZ79qXJnhkf3eE=; b=CJoZqJ7kREiAdHtYsLm3ATSW3CHxc3wTxRliP2y44PKCLWg/t7xqNYVIgQM9410LzZ lwT01iMfUnIl59MiSmR6mf+ddSau2FL3EAYvvXUfXWIgH7APE/d73icB2236VG3hhuF+ ELPcSkdwrnq082A+ZQ45tIz48LAvord/sWv11MiwX3u8bZ6l/YNIrbucXPuXLyiH5weQ lwF513Fw0WIHzG4fyCHNGXIgLOjzAnmE5++8T2swVwFJ3GOqfocSxaa3Tm3mzJOvi/Ov NNTxksj+Zt7o4eFsHM/jFk3W3m29H/mkmZkqR+Qy8ZFU3WM7wsPbWxrALBIibTdCtv6h +9nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=LeqeWJNw9pB2LQwGRgR99CfhuH62rRZ79qXJnhkf3eE=; b=I/WcYBEPPVx7hUjRTZXpixo6p74bXZzttp79hG2kZtHh9nZ/S00p3+dH9PEVuXP0Uv JiilEFdtXkBiPNFDnGWao+Jb6Y0QBNHmReJxdqIlXIBa31xVyY6Ue63buzYyA+obVd5l H+uqzjxWIHh1UQ8CCqUiFqGZn4Zp7TST/SJX4sjHD4EzEniN6fK+7yQmQZFbKZ/lHTxQ 4vh1gv7yCujdbdkQB6dEjByIZR6VZiPQ1MmPeEUM3Th322Ubx+ItdVMYk6NTjT6xQkD3 kB+6a98HoPMd1T8CkMHah1X/ygGDujks+kXUVQ+Y4qhXxI10S1c3Is1tutQ5V5vx886P SEYw== X-Gm-Message-State: AOAM530rEI1tMFVQtCvDMRpD2Yeirp5uMtc5MkJxKECmqkpvdTuiqBOK RY47GvW1HpV13ApfvIt3xqSPug== X-Google-Smtp-Source: ABdhPJyW/wBZCAaqct/Vo07XIKAGNfsGX83AnTlxufye+caTmmxnqlIgyCE+MLj4/VPZCmIIbD9USw== X-Received: by 2002:a2e:bf19:: with SMTP id c25mr13788165ljr.201.1619454831148; Mon, 26 Apr 2021 09:33:51 -0700 (PDT) Received: from toster.int.semihalf.com (89-73-146-138.dynamic.chello.pl. [89.73.146.138]) by smtp.gmail.com with ESMTPSA id f11sm52692ljo.57.2021.04.26.09.33.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Apr 2021 09:33:50 -0700 (PDT) From: Stanislaw Kardach To: David Hunt , dev@dpdk.org Cc: upstream@semihalf.com, Stanislaw Kardach , l.wojciechow@partner.samsung.com, David Marchand Date: Mon, 26 Apr 2021 18:33:10 +0200 Message-Id: <20210426163310.1043438-1-kda@semihalf.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 Subject: [dpdk-dev] [RFC PATCH] test/distributor: fix burst flush on worker quit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" While working on RISC-V port I have encountered a situation where worker threads get stuck in the rte_distributor_return_pkt() function in the burst test. After investigation some of the threads enter this function with flag RTE_DISTRIB_GET_BUF set in the d->retptr64[0]. At the same time main thread has already passed rte_distributor_process() so nobody will clear this flag and hence workers can't return. What I've noticed is that adding a flush just after the last _process(), similarly to how quit_workers() function is written in the test_distributor.c fixes the issue. Additionally the issue disappears when I remove the rdtsc delay code inside the rte_distributor_request_pkt(). However I can't get this to reproduce on x86 (even with SIMD forced off) and with artificial delays, which is why I wonder whether I'm not actually hiding some other issue. Looking at the implementation of the distributor, it is based on __atomic_* builtins and the only platform related bit in the fast-path is the rte_rdtsc() and rte_pause(). There may be some issues in the toolchain (I've tried so far with the Ubuntu one - 10.2.0-8ubuntu1). I should add that all unit tests for distributor are passing so either there's some coverage corner case or the implementation works on RISC-V. As for RDTSC I'm using a sleep-stable time counter with 1MHz frequency and switching to high resolution cycle counter also removes the issue but that's the same as removing the rdtsc delay as mentioned above. I'd love to hear from You if this fix makes any sense. While modifying this test, I've also pulled in a fix from test_distributor.c which ensures that each thread gets his own wakeup packet as it's possible that when sending a burst of packets, they won't be spread over all the workers. Signed-off-by: Stanislaw Kardach Fixes: 7c3287a10535 ("test/distributor: add performance test for burst mode") Cc: david.hunt@intel.com Cc: l.wojciechow@partner.samsung.com Cc: David Marchand Acked-by: David Hunt --- app/test/test_distributor_perf.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/app/test/test_distributor_perf.c b/app/test/test_distributor_perf.c index b25f79a34..fdbeae6d2 100644 --- a/app/test/test_distributor_perf.c +++ b/app/test/test_distributor_perf.c @@ -188,13 +188,15 @@ quit_workers(struct rte_distributor *d, struct rte_mempool *p) rte_mempool_get_bulk(p, (void *)bufs, num_workers); quit = 1; - for (i = 0; i < num_workers; i++) + for (i = 0; i < num_workers; i++) { bufs[i]->hash.usr = i << 1; - rte_distributor_process(d, bufs, num_workers); + rte_distributor_process(d, &bufs[i], 1); + } rte_mempool_put_bulk(p, (void *)bufs, num_workers); rte_distributor_process(d, NULL, 0); + rte_distributor_flush(d); rte_eal_mp_wait_lcore(); quit = 0; worker_idx = 0;