From patchwork Thu Jun 25 09:59:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Dewar X-Patchwork-Id: 72172 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7AB66A0350; Thu, 25 Jun 2020 11:59:47 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CE1C1DE0; Thu, 25 Jun 2020 11:59:46 +0200 (CEST) Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by dpdk.org (Postfix) with ESMTP id A010BCF3 for ; Thu, 25 Jun 2020 11:59:45 +0200 (CEST) Received: by mail-wr1-f67.google.com with SMTP id f7so2192077wrw.1 for ; Thu, 25 Jun 2020 02:59:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qA5Ho8WB/xRyWjxBMyzh2sDTb2RjIqq5r7sFiP71CcA=; b=BEn8CvSZ4qxZhJliv4T3ZQpIlifAsZCPu90dqBba4niXn7lcNUXoFOjB5wSLS5Rtdn timd/bvMbwB04ONaqxP4qQfY4kcYiI8vUtE88+/kkS/P5yN+aldW375deImoEWlnOmA+ EENsPmNhmW9wCnQ8VB4VANospC50tl6zyW93eRxDynUqSDPsXzPv4aB5iUUfLGlbGuqm I91Y1Ru+ZoAWNCV5V10r28FAZQhYO2Yx5xlH3Jwplmql7/t0Uxp3GqDEsOTfcGQh7Lk7 Vu/dhHS2iEmQL5RskbC1q6oS0zdQ0/1Lr20vEh1Q+WBnJt/7QBo4iPmAjEXsjqQKwkoG BTMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qA5Ho8WB/xRyWjxBMyzh2sDTb2RjIqq5r7sFiP71CcA=; b=kTo9Tchfp3YXfJx5YxsezIJJHs/3pChBcx2Ua7FA2x862Wlm3NrCb4lQnNzFqzTUU4 injOrAuozopZeTjPaRp8Ef6R2KRXL4JA50nCDqHYEoZNT7As96dZCU2tiSV67voOqOTu wSYFwVh7iMudtyzeeUGcIn7Wjv97c8zh0GVnugFN5BgCXZkCiNqYU71nvZeWraTghUBj F8wFFtJTzw1j7lUd4cGx13q7PZXvAytA9LlrV8woxPc6zDy+BdJk6RF8B9JJ+9f3LqYC Oe9xkBLVTkqcJlTpH2O1URb7BINeSkLJOHweQawrvLNUr1W1h4Q11QpEiURWPCSduWQN JbgA== X-Gm-Message-State: AOAM530quSCJI2hgXO7oE7W6knTgenH29rexp7uQjXrlbhsyWvSVmtCp m2UQym6EobrhSfC13uGHEQ== X-Google-Smtp-Source: ABdhPJy3EwiAdm32XOUFpouGtIbCSshK3jypzTe4OSbeziSQkji1C3i9GIP5l4aV3pPbVtUvZJBysg== X-Received: by 2002:a5d:4a0c:: with SMTP id m12mr36352637wrq.156.1593079185435; Thu, 25 Jun 2020 02:59:45 -0700 (PDT) Received: from adewar-Precision-7520.fritz.box (82-69-103-115.dsl.in-addr.zen.co.uk. [82.69.103.115]) by smtp.gmail.com with ESMTPSA id h29sm32881381wrc.78.2020.06.25.02.59.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jun 2020 02:59:44 -0700 (PDT) From: alangordondewar@gmail.com X-Google-Original-From: alan.dewar@att.com To: cristian.dumitrescu@intel.com Cc: dev@dpdk.org, Alan Dewar Date: Thu, 25 Jun 2020 10:59:30 +0100 Message-Id: <20200625095930.18429-1-alan.dewar@att.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200416084821.12591-1-alan.dewar@att.com> References: <20200416084821.12591-1-alan.dewar@att.com> Subject: [dpdk-dev] [PATCH v2] sched: fix port time rounding error X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Alan Dewar The QoS scheduler works off port time that is computed from the number of CPU cycles that have elapsed since the last time the port was polled. It divides the number of elapsed cycles to calculate how many bytes can be sent, however this division can generate rounding errors, where some fraction of a byte sent may be lost. Lose enough of these fractional bytes and the QoS scheduler underperforms. The problem is worse with low bandwidths. To compensate for this rounding error this fix doesn't advance the port's time_cpu_cycles by the number of cycles that have elapsed, but by multiplying the computed number of bytes that can be sent (which has been rounded down) by number of cycles per byte. This will mean that port's time_cpu_cycles will lag behind the CPU cycles momentarily. At the next poll, the lag will be taken into account. v2: If the cycles value wraps (100 year+) reset the port's cpu cycle back to zero. Fixes: de3cfa2c98 ("sched: initial import") Signed-off-by: Alan Dewar Acked-by: Jasvinder Singh --- lib/librte_sched/rte_sched.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/lib/librte_sched/rte_sched.c b/lib/librte_sched/rte_sched.c index c0983ddda..7c022cd61 100644 --- a/lib/librte_sched/rte_sched.c +++ b/lib/librte_sched/rte_sched.c @@ -222,6 +222,7 @@ struct rte_sched_port { uint64_t time_cpu_bytes; /* Current CPU time measured in bytes */ uint64_t time; /* Current NIC TX time measured in bytes */ struct rte_reciprocal inv_cycles_per_byte; /* CPU cycles per byte */ + uint64_t cycles_per_byte; /* Grinders */ struct rte_mbuf **pkts_out; @@ -852,6 +853,7 @@ rte_sched_port_config(struct rte_sched_port_params *params) cycles_per_byte = (rte_get_tsc_hz() << RTE_SCHED_TIME_SHIFT) / params->rate; port->inv_cycles_per_byte = rte_reciprocal_value(cycles_per_byte); + port->cycles_per_byte = cycles_per_byte; /* Grinders */ port->pkts_out = NULL; @@ -2673,16 +2675,21 @@ static inline void rte_sched_port_time_resync(struct rte_sched_port *port) { uint64_t cycles = rte_get_tsc_cycles(); - uint64_t cycles_diff = cycles - port->time_cpu_cycles; + uint64_t cycles_diff; uint64_t bytes_diff; uint32_t i; + if (cycles < port->time_cpu_cycles) + port->time_cpu_cycles = 0; + + cycles_diff = cycles - port->time_cpu_cycles; /* Compute elapsed time in bytes */ bytes_diff = rte_reciprocal_divide(cycles_diff << RTE_SCHED_TIME_SHIFT, port->inv_cycles_per_byte); /* Advance port time */ - port->time_cpu_cycles = cycles; + port->time_cpu_cycles += + (bytes_diff * port->cycles_per_byte) >> RTE_SCHED_TIME_SHIFT; port->time_cpu_bytes += bytes_diff; if (port->time < port->time_cpu_bytes) port->time = port->time_cpu_bytes;