From patchwork Thu Apr 26 08:13:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: junjie.j.chen@intel.com X-Patchwork-Id: 38993 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1314C6CD6; Thu, 26 Apr 2018 10:17:41 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id E86986CC9 for ; Thu, 26 Apr 2018 10:17:36 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Apr 2018 01:17:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,330,1520924400"; d="scan'208";a="219505863" Received: from cjj-s2600wft.sh.intel.com ([10.67.111.131]) by orsmga005.jf.intel.com with ESMTP; 26 Apr 2018 01:17:34 -0700 From: Junjie Chen To: maxime.coquelin@redhat.com, jianfeng.tan@intel.com, john.mcnamara@intel.com, marko.kovacevic@intel.com Cc: dev@dpdk.org, Junjie Chen Date: Thu, 26 Apr 2018 16:13:20 +0800 Message-Id: <1524730400-252654-1-git-send-email-junjie.j.chen@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [dpdk-dev] [PATCH] doc/vhost: update zero copy perfromance tip X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In VM2NIC case zero copy may need some tuning to get best performance. This patch describes the zero copy starved case and provides a tuning tip. Signed-off-by: Junjie Chen Acked-by: Maxime Coquelin --- doc/guides/prog_guide/vhost_lib.rst | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst index ef9a178..92dcdb5 100644 --- a/doc/guides/prog_guide/vhost_lib.rst +++ b/doc/guides/prog_guide/vhost_lib.rst @@ -65,14 +65,18 @@ The following is an overview of some key Vhost API functions: * zero copy is really good for VM2VM case. For iperf between two VMs, the boost could be above 70% (when TSO is enableld). - * for VM2NIC case, the ``nb_tx_desc`` has to be small enough: <= 64 if virtio - indirect feature is not enabled and <= 128 if it is enabled. - - This is because when dequeue zero copy is enabled, guest Tx used vring will - be updated only when corresponding mbuf is freed. Thus, the nb_tx_desc - has to be small enough so that the PMD driver will run out of available - Tx descriptors and free mbufs timely. Otherwise, guest Tx vring would be - starved. + * For zero copy in VM2NIC case, guest Tx used vring may be starved if the + PMD driver consume the mbuf but not release them timely. + + For example, i40e driver has an optimization to maximum NIC pipeline which + postpones returning transmitted mbuf until only tx_free_threshold free + descs left. The virtio TX used ring will be starved if the formula + (num_i40e_tx_desc - num_virtio_tx_desc > tx_free_threshold) is true, since + i40e will not return back mbuf. + + A performance tip for tuning zero copy in VM2NIC case is to adjust the + frequency of mbuf free (i.e. adjust tx_free_threshold of i40e driver) to + balance consumer and producer. * Guest memory should be backended with huge pages to achieve better performance. Using 1G page size is the best.