From patchwork Wed Jul 22 10:57:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Fu X-Patchwork-Id: 74616 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CA5F2A0526; Wed, 22 Jul 2020 13:00:11 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0DDF71C012; Wed, 22 Jul 2020 13:00:09 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 88FD01C010 for ; Wed, 22 Jul 2020 13:00:07 +0200 (CEST) IronPort-SDR: RVSdVnEPm1ZgMf23nbwbl7TgdDr/eCs+Tyjxv2gGlNuZ3oTx2h5JjHRw0C1mhzNz15ozRe5AoX Uqj1K9Gol+Vw== X-IronPort-AV: E=McAfee;i="6000,8403,9689"; a="148240087" X-IronPort-AV: E=Sophos;i="5.75,381,1589266800"; d="scan'208";a="148240087" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jul 2020 04:00:05 -0700 IronPort-SDR: +j1Ux1ZOlM0L0sw1R/fMEAjLpMMeGQBavrkXKXqncfhwtiaohmjnDGrYPlKRJFxMIWqxZnWgnQ gL0mRlOBMrAA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,381,1589266800"; d="scan'208";a="328178835" Received: from npg-dpdk-patrickfu-casc2.sh.intel.com ([10.67.119.92]) by orsmga007.jf.intel.com with ESMTP; 22 Jul 2020 04:00:04 -0700 From: patrick.fu@intel.com To: dev@dpdk.org, maxime.coquelin@redhat.com, chenbo.xia@intel.com Cc: Patrick Fu Date: Wed, 22 Jul 2020 18:57:40 +0800 Message-Id: <20200722105741.3421255-2-patrick.fu@intel.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20200722105741.3421255-1-patrick.fu@intel.com> References: <20200722105741.3421255-1-patrick.fu@intel.com> Subject: [dpdk-dev] [PATCH v1 1/2] doc: update guides for vhost async APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Patrick Fu Update vhost guides to document vhost async APIs Signed-off-by: Patrick Fu --- doc/guides/prog_guide/vhost_lib.rst | 86 ++++++++++++++++++++++++++--- 1 file changed, 77 insertions(+), 9 deletions(-) diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst index db921f922..cce8b6ae7 100644 --- a/doc/guides/prog_guide/vhost_lib.rst +++ b/doc/guides/prog_guide/vhost_lib.rst @@ -147,6 +147,21 @@ The following is an overview of some key Vhost API functions: It is disabled by default. + - ``RTE_VHOST_USER_ASYNC_COPY`` + + Asynchronous data path will be enabled when this flag is set. Async data + path allows applications to register async copy devices (typically + hardware DMA channels) to the vhost queues. Vhost leverages the copy + device registered to offload CPU from memory copy operations. A set of + async data path APIs are defined for DPDK applications to make use of + the async capability. Only packets enqueued/dequeued by async APIs are + processed through the async data path. + + Currently this feature is only implemented on split ring enqueue data + path. + + It is disabled by default. + * ``rte_vhost_driver_set_features(path, features)`` This function sets the feature bits the vhost-user driver supports. The @@ -235,6 +250,59 @@ The following is an overview of some key Vhost API functions: Enable or disable zero copy feature of the vhost crypto backend. +* ``rte_vhost_async_channel_register(vid, queue_id, features, ops)`` + + Register a vhost queue with async copy device channel. + Following device ``features`` must be specified together with the + registration: + + * ``async_inorder`` + + Async copy device can guarantee the ordering of copy completion + sequence. Copies are completed in the same order with that at + the submission time. + + Currently, only ``async_inorder`` capable device is supported by vhost. + + * ``async_threshold`` + + The copy length (in bytes) below which CPU copy will be used even if + applications call async vhost APIs to enqueue/dequeue data. + + Typical value is 512~1024 depending on the async device capability. + + Applications must provide following ``ops`` callbacks for vhost lib to + work with the async copye devices: + + * ``transfer_data(vid, queue_id, descs, opaque_data, count)`` + + vhost invokes this function to submit copy data to the async devices. + For non-async_inorder capable devices, ``opaque_data`` could be used + for identifying the completed packets. + + * ``check_completed_copies(vid, queue_id, opaque_data, max_packets)`` + + vhost invokes this function to get the copy data completed by async + devices. + +* ``rte_vhost_async_channel_unregister(vid, queue_id)`` + + Unregister the async copy device from a vhost queue. + +* ``rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count)`` + + Submit an enqueue request to transmit ``count`` packets from host to guest + by async data path. Enqueue is not guaranteed to finish upon the return of + this API call. + + Applications must not free the packets submitted for enqueue until the + packets are completed. + +* ``rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count)`` + + Poll enqueue completion status from async data path. Completed packets + are returned to applications through ``pkts``. + Vhost-user Implementations -------------------------- @@ -294,16 +362,16 @@ Guest memory requirement * Memory pre-allocation - For non-zerocopy, guest memory pre-allocation is not a must. This can help - save of memory. If users really want the guest memory to be pre-allocated - (e.g., for performance reason), we can add option ``-mem-prealloc`` when - starting QEMU. Or, we can lock all memory at vhost side which will force - memory to be allocated when mmap at vhost side; option --mlockall in - ovs-dpdk is an example in hand. + For non-zerocopy non-async data path, guest memory pre-allocation is not a + must. This can help save of memory. If users really want the guest memory + to be pre-allocated (e.g., for performance reason), we can add option + ``-mem-prealloc`` when starting QEMU. Or, we can lock all memory at vhost + side which will force memory to be allocated when mmap at vhost side; + option --mlockall in ovs-dpdk is an example in hand. - For zerocopy, we force the VM memory to be pre-allocated at vhost lib when - mapping the guest memory; and also we need to lock the memory to prevent - pages being swapped out to disk. + For async data path or zerocopy, we force the VM memory to be pre-allocated + at vhost lib when mapping the guest memory; and also we need to lock the + memory to prevent pages being swapped out to disk. * Memory sharing From patchwork Wed Jul 22 10:57:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Fu X-Patchwork-Id: 74617 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4F202A0526; Wed, 22 Jul 2020 13:00:21 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6FD191C010; Wed, 22 Jul 2020 13:00:15 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id AE3F51BFEF for ; Wed, 22 Jul 2020 13:00:12 +0200 (CEST) IronPort-SDR: vbidxePwHwla67YvX8celEaDl75MkgBUSU2D5nDEj1R4ew5MQbkA1Ssi/qGov5XfKwidl7Fl5i 7x7lyhuihgow== X-IronPort-AV: E=McAfee;i="6000,8403,9689"; a="148240103" X-IronPort-AV: E=Sophos;i="5.75,381,1589266800"; d="scan'208";a="148240103" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jul 2020 04:00:12 -0700 IronPort-SDR: mavGEEo1Z6cTwaErwSESiJxggidhLJ/JbY4hzhy0M9K9oGL/g35pv0VgFcuQilb50WBoBw722t H1e26CACOuGg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,381,1589266800"; d="scan'208";a="328178870" Received: from npg-dpdk-patrickfu-casc2.sh.intel.com ([10.67.119.92]) by orsmga007.jf.intel.com with ESMTP; 22 Jul 2020 04:00:10 -0700 From: patrick.fu@intel.com To: dev@dpdk.org, maxime.coquelin@redhat.com, chenbo.xia@intel.com Cc: Patrick Fu Date: Wed, 22 Jul 2020 18:57:41 +0800 Message-Id: <20200722105741.3421255-3-patrick.fu@intel.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20200722105741.3421255-1-patrick.fu@intel.com> References: <20200722105741.3421255-1-patrick.fu@intel.com> Subject: [dpdk-dev] [PATCH v1 2/2] doc: update release notes for vhost async APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Patrick Fu Update 20.08 release notes to include vhost async APIs Signed-off-by: Patrick Fu --- doc/guides/rel_notes/release_20_08.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/doc/guides/rel_notes/release_20_08.rst b/doc/guides/rel_notes/release_20_08.rst index f19b74872..47ed488ed 100644 --- a/doc/guides/rel_notes/release_20_08.rst +++ b/doc/guides/rel_notes/release_20_08.rst @@ -225,6 +225,11 @@ New Features See the :doc:`../sample_app_ug/l2_forward_real_virtual` for more details of this parameter usage. +* **Added vhost async data path APIs.** + + 4 new APIs has been added to enable vhost async data path. With the new + data path, applications can register vhost with copy devices, such as DMA + engines, to offload memory copy operations. Removed Items -------------