From patchwork Thu Jul 22 04:09:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jiang, Cheng1" X-Patchwork-Id: 96184 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DECE4A0C45; Thu, 22 Jul 2021 06:26:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AAD88410E0; Thu, 22 Jul 2021 06:26:26 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id 6BBD440E50; Thu, 22 Jul 2021 06:26:25 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10052"; a="209664601" X-IronPort-AV: E=Sophos;i="5.84,259,1620716400"; d="scan'208";a="209664601" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2021 21:26:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,259,1620716400"; d="scan'208";a="658668658" Received: from dpdk_jiangcheng.sh.intel.com ([10.67.119.149]) by fmsmga006.fm.intel.com with ESMTP; 21 Jul 2021 21:26:23 -0700 From: Cheng Jiang To: maxime.coquelin@redhat.com, Chenbo.Xia@intel.com Cc: dev@dpdk.org, jiayu.hu@intel.com, yvonnex.yang@intel.com, Cheng Jiang , stable@dpdk.org Date: Thu, 22 Jul 2021 04:09:03 +0000 Message-Id: <20210722040907.20468-2-cheng1.jiang@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210722040907.20468-1-cheng1.jiang@intel.com> References: <20210602042802.31943-1-cheng1.jiang@intel.com> <20210722040907.20468-1-cheng1.jiang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v7 1/5] vhost: fix async vhost ops return type X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The async vhost callback ops should return negative value when there are something wrong in the callback, so the return type should be changed into int32_t. The issue in vhost example is also fixed. Fixes: cd6760da1076 ("vhost: introduce async enqueue for split ring") Fixes: 819a71685826 ("vhost: fix async callback return type") Fixes: 6b3c81db8bb7 ("vhost: simplify async copy completion") Fixes: abec60e7115d ("examples/vhost: support vhost async data path") Fixes: 873e8dad6f49 ("vhost: support packed ring in async datapath") Cc: stable@dpdk.org Signed-off-by: Cheng Jiang Reviewed-by: Maxime Coquelin --- examples/vhost/ioat.c | 4 +-- examples/vhost/ioat.h | 4 +-- lib/vhost/rte_vhost_async.h | 8 ++--- lib/vhost/virtio_net.c | 61 ++++++++++++++++++++++++++++++++----- 4 files changed, 61 insertions(+), 16 deletions(-) diff --git a/examples/vhost/ioat.c b/examples/vhost/ioat.c index 2a2c2d7202..457f8171f0 100644 --- a/examples/vhost/ioat.c +++ b/examples/vhost/ioat.c @@ -122,7 +122,7 @@ open_ioat(const char *value) return ret; } -uint32_t +int32_t ioat_transfer_data_cb(int vid, uint16_t queue_id, struct rte_vhost_async_desc *descs, struct rte_vhost_async_status *opaque_data, uint16_t count) @@ -168,7 +168,7 @@ ioat_transfer_data_cb(int vid, uint16_t queue_id, return i_desc; } -uint32_t +int32_t ioat_check_completed_copies_cb(int vid, uint16_t queue_id, struct rte_vhost_async_status *opaque_data, uint16_t max_packets) diff --git a/examples/vhost/ioat.h b/examples/vhost/ioat.h index 1aa28ed6a3..b57b5645b0 100644 --- a/examples/vhost/ioat.h +++ b/examples/vhost/ioat.h @@ -27,12 +27,12 @@ struct dma_for_vhost { #ifdef RTE_RAW_IOAT int open_ioat(const char *value); -uint32_t +int32_t ioat_transfer_data_cb(int vid, uint16_t queue_id, struct rte_vhost_async_desc *descs, struct rte_vhost_async_status *opaque_data, uint16_t count); -uint32_t +int32_t ioat_check_completed_copies_cb(int vid, uint16_t queue_id, struct rte_vhost_async_status *opaque_data, uint16_t max_packets); diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h index 69ec66bba5..02d012ae23 100644 --- a/lib/vhost/rte_vhost_async.h +++ b/lib/vhost/rte_vhost_async.h @@ -59,9 +59,9 @@ struct rte_vhost_async_channel_ops { * @param count * number of elements in the "descs" array * @return - * number of descs processed + * number of descs processed, negative value means error */ - uint32_t (*transfer_data)(int vid, uint16_t queue_id, + int32_t (*transfer_data)(int vid, uint16_t queue_id, struct rte_vhost_async_desc *descs, struct rte_vhost_async_status *opaque_data, uint16_t count); @@ -76,9 +76,9 @@ struct rte_vhost_async_channel_ops { * @param max_packets * max number of packets could be completed * @return - * number of async descs completed + * number of async descs completed, negative value means error */ - uint32_t (*check_completed_copies)(int vid, uint16_t queue_id, + int32_t (*check_completed_copies)(int vid, uint16_t queue_id, struct rte_vhost_async_status *opaque_data, uint16_t max_packets); }; diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c index 6e5d82c1a8..3ab5229f76 100644 --- a/lib/vhost/virtio_net.c +++ b/lib/vhost/virtio_net.c @@ -1644,6 +1644,7 @@ virtio_dev_rx_async_submit_split(struct virtio_net *dev, struct async_inflight_info *pkts_info = vq->async_pkts_info; uint32_t n_pkts = 0, pkt_err = 0; uint32_t num_async_pkts = 0, num_done_pkts = 0; + int32_t n_xfer; struct { uint16_t pkt_idx; uint16_t last_avail_idx; @@ -1724,8 +1725,17 @@ virtio_dev_rx_async_submit_split(struct virtio_net *dev, if (unlikely(pkt_burst_idx >= VHOST_ASYNC_BATCH_THRESHOLD || ((VHOST_MAX_ASYNC_VEC >> 1) - segs_await < BUF_VECTOR_MAX))) { - n_pkts = vq->async_ops.transfer_data(dev->vid, + n_xfer = vq->async_ops.transfer_data(dev->vid, queue_id, tdes, 0, pkt_burst_idx); + if (n_xfer >= 0) { + n_pkts = n_xfer; + } else { + VHOST_LOG_DATA(ERR, + "(%d) %s: failed to transfer data for queue id %d.\n", + dev->vid, __func__, queue_id); + n_pkts = 0; + } + iovec_idx = 0; it_idx = 0; @@ -1748,8 +1758,15 @@ virtio_dev_rx_async_submit_split(struct virtio_net *dev, } if (pkt_burst_idx) { - n_pkts = vq->async_ops.transfer_data(dev->vid, - queue_id, tdes, 0, pkt_burst_idx); + n_xfer = vq->async_ops.transfer_data(dev->vid, queue_id, tdes, 0, pkt_burst_idx); + if (n_xfer >= 0) { + n_pkts = n_xfer; + } else { + VHOST_LOG_DATA(ERR, "(%d) %s: failed to transfer data for queue id %d.\n", + dev->vid, __func__, queue_id); + n_pkts = 0; + } + vq->async_pkts_inflight_n += n_pkts; if (unlikely(n_pkts < pkt_burst_idx)) @@ -1996,6 +2013,7 @@ virtio_dev_rx_async_submit_packed(struct virtio_net *dev, uint16_t async_descs_idx = 0; uint16_t num_buffers; uint16_t num_descs; + int32_t n_xfer; struct rte_vhost_iov_iter *it_pool = vq->it_pool; struct iovec *vec_pool = vq->vec_pool; @@ -2078,8 +2096,17 @@ virtio_dev_rx_async_submit_packed(struct virtio_net *dev, */ if (unlikely(pkt_burst_idx >= VHOST_ASYNC_BATCH_THRESHOLD || ((VHOST_MAX_ASYNC_VEC >> 1) - segs_await < BUF_VECTOR_MAX))) { - n_pkts = vq->async_ops.transfer_data(dev->vid, queue_id, - tdes, 0, pkt_burst_idx); + n_xfer = vq->async_ops.transfer_data(dev->vid, + queue_id, tdes, 0, pkt_burst_idx); + if (n_xfer >= 0) { + n_pkts = n_xfer; + } else { + VHOST_LOG_DATA(ERR, + "(%d) %s: failed to transfer data for queue id %d.\n", + dev->vid, __func__, queue_id); + n_pkts = 0; + } + iovec_idx = 0; it_idx = 0; segs_await = 0; @@ -2101,7 +2128,15 @@ virtio_dev_rx_async_submit_packed(struct virtio_net *dev, } while (pkt_idx < count); if (pkt_burst_idx) { - n_pkts = vq->async_ops.transfer_data(dev->vid, queue_id, tdes, 0, pkt_burst_idx); + n_xfer = vq->async_ops.transfer_data(dev->vid, queue_id, tdes, 0, pkt_burst_idx); + if (n_xfer >= 0) { + n_pkts = n_xfer; + } else { + VHOST_LOG_DATA(ERR, "(%d) %s: failed to transfer data for queue id %d.\n", + dev->vid, __func__, queue_id); + n_pkts = 0; + } + vq->async_pkts_inflight_n += n_pkts; if (unlikely(n_pkts < pkt_burst_idx)) @@ -2188,6 +2223,7 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id, uint16_t start_idx, pkts_idx, vq_size; struct async_inflight_info *pkts_info; uint16_t from, i; + int32_t n_cpl; if (!dev) return 0; @@ -2215,9 +2251,18 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id, start_idx = virtio_dev_rx_async_get_info_idx(pkts_idx, vq_size, vq->async_pkts_inflight_n); - if (count > vq->async_last_pkts_n) - n_pkts_cpl = vq->async_ops.check_completed_copies(vid, + if (count > vq->async_last_pkts_n) { + n_cpl = vq->async_ops.check_completed_copies(vid, queue_id, 0, count - vq->async_last_pkts_n); + if (n_cpl >= 0) { + n_pkts_cpl = n_cpl; + } else { + VHOST_LOG_DATA(ERR, + "(%d) %s: failed to check completed copies for queue id %d.\n", + dev->vid, __func__, queue_id); + n_pkts_cpl = 0; + } + } n_pkts_cpl += vq->async_last_pkts_n; n_pkts_put = RTE_MIN(count, n_pkts_cpl); From patchwork Thu Jul 22 04:09:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jiang, Cheng1" X-Patchwork-Id: 96185 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 07630A0C45; Thu, 22 Jul 2021 06:26:38 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D8C00410EC; Thu, 22 Jul 2021 06:26:29 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id 20DAA410EC for ; Thu, 22 Jul 2021 06:26:27 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10052"; a="209664605" X-IronPort-AV: E=Sophos;i="5.84,259,1620716400"; d="scan'208";a="209664605" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2021 21:26:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,259,1620716400"; d="scan'208";a="658668666" Received: from dpdk_jiangcheng.sh.intel.com ([10.67.119.149]) by fmsmga006.fm.intel.com with ESMTP; 21 Jul 2021 21:26:26 -0700 From: Cheng Jiang To: maxime.coquelin@redhat.com, Chenbo.Xia@intel.com Cc: dev@dpdk.org, jiayu.hu@intel.com, yvonnex.yang@intel.com, Cheng Jiang Date: Thu, 22 Jul 2021 04:09:04 +0000 Message-Id: <20210722040907.20468-3-cheng1.jiang@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210722040907.20468-1-cheng1.jiang@intel.com> References: <20210602042802.31943-1-cheng1.jiang@intel.com> <20210722040907.20468-1-cheng1.jiang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v7 2/5] vhost: add unsafe API to clear packets in async vhost X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Applications need to stop DMA transfers and finish all the inflight packets when in VM memory hot-plug case and async vhost is used. This patch is to provide an unsafe API to clear inflight packets which are submitted to DMA engine in vhost async data path. Signed-off-by: Cheng Jiang Reviewed-by: Maxime Coquelin --- lib/vhost/rte_vhost_async.h | 22 +++++++++ lib/vhost/version.map | 1 + lib/vhost/virtio_net.c | 93 +++++++++++++++++++++++++++---------- 3 files changed, 92 insertions(+), 24 deletions(-) diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h index 02d012ae23..b25ff446f7 100644 --- a/lib/vhost/rte_vhost_async.h +++ b/lib/vhost/rte_vhost_async.h @@ -246,4 +246,26 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id, __rte_experimental int rte_vhost_async_get_inflight(int vid, uint16_t queue_id); +/** + * This function checks async completion status and clear packets for + * a specific vhost device queue. Packets which are inflight will be + * returned in an array. + * + * @note This function does not perform any locking + * + * @param vid + * ID of vhost device to clear data + * @param queue_id + * Queue id to clear data + * @param pkts + * Blank array to get return packet pointer + * @param count + * Size of the packet array + * @return + * Number of packets returned + */ +__rte_experimental +uint16_t rte_vhost_clear_queue_thread_unsafe(int vid, uint16_t queue_id, + struct rte_mbuf **pkts, uint16_t count); + #endif /* _RTE_VHOST_ASYNC_H_ */ diff --git a/lib/vhost/version.map b/lib/vhost/version.map index e0c89646e8..e2504ba657 100644 --- a/lib/vhost/version.map +++ b/lib/vhost/version.map @@ -84,4 +84,5 @@ EXPERIMENTAL { rte_vhost_async_get_inflight; rte_vhost_async_channel_register_thread_unsafe; rte_vhost_async_channel_unregister_thread_unsafe; + rte_vhost_clear_queue_thread_unsafe; }; diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c index 3ab5229f76..8549afbbe1 100644 --- a/lib/vhost/virtio_net.c +++ b/lib/vhost/virtio_net.c @@ -2214,10 +2214,10 @@ write_back_completed_descs_packed(struct vhost_virtqueue *vq, } while (nr_left > 0); } -uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id, +static __rte_always_inline uint16_t +vhost_poll_enqueue_completed(struct virtio_net *dev, uint16_t queue_id, struct rte_mbuf **pkts, uint16_t count) { - struct virtio_net *dev = get_device(vid); struct vhost_virtqueue *vq; uint16_t n_pkts_cpl = 0, n_pkts_put = 0, n_descs = 0, n_buffers = 0; uint16_t start_idx, pkts_idx, vq_size; @@ -2225,26 +2225,8 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id, uint16_t from, i; int32_t n_cpl; - if (!dev) - return 0; - - VHOST_LOG_DATA(DEBUG, "(%d) %s\n", dev->vid, __func__); - if (unlikely(!is_valid_virt_queue_idx(queue_id, 0, dev->nr_vring))) { - VHOST_LOG_DATA(ERR, "(%d) %s: invalid virtqueue idx %d.\n", - dev->vid, __func__, queue_id); - return 0; - } - vq = dev->virtqueue[queue_id]; - if (unlikely(!vq->async_registered)) { - VHOST_LOG_DATA(ERR, "(%d) %s: async not registered for queue id %d.\n", - dev->vid, __func__, queue_id); - return 0; - } - - rte_spinlock_lock(&vq->access_lock); - pkts_idx = vq->async_pkts_idx % vq->size; pkts_info = vq->async_pkts_info; vq_size = vq->size; @@ -2252,7 +2234,7 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id, vq_size, vq->async_pkts_inflight_n); if (count > vq->async_last_pkts_n) { - n_cpl = vq->async_ops.check_completed_copies(vid, + n_cpl = vq->async_ops.check_completed_copies(dev->vid, queue_id, 0, count - vq->async_last_pkts_n); if (n_cpl >= 0) { n_pkts_cpl = n_cpl; @@ -2268,7 +2250,7 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id, n_pkts_put = RTE_MIN(count, n_pkts_cpl); if (unlikely(n_pkts_put == 0)) { vq->async_last_pkts_n = n_pkts_cpl; - goto done; + return 0; } if (vq_is_packed(dev)) { @@ -2310,10 +2292,73 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id, } } -done: + return n_pkts_put; +} + +uint16_t +rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id, + struct rte_mbuf **pkts, uint16_t count) +{ + struct virtio_net *dev = get_device(vid); + struct vhost_virtqueue *vq; + uint16_t n_pkts_cpl = 0; + + if (!dev) + return 0; + + VHOST_LOG_DATA(DEBUG, "(%d) %s\n", dev->vid, __func__); + if (unlikely(!is_valid_virt_queue_idx(queue_id, 0, dev->nr_vring))) { + VHOST_LOG_DATA(ERR, "(%d) %s: invalid virtqueue idx %d.\n", + dev->vid, __func__, queue_id); + return 0; + } + + vq = dev->virtqueue[queue_id]; + + if (unlikely(!vq->async_registered)) { + VHOST_LOG_DATA(ERR, "(%d) %s: async not registered for queue id %d.\n", + dev->vid, __func__, queue_id); + return 0; + } + + rte_spinlock_lock(&vq->access_lock); + + n_pkts_cpl = vhost_poll_enqueue_completed(dev, queue_id, pkts, count); + rte_spinlock_unlock(&vq->access_lock); - return n_pkts_put; + return n_pkts_cpl; +} + +uint16_t +rte_vhost_clear_queue_thread_unsafe(int vid, uint16_t queue_id, + struct rte_mbuf **pkts, uint16_t count) +{ + struct virtio_net *dev = get_device(vid); + struct vhost_virtqueue *vq; + uint16_t n_pkts_cpl = 0; + + if (!dev) + return 0; + + VHOST_LOG_DATA(DEBUG, "(%d) %s\n", dev->vid, __func__); + if (unlikely(!is_valid_virt_queue_idx(queue_id, 0, dev->nr_vring))) { + VHOST_LOG_DATA(ERR, "(%d) %s: invalid virtqueue idx %d.\n", + dev->vid, __func__, queue_id); + return 0; + } + + vq = dev->virtqueue[queue_id]; + + if (unlikely(!vq->async_registered)) { + VHOST_LOG_DATA(ERR, "(%d) %s: async not registered for queue id %d.\n", + dev->vid, __func__, queue_id); + return 0; + } + + n_pkts_cpl = vhost_poll_enqueue_completed(dev, queue_id, pkts, count); + + return n_pkts_cpl; } static __rte_always_inline uint32_t From patchwork Thu Jul 22 04:09:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jiang, Cheng1" X-Patchwork-Id: 96186 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EE598A0C45; Thu, 22 Jul 2021 06:26:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 045CD410F8; Thu, 22 Jul 2021 06:26:32 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id 5650C410F1 for ; Thu, 22 Jul 2021 06:26:30 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10052"; a="209664606" X-IronPort-AV: E=Sophos;i="5.84,259,1620716400"; d="scan'208";a="209664606" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2021 21:26:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,259,1620716400"; d="scan'208";a="658668671" Received: from dpdk_jiangcheng.sh.intel.com ([10.67.119.149]) by fmsmga006.fm.intel.com with ESMTP; 21 Jul 2021 21:26:28 -0700 From: Cheng Jiang To: maxime.coquelin@redhat.com, Chenbo.Xia@intel.com Cc: dev@dpdk.org, jiayu.hu@intel.com, yvonnex.yang@intel.com Date: Thu, 22 Jul 2021 04:09:05 +0000 Message-Id: <20210722040907.20468-4-cheng1.jiang@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210722040907.20468-1-cheng1.jiang@intel.com> References: <20210602042802.31943-1-cheng1.jiang@intel.com> <20210722040907.20468-1-cheng1.jiang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v7 3/5] vhost: handle memory hotplug for async vhost X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jiayu Hu When the guest memory is hotplugged, the vhost application which enables DMA acceleration must stop DMA transfers before the vhost re-maps the guest memory. This patch is to notify the vhost application of stopping DMA transfers. Signed-off-by: Jiayu Hu Reviewed-by: Maxime Coquelin --- lib/vhost/vhost_user.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 31300e194f..433f412fa8 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -1248,6 +1248,7 @@ vhost_user_set_mem_table(struct virtio_net **pdev, struct VhostUserMsg *msg, int numa_node = SOCKET_ID_ANY; uint64_t mmap_offset; uint32_t i; + bool async_notify = false; if (validate_msg_fds(msg, memory->nregions) != 0) return RTE_VHOST_MSG_RESULT_ERR; @@ -1275,6 +1276,16 @@ vhost_user_set_mem_table(struct virtio_net **pdev, struct VhostUserMsg *msg, vdpa_dev->ops->dev_close(dev->vid); dev->flags &= ~VIRTIO_DEV_VDPA_CONFIGURED; } + + /* notify the vhost application to stop DMA transfers */ + if (dev->async_copy && dev->notify_ops->vring_state_changed) { + for (i = 0; i < dev->nr_vring; i++) { + dev->notify_ops->vring_state_changed(dev->vid, + i, 0); + } + async_notify = true; + } + free_mem_region(dev); rte_free(dev->mem); dev->mem = NULL; @@ -1371,6 +1382,11 @@ vhost_user_set_mem_table(struct virtio_net **pdev, struct VhostUserMsg *msg, dump_guest_pages(dev); + if (async_notify) { + for (i = 0; i < dev->nr_vring; i++) + dev->notify_ops->vring_state_changed(dev->vid, i, 1); + } + return RTE_VHOST_MSG_RESULT_OK; free_mem_table: From patchwork Thu Jul 22 04:09:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jiang, Cheng1" X-Patchwork-Id: 96187 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 65113A0C45; Thu, 22 Jul 2021 06:26:49 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 22896410FE; Thu, 22 Jul 2021 06:26:34 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id BE5C7410FC for ; Thu, 22 Jul 2021 06:26:32 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10052"; a="209664611" X-IronPort-AV: E=Sophos;i="5.84,259,1620716400"; d="scan'208";a="209664611" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2021 21:26:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,259,1620716400"; d="scan'208";a="658668680" Received: from dpdk_jiangcheng.sh.intel.com ([10.67.119.149]) by fmsmga006.fm.intel.com with ESMTP; 21 Jul 2021 21:26:30 -0700 From: Cheng Jiang To: maxime.coquelin@redhat.com, Chenbo.Xia@intel.com Cc: dev@dpdk.org, jiayu.hu@intel.com, yvonnex.yang@intel.com, Cheng Jiang Date: Thu, 22 Jul 2021 04:09:06 +0000 Message-Id: <20210722040907.20468-5-cheng1.jiang@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210722040907.20468-1-cheng1.jiang@intel.com> References: <20210602042802.31943-1-cheng1.jiang@intel.com> <20210722040907.20468-1-cheng1.jiang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v7 4/5] examples/vhost: handle memory hotplug for async vhost X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When the guest memory is hotplugged, the vhost application which enables DMA acceleration must stop DMA transfers before the vhost re-maps the guest memory. To accomplish that, we need to do these changes in the vhost sample: 1. add inflight packets count. 2. add vring_state_changed() callback. 3. add inflight packets clear process in destroy_device() and vring_state_changed(). Signed-off-by: Cheng Jiang Reviewed-by: Maxime Coquelin --- examples/vhost/main.c | 55 +++++++++++++++++++++++++++++++++++++++++-- examples/vhost/main.h | 1 + 2 files changed, 54 insertions(+), 2 deletions(-) diff --git a/examples/vhost/main.c b/examples/vhost/main.c index 9cd855a696..bc3d71c898 100644 --- a/examples/vhost/main.c +++ b/examples/vhost/main.c @@ -851,8 +851,11 @@ complete_async_pkts(struct vhost_dev *vdev) complete_count = rte_vhost_poll_enqueue_completed(vdev->vid, VIRTIO_RXQ, p_cpl, MAX_PKT_BURST); - if (complete_count) + if (complete_count) { free_pkts(p_cpl, complete_count); + __atomic_sub_fetch(&vdev->pkts_inflight, complete_count, __ATOMIC_SEQ_CST); + } + } static __rte_always_inline void @@ -895,6 +898,7 @@ drain_vhost(struct vhost_dev *vdev) complete_async_pkts(vdev); ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ, m, nr_xmit, m_cpu_cpl, &cpu_cpl_nr); + __atomic_add_fetch(&vdev->pkts_inflight, ret - cpu_cpl_nr, __ATOMIC_SEQ_CST); if (cpu_cpl_nr) free_pkts(m_cpu_cpl, cpu_cpl_nr); @@ -1226,6 +1230,9 @@ drain_eth_rx(struct vhost_dev *vdev) enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ, pkts, rx_count, m_cpu_cpl, &cpu_cpl_nr); + __atomic_add_fetch(&vdev->pkts_inflight, enqueue_count - cpu_cpl_nr, + __ATOMIC_SEQ_CST); + if (cpu_cpl_nr) free_pkts(m_cpu_cpl, cpu_cpl_nr); @@ -1397,8 +1404,19 @@ destroy_device(int vid) "(%d) device has been removed from data core\n", vdev->vid); - if (async_vhost_driver) + if (async_vhost_driver) { + uint16_t n_pkt = 0; + struct rte_mbuf *m_cpl[vdev->pkts_inflight]; + + while (vdev->pkts_inflight) { + n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, VIRTIO_RXQ, + m_cpl, vdev->pkts_inflight); + free_pkts(m_cpl, n_pkt); + __atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST); + } + rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ); + } rte_free(vdev); } @@ -1487,6 +1505,38 @@ new_device(int vid) return 0; } +static int +vring_state_changed(int vid, uint16_t queue_id, int enable) +{ + struct vhost_dev *vdev = NULL; + + TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) { + if (vdev->vid == vid) + break; + } + if (!vdev) + return -1; + + if (queue_id != VIRTIO_RXQ) + return 0; + + if (async_vhost_driver) { + if (!enable) { + uint16_t n_pkt = 0; + struct rte_mbuf *m_cpl[vdev->pkts_inflight]; + + while (vdev->pkts_inflight) { + n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, queue_id, + m_cpl, vdev->pkts_inflight); + free_pkts(m_cpl, n_pkt); + __atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST); + } + } + } + + return 0; +} + /* * These callback allow devices to be added to the data core when configuration * has been fully complete. @@ -1495,6 +1545,7 @@ static const struct vhost_device_ops virtio_net_device_ops = { .new_device = new_device, .destroy_device = destroy_device, + .vring_state_changed = vring_state_changed, }; /* diff --git a/examples/vhost/main.h b/examples/vhost/main.h index 0ccdce4b4a..e7b1ac60a6 100644 --- a/examples/vhost/main.h +++ b/examples/vhost/main.h @@ -51,6 +51,7 @@ struct vhost_dev { uint64_t features; size_t hdr_len; uint16_t nr_vrings; + uint16_t pkts_inflight; struct rte_vhost_memory *mem; struct device_statistics stats; TAILQ_ENTRY(vhost_dev) global_vdev_entry; From patchwork Thu Jul 22 04:09:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jiang, Cheng1" X-Patchwork-Id: 96188 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 522E3A0C45; Thu, 22 Jul 2021 06:26:55 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 476F54111B; Thu, 22 Jul 2021 06:26:36 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id E5CE641109 for ; Thu, 22 Jul 2021 06:26:34 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10052"; a="209664614" X-IronPort-AV: E=Sophos;i="5.84,259,1620716400"; d="scan'208";a="209664614" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2021 21:26:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,259,1620716400"; d="scan'208";a="658668688" Received: from dpdk_jiangcheng.sh.intel.com ([10.67.119.149]) by fmsmga006.fm.intel.com with ESMTP; 21 Jul 2021 21:26:33 -0700 From: Cheng Jiang To: maxime.coquelin@redhat.com, Chenbo.Xia@intel.com Cc: dev@dpdk.org, jiayu.hu@intel.com, yvonnex.yang@intel.com, Cheng Jiang Date: Thu, 22 Jul 2021 04:09:07 +0000 Message-Id: <20210722040907.20468-6-cheng1.jiang@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210722040907.20468-1-cheng1.jiang@intel.com> References: <20210602042802.31943-1-cheng1.jiang@intel.com> <20210722040907.20468-1-cheng1.jiang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v7 5/5] doc: update doc for queue clear API in vhost lib X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Update the program guide and release notes for virtqueue inflight packets clear API in vhost lib. Signed-off-by: Cheng Jiang Reviewed-by: Maxime Coquelin --- doc/guides/prog_guide/vhost_lib.rst | 5 +++++ doc/guides/rel_notes/release_21_08.rst | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst index 70ce4974df..8874033165 100644 --- a/doc/guides/prog_guide/vhost_lib.rst +++ b/doc/guides/prog_guide/vhost_lib.rst @@ -305,6 +305,11 @@ The following is an overview of some key Vhost API functions: This function returns the amount of in-flight packets for the vhost queue using async acceleration. +* ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)`` + + Clear inflight packets which are submitted to DMA engine in vhost async data + path. Completed packets are returned to applications through ``pkts``. + Vhost-user Implementations -------------------------- diff --git a/doc/guides/rel_notes/release_21_08.rst b/doc/guides/rel_notes/release_21_08.rst index 543e93ff1d..d9c4cc5df0 100644 --- a/doc/guides/rel_notes/release_21_08.rst +++ b/doc/guides/rel_notes/release_21_08.rst @@ -155,6 +155,11 @@ New Features The experimental PMD power management API now supports managing multiple Ethernet Rx queues per lcore. +* **Added inflight packets clear API in vhost library.** + + Added an API which can clear the inflight packets submitted to DMA + engine in vhost async data path. + Removed Items -------------