Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/114853/?format=api
{ "id": 114853, "url": "https://patches.dpdk.org/api/patches/114853/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/20220812064517.272530-1-wenwux.ma@intel.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20220812064517.272530-1-wenwux.ma@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20220812064517.272530-1-wenwux.ma@intel.com", "date": "2022-08-12T06:45:17", "name": "vhost: support CPU copy for small packets", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "c44f9a9e82da4de888f74a4e68ee58f6d3cfc806", "submitter": { "id": 2163, "url": "https://patches.dpdk.org/api/people/2163/?format=api", "name": "Ma, WenwuX", "email": "wenwux.ma@intel.com" }, "delegate": { "id": 2642, "url": "https://patches.dpdk.org/api/users/2642/?format=api", "username": "mcoquelin", "first_name": "Maxime", "last_name": "Coquelin", "email": "maxime.coquelin@redhat.com" }, "mbox": "https://patches.dpdk.org/project/dpdk/patch/20220812064517.272530-1-wenwux.ma@intel.com/mbox/", "series": [ { "id": 24280, "url": "https://patches.dpdk.org/api/series/24280/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/list/?series=24280", "date": "2022-08-12T06:45:17", "name": "vhost: support CPU copy for small packets", "version": 1, "mbox": "https://patches.dpdk.org/series/24280/mbox/" } ], "comments": "https://patches.dpdk.org/api/patches/114853/comments/", "check": "fail", "checks": "https://patches.dpdk.org/api/patches/114853/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 8FB78A034C;\n\tFri, 12 Aug 2022 08:46:40 +0200 (CEST)", "from [217.70.189.124] (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 390C3406A2;\n\tFri, 12 Aug 2022 08:46:40 +0200 (CEST)", "from mga12.intel.com (mga12.intel.com [192.55.52.136])\n by mails.dpdk.org (Postfix) with ESMTP id BBD5D40697\n for <dev@dpdk.org>; Fri, 12 Aug 2022 08:46:38 +0200 (CEST)", "from fmsmga005.fm.intel.com ([10.253.24.32])\n by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 11 Aug 2022 23:46:37 -0700", "from unknown (HELO localhost.localdomain) ([10.239.252.251])\n by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 11 Aug 2022 23:46:34 -0700" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple;\n d=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n t=1660286798; x=1691822798;\n h=from:to:cc:subject:date:message-id:mime-version:\n content-transfer-encoding;\n bh=Oqj4fByu8he/7OORxfwJGMo7TJAC9kqKrfulMRZlTnE=;\n b=IlXP27AEadTLnLxSNu3SfVzTabBAVAKgz1ZZC2manfBbO+V3ZERxF9kR\n wAHygTOdgpGux3S+QWNU19nhkT4rR0WBJ0eVLk2WJm3ZiUmriAQX6TV84\n OHXeJ6xeFvoi9WF7YPU9GFHjBa/cJBio8Bcl5oMpTgZNF9nrf7aLsQfeC\n q/1C6AbOopcBW8Wh5Mb8KS7FuZhdtbdx8DqLmngSDEM9lDtyumQJ7FFN8\n MlcMdk/8COmfgF/TG/QFKxsNcCdMamVQIEB9lpN5ntqoDIIo7YkYv5ZjR\n /WQg0YQKfSzmSgEuc8AFPAo5MRbF71KqoE0FYbDRWFeYBHIHLdF7HB2jQ w==;", "X-IronPort-AV": [ "E=McAfee;i=\"6400,9594,10436\"; a=\"271308963\"", "E=Sophos;i=\"5.93,231,1654585200\"; d=\"scan'208\";a=\"271308963\"", "E=Sophos;i=\"5.93,231,1654585200\"; d=\"scan'208\";a=\"933610647\"" ], "From": "Wenwu Ma <wenwux.ma@intel.com>", "To": "maxime.coquelin@redhat.com,\n\tchenbo.xia@intel.com,\n\tdev@dpdk.org", "Cc": "jiayu.hu@intel.com, yinan.wang@intel.com, xingguang.he@intel.com,\n xuan.ding@intel.com, cheng1.jiang@intel.com, yuanx.wang@intel.com,\n Wenwu Ma <wenwux.ma@intel.com>", "Subject": "[PATCH] vhost: support CPU copy for small packets", "Date": "Fri, 12 Aug 2022 14:45:17 +0800", "Message-Id": "<20220812064517.272530-1-wenwux.ma@intel.com>", "X-Mailer": "git-send-email 2.25.1", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Offloading small packets to DMA degrades throughput 10%~20%,\nand this is because DMA offloading is not free and DMA is not\ngood at processing small packets. In addition, control plane\npackets are usually small, and assign those packets to DMA will\nsignificantly increase latency, which may cause timeout like\nTCP handshake packets. Therefore, this patch use CPU to perform\nsmall copies in vhost.\n\nSigned-off-by: Wenwu Ma <wenwux.ma@intel.com>\n---\n lib/vhost/vhost.h | 6 ++-\n lib/vhost/virtio_net.c | 87 +++++++++++++++++++++++++++---------------\n 2 files changed, 61 insertions(+), 32 deletions(-)", "diff": "diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h\nindex 40fac3b7c6..b4523175a8 100644\n--- a/lib/vhost/vhost.h\n+++ b/lib/vhost/vhost.h\n@@ -142,8 +142,10 @@ struct virtqueue_stats {\n * iovec\n */\n struct vhost_iovec {\n-\tvoid *src_addr;\n-\tvoid *dst_addr;\n+\tvoid *src_iov_addr;\n+\tvoid *dst_iov_addr;\n+\tvoid *src_virt_addr;\n+\tvoid *dst_virt_addr;\n \tsize_t len;\n };\n \ndiff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c\nindex 35fa4670fd..b3bed93de7 100644\n--- a/lib/vhost/virtio_net.c\n+++ b/lib/vhost/virtio_net.c\n@@ -26,6 +26,8 @@\n \n #define MAX_BATCH_LEN 256\n \n+#define CPU_COPY_THRESHOLD_LEN 256\n+\n static __rte_always_inline uint16_t\n async_poll_dequeue_completed(struct virtio_net *dev, struct vhost_virtqueue *vq,\n \t\tstruct rte_mbuf **pkts, uint16_t count, int16_t dma_id,\n@@ -114,29 +116,36 @@ vhost_async_dma_transfer_one(struct virtio_net *dev, struct vhost_virtqueue *vq,\n \tint copy_idx = 0;\n \tuint32_t nr_segs = pkt->nr_segs;\n \tuint16_t i;\n+\tbool cpu_copy = true;\n \n \tif (rte_dma_burst_capacity(dma_id, vchan_id) < nr_segs)\n \t\treturn -1;\n \n \tfor (i = 0; i < nr_segs; i++) {\n-\t\tcopy_idx = rte_dma_copy(dma_id, vchan_id, (rte_iova_t)iov[i].src_addr,\n-\t\t\t\t(rte_iova_t)iov[i].dst_addr, iov[i].len, RTE_DMA_OP_FLAG_LLC);\n-\t\t/**\n-\t\t * Since all memory is pinned and DMA vChannel\n-\t\t * ring has enough space, failure should be a\n-\t\t * rare case. If failure happens, it means DMA\n-\t\t * device encounters serious errors; in this\n-\t\t * case, please stop async data-path and check\n-\t\t * what has happened to DMA device.\n-\t\t */\n-\t\tif (unlikely(copy_idx < 0)) {\n-\t\t\tif (!vhost_async_dma_copy_log) {\n-\t\t\t\tVHOST_LOG_DATA(dev->ifname, ERR,\n-\t\t\t\t\t\"DMA copy failed for channel %d:%u\\n\",\n-\t\t\t\t\tdma_id, vchan_id);\n-\t\t\t\tvhost_async_dma_copy_log = true;\n+\t\tif (iov[i].len > CPU_COPY_THRESHOLD_LEN) {\n+\t\t\tcopy_idx = rte_dma_copy(dma_id, vchan_id, (rte_iova_t)iov[i].src_iov_addr,\n+\t\t\t\t\t(rte_iova_t)iov[i].dst_iov_addr,\n+\t\t\t\t\tiov[i].len, RTE_DMA_OP_FLAG_LLC);\n+\t\t\t/**\n+\t\t\t * Since all memory is pinned and DMA vChannel\n+\t\t\t * ring has enough space, failure should be a\n+\t\t\t * rare case. If failure happens, it means DMA\n+\t\t\t * device encounters serious errors; in this\n+\t\t\t * case, please stop async data-path and check\n+\t\t\t * what has happened to DMA device.\n+\t\t\t */\n+\t\t\tif (unlikely(copy_idx < 0)) {\n+\t\t\t\tif (!vhost_async_dma_copy_log) {\n+\t\t\t\t\tVHOST_LOG_DATA(dev->ifname, ERR,\n+\t\t\t\t\t\t\"DMA copy failed for channel %d:%u\\n\",\n+\t\t\t\t\t\tdma_id, vchan_id);\n+\t\t\t\t\tvhost_async_dma_copy_log = true;\n+\t\t\t\t}\n+\t\t\t\treturn -1;\n \t\t\t}\n-\t\t\treturn -1;\n+\t\t\tcpu_copy = false;\n+\t\t} else {\n+\t\t\trte_memcpy(iov[i].dst_virt_addr, iov[i].src_virt_addr, iov[i].len);\n \t\t}\n \t}\n \n@@ -144,7 +153,13 @@ vhost_async_dma_transfer_one(struct virtio_net *dev, struct vhost_virtqueue *vq,\n \t * Only store packet completion flag address in the last copy's\n \t * slot, and other slots are set to NULL.\n \t */\n-\tdma_info->pkts_cmpl_flag_addr[copy_idx & ring_mask] = &vq->async->pkts_cmpl_flag[flag_idx];\n+\tif (cpu_copy == false) {\n+\t\tdma_info->pkts_cmpl_flag_addr[copy_idx & ring_mask] =\n+\t\t\t&vq->async->pkts_cmpl_flag[flag_idx];\n+\t} else {\n+\t\tvq->async->pkts_cmpl_flag[flag_idx] = true;\n+\t\tnr_segs = 0;\n+\t}\n \n \treturn nr_segs;\n }\n@@ -1008,7 +1023,7 @@ async_iter_initialize(struct virtio_net *dev, struct vhost_async *async)\n \n static __rte_always_inline int\n async_iter_add_iovec(struct virtio_net *dev, struct vhost_async *async,\n-\t\tvoid *src, void *dst, size_t len)\n+\t\tvoid *src_iova, void *dst_iova, void *src_addr, void *dst_addr, size_t len)\n {\n \tstruct vhost_iov_iter *iter;\n \tstruct vhost_iovec *iovec;\n@@ -1027,8 +1042,10 @@ async_iter_add_iovec(struct virtio_net *dev, struct vhost_async *async,\n \titer = async->iov_iter + async->iter_idx;\n \tiovec = async->iovec + async->iovec_idx;\n \n-\tiovec->src_addr = src;\n-\tiovec->dst_addr = dst;\n+\tiovec->src_iov_addr = src_iova;\n+\tiovec->dst_iov_addr = dst_iova;\n+\tiovec->src_virt_addr = src_addr;\n+\tiovec->dst_virt_addr = dst_addr;\n \tiovec->len = len;\n \n \titer->nr_segs++;\n@@ -1064,12 +1081,13 @@ async_iter_reset(struct vhost_async *async)\n static __rte_always_inline int\n async_fill_seg(struct virtio_net *dev, struct vhost_virtqueue *vq,\n \t\tstruct rte_mbuf *m, uint32_t mbuf_offset,\n-\t\tuint64_t buf_iova, uint32_t cpy_len, bool to_desc)\n+\t\tuint64_t buf_iova, uint64_t buf_addr, uint32_t cpy_len, bool to_desc)\n {\n \tstruct vhost_async *async = vq->async;\n \tuint64_t mapped_len;\n \tuint32_t buf_offset = 0;\n-\tvoid *src, *dst;\n+\tvoid *src_iova, *dst_iova;\n+\tvoid *src_addr, *dst_addr;\n \tvoid *host_iova;\n \n \twhile (cpy_len) {\n@@ -1083,14 +1101,21 @@ async_fill_seg(struct virtio_net *dev, struct vhost_virtqueue *vq,\n \t\t}\n \n \t\tif (to_desc) {\n-\t\t\tsrc = (void *)(uintptr_t)rte_pktmbuf_iova_offset(m, mbuf_offset);\n-\t\t\tdst = host_iova;\n+\t\t\tsrc_iova = (void *)(uintptr_t)rte_pktmbuf_iova_offset(m, mbuf_offset);\n+\t\t\tdst_iova = host_iova;\n+\n+\t\t\tsrc_addr = rte_pktmbuf_mtod_offset(m, void *, mbuf_offset);\n+\t\t\tdst_addr = (void *)(buf_addr + buf_offset);\n \t\t} else {\n-\t\t\tsrc = host_iova;\n-\t\t\tdst = (void *)(uintptr_t)rte_pktmbuf_iova_offset(m, mbuf_offset);\n+\t\t\tsrc_iova = host_iova;\n+\t\t\tdst_iova = (void *)(uintptr_t)rte_pktmbuf_iova_offset(m, mbuf_offset);\n+\n+\t\t\tsrc_addr = (void *)(buf_addr + buf_offset);\n+\t\t\tdst_addr = rte_pktmbuf_mtod_offset(m, void *, mbuf_offset);\n \t\t}\n \n-\t\tif (unlikely(async_iter_add_iovec(dev, async, src, dst, (size_t)mapped_len)))\n+\t\tif (unlikely(async_iter_add_iovec(dev, async, src_iova, dst_iova,\n+\t\t\t\t\t\tsrc_addr, dst_addr, (size_t)mapped_len)))\n \t\t\treturn -1;\n \n \t\tcpy_len -= (uint32_t)mapped_len;\n@@ -1239,7 +1264,8 @@ mbuf_to_desc(struct virtio_net *dev, struct vhost_virtqueue *vq,\n \n \t\tif (is_async) {\n \t\t\tif (async_fill_seg(dev, vq, m, mbuf_offset,\n-\t\t\t\t\t buf_iova + buf_offset, cpy_len, true) < 0)\n+\t\t\t\t\t buf_iova + buf_offset, buf_addr + buf_offset,\n+\t\t\t\t\t cpy_len, true) < 0)\n \t\t\t\tgoto error;\n \t\t} else {\n \t\t\tsync_fill_seg(dev, vq, m, mbuf_offset,\n@@ -2737,7 +2763,8 @@ desc_to_mbuf(struct virtio_net *dev, struct vhost_virtqueue *vq,\n \n \t\tif (is_async) {\n \t\t\tif (async_fill_seg(dev, vq, cur, mbuf_offset,\n-\t\t\t\t\t buf_iova + buf_offset, cpy_len, false) < 0)\n+\t\t\t\t\t buf_iova + buf_offset, buf_addr + buf_offset,\n+\t\t\t\t\t cpy_len, false) < 0)\n \t\t\t\tgoto error;\n \t\t} else if (likely(hdr && cur == m)) {\n \t\t\trte_memcpy(rte_pktmbuf_mtod_offset(cur, void *, mbuf_offset),\n", "prefixes": [] }