Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/111183/?format=api
{ "id": 111183, "url": "https://patches.dpdk.org/api/patches/111183/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/20220516111041.63914-6-xuan.ding@intel.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20220516111041.63914-6-xuan.ding@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20220516111041.63914-6-xuan.ding@intel.com", "date": "2022-05-16T11:10:41", "name": "[v8,5/5] examples/vhost: support async dequeue data path", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": true, "hash": "3c63bb09d68f28be3ac5f1fcd462f63ac1fd1908", "submitter": { "id": 1401, "url": "https://patches.dpdk.org/api/people/1401/?format=api", "name": "Ding, Xuan", "email": "xuan.ding@intel.com" }, "delegate": { "id": 2642, "url": "https://patches.dpdk.org/api/users/2642/?format=api", "username": "mcoquelin", "first_name": "Maxime", "last_name": "Coquelin", "email": "maxime.coquelin@redhat.com" }, "mbox": "https://patches.dpdk.org/project/dpdk/patch/20220516111041.63914-6-xuan.ding@intel.com/mbox/", "series": [ { "id": 22951, "url": "https://patches.dpdk.org/api/series/22951/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/list/?series=22951", "date": "2022-05-16T11:10:36", "name": "vhost: support async dequeue data path", "version": 8, "mbox": "https://patches.dpdk.org/series/22951/mbox/" } ], "comments": "https://patches.dpdk.org/api/patches/111183/comments/", "check": "success", "checks": "https://patches.dpdk.org/api/patches/111183/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id A6ABDA00BE;\n\tMon, 16 May 2022 13:16:25 +0200 (CEST)", "from [217.70.189.124] (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 9359B42B5A;\n\tMon, 16 May 2022 13:16:25 +0200 (CEST)", "from mga03.intel.com (mga03.intel.com [134.134.136.65])\n by mails.dpdk.org (Postfix) with ESMTP id ECC6942B5A\n for <dev@dpdk.org>; Mon, 16 May 2022 13:16:23 +0200 (CEST)", "from orsmga007.jf.intel.com ([10.7.209.58])\n by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 16 May 2022 04:15:54 -0700", "from npg-dpdk-xuan-cbdma.sh.intel.com ([10.67.110.228])\n by orsmga007.jf.intel.com with ESMTP; 16 May 2022 04:15:51 -0700" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple;\n d=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n t=1652699784; x=1684235784;\n h=from:to:cc:subject:date:message-id:in-reply-to: references;\n bh=XBC2fDe89/vJrc1PsydnHPL1/p/PJ0t9YJp8N1jCvJE=;\n b=gO4KZIcO+Zccmg6wgGxIrvY+hbvbS2SMFXYSecsFHaDYPIERgusgE9Yr\n EoOPQA4ryv1PfVqd9jVS28izUmqc8v0qdKcX+AM4yTE3gdUSfI+VKcd8r\n o3sMLDZbaAqMjDzSeBNad9hG7+Wuc/JkmdRuvI6WOdMfq2VrMB8DWkC1u\n t5ETB6h6PMMAoPOuhZKAdUSvc5x6JI82oIT+tU6k5EgYF66J/K0rtj0kk\n ffo257VeuZ+iwgrC9ylrAn2lvnEbD1UTKtRA9GKn1HxQO/L4hN2Oxf0ty\n /RDQJpGOdkpRk4Cocw5CFdqrPHGOVQN4SuoTW1WG5IPCI/lE68hoZ7+Hu Q==;", "X-IronPort-AV": [ "E=McAfee;i=\"6400,9594,10348\"; a=\"270752330\"", "E=Sophos;i=\"5.91,229,1647327600\"; d=\"scan'208\";a=\"270752330\"", "E=Sophos;i=\"5.91,229,1647327600\"; d=\"scan'208\";a=\"568272441\"" ], "X-ExtLoop1": "1", "From": "xuan.ding@intel.com", "To": "maxime.coquelin@redhat.com,\n\tchenbo.xia@intel.com", "Cc": "dev@dpdk.org, jiayu.hu@intel.com, cheng1.jiang@intel.com,\n sunil.pai.g@intel.com, liangma@liangbit.com,\n Xuan Ding <xuan.ding@intel.com>, Wenwu Ma <wenwux.ma@intel.com>,\n Yuan Wang <yuanx.wang@intel.com>", "Subject": "[PATCH v8 5/5] examples/vhost: support async dequeue data path", "Date": "Mon, 16 May 2022 11:10:41 +0000", "Message-Id": "<20220516111041.63914-6-xuan.ding@intel.com>", "X-Mailer": "git-send-email 2.17.1", "In-Reply-To": "<20220516111041.63914-1-xuan.ding@intel.com>", "References": "<20220407152546.38167-1-xuan.ding@intel.com>\n <20220516111041.63914-1-xuan.ding@intel.com>", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "From: Xuan Ding <xuan.ding@intel.com>\n\nThis patch adds the use case for async dequeue API. Vswitch can\nleverage DMA device to accelerate vhost async dequeue path.\n\nSigned-off-by: Wenwu Ma <wenwux.ma@intel.com>\nSigned-off-by: Yuan Wang <yuanx.wang@intel.com>\nSigned-off-by: Xuan Ding <xuan.ding@intel.com>\nTested-by: Yvonne Yang <yvonnex.yang@intel.com>\nReviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>\n---\n doc/guides/sample_app_ug/vhost.rst | 9 +-\n examples/vhost/main.c | 286 ++++++++++++++++++++---------\n examples/vhost/main.h | 32 +++-\n examples/vhost/virtio_net.c | 16 +-\n 4 files changed, 245 insertions(+), 98 deletions(-)", "diff": "diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst\nindex a6ce4bc8ac..09db965e70 100644\n--- a/doc/guides/sample_app_ug/vhost.rst\n+++ b/doc/guides/sample_app_ug/vhost.rst\n@@ -169,9 +169,12 @@ demonstrates how to use the async vhost APIs. It's used in combination with dmas\n **--dmas**\n This parameter is used to specify the assigned DMA device of a vhost device.\n Async vhost-user net driver will be used if --dmas is set. For example\n---dmas [txd0@00:04.0,txd1@00:04.1] means use DMA channel 00:04.0 for vhost\n-device 0 enqueue operation and use DMA channel 00:04.1 for vhost device 1\n-enqueue operation.\n+--dmas [txd0@00:04.0,txd1@00:04.1,rxd0@00:04.2,rxd1@00:04.3] means use\n+DMA channel 00:04.0/00:04.2 for vhost device 0 enqueue/dequeue operation\n+and use DMA channel 00:04.1/00:04.3 for vhost device 1 enqueue/dequeue\n+operation. The index of the device corresponds to the socket file in order,\n+that means vhost device 0 is created through the first socket file, vhost\n+device 1 is created through the second socket file, and so on.\n \n Common Issues\n -------------\ndiff --git a/examples/vhost/main.c b/examples/vhost/main.c\nindex c4d46de1c5..5bc34b0c52 100644\n--- a/examples/vhost/main.c\n+++ b/examples/vhost/main.c\n@@ -63,6 +63,9 @@\n \n #define DMA_RING_SIZE 4096\n \n+#define ASYNC_ENQUEUE_VHOST 1\n+#define ASYNC_DEQUEUE_VHOST 2\n+\n /* number of mbufs in all pools - if specified on command-line. */\n static int total_num_mbufs = NUM_MBUFS_DEFAULT;\n \n@@ -116,6 +119,8 @@ static uint32_t burst_rx_retry_num = BURST_RX_RETRIES;\n static char *socket_files;\n static int nb_sockets;\n \n+static struct vhost_queue_ops vdev_queue_ops[RTE_MAX_VHOST_DEVICE];\n+\n /* empty VMDq configuration structure. Filled in programmatically */\n static struct rte_eth_conf vmdq_conf_default = {\n \t.rxmode = {\n@@ -205,6 +210,20 @@ struct vhost_bufftable *vhost_txbuff[RTE_MAX_LCORE * RTE_MAX_VHOST_DEVICE];\n #define MBUF_TABLE_DRAIN_TSC\t((rte_get_tsc_hz() + US_PER_S - 1) \\\n \t\t\t\t / US_PER_S * BURST_TX_DRAIN_US)\n \n+static int vid2socketid[RTE_MAX_VHOST_DEVICE];\n+\n+static inline uint32_t\n+get_async_flag_by_socketid(int socketid)\n+{\n+\treturn dma_bind[socketid].async_flag;\n+}\n+\n+static inline void\n+init_vid2socketid_array(int vid, int socketid)\n+{\n+\tvid2socketid[vid] = socketid;\n+}\n+\n static inline bool\n is_dma_configured(int16_t dev_id)\n {\n@@ -224,7 +243,7 @@ open_dma(const char *value)\n \tchar *addrs = input;\n \tchar *ptrs[2];\n \tchar *start, *end, *substr;\n-\tint64_t vid;\n+\tint64_t socketid, vring_id;\n \n \tstruct rte_dma_info info;\n \tstruct rte_dma_conf dev_config = { .nb_vchans = 1 };\n@@ -262,7 +281,9 @@ open_dma(const char *value)\n \n \twhile (i < args_nr) {\n \t\tchar *arg_temp = dma_arg[i];\n+\t\tchar *txd, *rxd;\n \t\tuint8_t sub_nr;\n+\t\tint async_flag;\n \n \t\tsub_nr = rte_strsplit(arg_temp, strlen(arg_temp), ptrs, 2, '@');\n \t\tif (sub_nr != 2) {\n@@ -270,14 +291,23 @@ open_dma(const char *value)\n \t\t\tgoto out;\n \t\t}\n \n-\t\tstart = strstr(ptrs[0], \"txd\");\n-\t\tif (start == NULL) {\n+\t\ttxd = strstr(ptrs[0], \"txd\");\n+\t\trxd = strstr(ptrs[0], \"rxd\");\n+\t\tif (txd) {\n+\t\t\tstart = txd;\n+\t\t\tvring_id = VIRTIO_RXQ;\n+\t\t\tasync_flag = ASYNC_ENQUEUE_VHOST;\n+\t\t} else if (rxd) {\n+\t\t\tstart = rxd;\n+\t\t\tvring_id = VIRTIO_TXQ;\n+\t\t\tasync_flag = ASYNC_DEQUEUE_VHOST;\n+\t\t} else {\n \t\t\tret = -1;\n \t\t\tgoto out;\n \t\t}\n \n \t\tstart += 3;\n-\t\tvid = strtol(start, &end, 0);\n+\t\tsocketid = strtol(start, &end, 0);\n \t\tif (end == start) {\n \t\t\tret = -1;\n \t\t\tgoto out;\n@@ -338,7 +368,8 @@ open_dma(const char *value)\n \t\tdmas_id[dma_count++] = dev_id;\n \n done:\n-\t\t(dma_info + vid)->dmas[VIRTIO_RXQ].dev_id = dev_id;\n+\t\t(dma_info + socketid)->dmas[vring_id].dev_id = dev_id;\n+\t\t(dma_info + socketid)->async_flag |= async_flag;\n \t\ti++;\n \t}\n out:\n@@ -990,7 +1021,7 @@ complete_async_pkts(struct vhost_dev *vdev)\n {\n \tstruct rte_mbuf *p_cpl[MAX_PKT_BURST];\n \tuint16_t complete_count;\n-\tint16_t dma_id = dma_bind[vdev->vid].dmas[VIRTIO_RXQ].dev_id;\n+\tint16_t dma_id = dma_bind[vid2socketid[vdev->vid]].dmas[VIRTIO_RXQ].dev_id;\n \n \tcomplete_count = rte_vhost_poll_enqueue_completed(vdev->vid,\n \t\t\t\t\tVIRTIO_RXQ, p_cpl, MAX_PKT_BURST, dma_id, 0);\n@@ -1029,22 +1060,7 @@ drain_vhost(struct vhost_dev *vdev)\n \tuint16_t nr_xmit = vhost_txbuff[buff_idx]->len;\n \tstruct rte_mbuf **m = vhost_txbuff[buff_idx]->m_table;\n \n-\tif (builtin_net_driver) {\n-\t\tret = vs_enqueue_pkts(vdev, VIRTIO_RXQ, m, nr_xmit);\n-\t} else if (dma_bind[vdev->vid].dmas[VIRTIO_RXQ].async_enabled) {\n-\t\tuint16_t enqueue_fail = 0;\n-\t\tint16_t dma_id = dma_bind[vdev->vid].dmas[VIRTIO_RXQ].dev_id;\n-\n-\t\tcomplete_async_pkts(vdev);\n-\t\tret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ, m, nr_xmit, dma_id, 0);\n-\n-\t\tenqueue_fail = nr_xmit - ret;\n-\t\tif (enqueue_fail)\n-\t\t\tfree_pkts(&m[ret], nr_xmit - ret);\n-\t} else {\n-\t\tret = rte_vhost_enqueue_burst(vdev->vid, VIRTIO_RXQ,\n-\t\t\t\t\t\tm, nr_xmit);\n-\t}\n+\tret = vdev_queue_ops[vdev->vid].enqueue_pkt_burst(vdev, VIRTIO_RXQ, m, nr_xmit);\n \n \tif (enable_stats) {\n \t\t__atomic_add_fetch(&vdev->stats.rx_total_atomic, nr_xmit,\n@@ -1053,7 +1069,7 @@ drain_vhost(struct vhost_dev *vdev)\n \t\t\t\t__ATOMIC_SEQ_CST);\n \t}\n \n-\tif (!dma_bind[vdev->vid].dmas[VIRTIO_RXQ].async_enabled)\n+\tif (!dma_bind[vid2socketid[vdev->vid]].dmas[VIRTIO_RXQ].async_enabled)\n \t\tfree_pkts(m, nr_xmit);\n }\n \n@@ -1325,6 +1341,32 @@ drain_mbuf_table(struct mbuf_table *tx_q)\n \t}\n }\n \n+uint16_t\n+async_enqueue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\tstruct rte_mbuf **pkts, uint32_t rx_count)\n+{\n+\tuint16_t enqueue_count;\n+\tuint16_t enqueue_fail = 0;\n+\tuint16_t dma_id = dma_bind[vid2socketid[dev->vid]].dmas[VIRTIO_RXQ].dev_id;\n+\n+\tcomplete_async_pkts(dev);\n+\tenqueue_count = rte_vhost_submit_enqueue_burst(dev->vid, queue_id,\n+\t\t\t\t\tpkts, rx_count, dma_id, 0);\n+\n+\tenqueue_fail = rx_count - enqueue_count;\n+\tif (enqueue_fail)\n+\t\tfree_pkts(&pkts[enqueue_count], enqueue_fail);\n+\n+\treturn enqueue_count;\n+}\n+\n+uint16_t\n+sync_enqueue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\tstruct rte_mbuf **pkts, uint32_t rx_count)\n+{\n+\treturn rte_vhost_enqueue_burst(dev->vid, queue_id, pkts, rx_count);\n+}\n+\n static __rte_always_inline void\n drain_eth_rx(struct vhost_dev *vdev)\n {\n@@ -1355,25 +1397,8 @@ drain_eth_rx(struct vhost_dev *vdev)\n \t\t}\n \t}\n \n-\tif (builtin_net_driver) {\n-\t\tenqueue_count = vs_enqueue_pkts(vdev, VIRTIO_RXQ,\n-\t\t\t\t\t\tpkts, rx_count);\n-\t} else if (dma_bind[vdev->vid].dmas[VIRTIO_RXQ].async_enabled) {\n-\t\tuint16_t enqueue_fail = 0;\n-\t\tint16_t dma_id = dma_bind[vdev->vid].dmas[VIRTIO_RXQ].dev_id;\n-\n-\t\tcomplete_async_pkts(vdev);\n-\t\tenqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,\n-\t\t\t\t\tVIRTIO_RXQ, pkts, rx_count, dma_id, 0);\n-\n-\t\tenqueue_fail = rx_count - enqueue_count;\n-\t\tif (enqueue_fail)\n-\t\t\tfree_pkts(&pkts[enqueue_count], enqueue_fail);\n-\n-\t} else {\n-\t\tenqueue_count = rte_vhost_enqueue_burst(vdev->vid, VIRTIO_RXQ,\n-\t\t\t\t\t\tpkts, rx_count);\n-\t}\n+\tenqueue_count = vdev_queue_ops[vdev->vid].enqueue_pkt_burst(vdev,\n+\t\t\t\t\tVIRTIO_RXQ, pkts, rx_count);\n \n \tif (enable_stats) {\n \t\t__atomic_add_fetch(&vdev->stats.rx_total_atomic, rx_count,\n@@ -1382,10 +1407,31 @@ drain_eth_rx(struct vhost_dev *vdev)\n \t\t\t\t__ATOMIC_SEQ_CST);\n \t}\n \n-\tif (!dma_bind[vdev->vid].dmas[VIRTIO_RXQ].async_enabled)\n+\tif (!dma_bind[vid2socketid[vdev->vid]].dmas[VIRTIO_RXQ].async_enabled)\n \t\tfree_pkts(pkts, rx_count);\n }\n \n+uint16_t async_dequeue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\t\t struct rte_mempool *mbuf_pool,\n+\t\t\t struct rte_mbuf **pkts, uint16_t count)\n+{\n+\tint nr_inflight;\n+\tuint16_t dequeue_count;\n+\tint16_t dma_id = dma_bind[vid2socketid[dev->vid]].dmas[VIRTIO_TXQ].dev_id;\n+\n+\tdequeue_count = rte_vhost_async_try_dequeue_burst(dev->vid, queue_id,\n+\t\t\tmbuf_pool, pkts, count, &nr_inflight, dma_id, 0);\n+\n+\treturn dequeue_count;\n+}\n+\n+uint16_t sync_dequeue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\t\t struct rte_mempool *mbuf_pool,\n+\t\t\t struct rte_mbuf **pkts, uint16_t count)\n+{\n+\treturn rte_vhost_dequeue_burst(dev->vid, queue_id, mbuf_pool, pkts, count);\n+}\n+\n static __rte_always_inline void\n drain_virtio_tx(struct vhost_dev *vdev)\n {\n@@ -1393,13 +1439,8 @@ drain_virtio_tx(struct vhost_dev *vdev)\n \tuint16_t count;\n \tuint16_t i;\n \n-\tif (builtin_net_driver) {\n-\t\tcount = vs_dequeue_pkts(vdev, VIRTIO_TXQ, mbuf_pool,\n-\t\t\t\t\tpkts, MAX_PKT_BURST);\n-\t} else {\n-\t\tcount = rte_vhost_dequeue_burst(vdev->vid, VIRTIO_TXQ,\n-\t\t\t\t\tmbuf_pool, pkts, MAX_PKT_BURST);\n-\t}\n+\tcount = vdev_queue_ops[vdev->vid].dequeue_pkt_burst(vdev,\n+\t\t\t\tVIRTIO_TXQ, mbuf_pool, pkts, MAX_PKT_BURST);\n \n \t/* setup VMDq for the first packet */\n \tif (unlikely(vdev->ready == DEVICE_MAC_LEARNING) && count) {\n@@ -1478,6 +1519,26 @@ switch_worker(void *arg __rte_unused)\n \treturn 0;\n }\n \n+static void\n+vhost_clear_queue_thread_unsafe(struct vhost_dev *vdev, uint16_t queue_id)\n+{\n+\tuint16_t n_pkt = 0;\n+\tint pkts_inflight;\n+\n+\tint16_t dma_id = dma_bind[vid2socketid[vdev->vid]].dmas[queue_id].dev_id;\n+\tpkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vdev->vid, queue_id);\n+\n+\tstruct rte_mbuf *m_cpl[pkts_inflight];\n+\n+\twhile (pkts_inflight) {\n+\t\tn_pkt = rte_vhost_clear_queue_thread_unsafe(vdev->vid, queue_id, m_cpl,\n+\t\t\t\t\t\t\tpkts_inflight, dma_id, 0);\n+\t\tfree_pkts(m_cpl, n_pkt);\n+\t\tpkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vdev->vid,\n+\t\t\t\t\t\t\t\t\tqueue_id);\n+\t}\n+}\n+\n /*\n * Remove a device from the specific data core linked list and from the\n * main linked list. Synchronization occurs through the use of the\n@@ -1535,27 +1596,79 @@ destroy_device(int vid)\n \t\tvdev->vid);\n \n \tif (dma_bind[vid].dmas[VIRTIO_RXQ].async_enabled) {\n-\t\tuint16_t n_pkt = 0;\n-\t\tint pkts_inflight;\n-\t\tint16_t dma_id = dma_bind[vid].dmas[VIRTIO_RXQ].dev_id;\n-\t\tpkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, VIRTIO_RXQ);\n-\t\tstruct rte_mbuf *m_cpl[pkts_inflight];\n-\n-\t\twhile (pkts_inflight) {\n-\t\t\tn_pkt = rte_vhost_clear_queue_thread_unsafe(vid, VIRTIO_RXQ,\n-\t\t\t\t\t\tm_cpl, pkts_inflight, dma_id, 0);\n-\t\t\tfree_pkts(m_cpl, n_pkt);\n-\t\t\tpkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid,\n-\t\t\t\t\t\t\t\t\t\tVIRTIO_RXQ);\n-\t\t}\n-\n+\t\tvhost_clear_queue_thread_unsafe(vdev, VIRTIO_RXQ);\n \t\trte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);\n \t\tdma_bind[vid].dmas[VIRTIO_RXQ].async_enabled = false;\n \t}\n \n+\tif (dma_bind[vid].dmas[VIRTIO_TXQ].async_enabled) {\n+\t\tvhost_clear_queue_thread_unsafe(vdev, VIRTIO_TXQ);\n+\t\trte_vhost_async_channel_unregister(vid, VIRTIO_TXQ);\n+\t\tdma_bind[vid].dmas[VIRTIO_TXQ].async_enabled = false;\n+\t}\n+\n \trte_free(vdev);\n }\n \n+static inline int\n+get_socketid_by_vid(int vid)\n+{\n+\tint i;\n+\tchar ifname[PATH_MAX];\n+\trte_vhost_get_ifname(vid, ifname, sizeof(ifname));\n+\n+\tfor (i = 0; i < nb_sockets; i++) {\n+\t\tchar *file = socket_files + i * PATH_MAX;\n+\t\tif (strcmp(file, ifname) == 0)\n+\t\t\treturn i;\n+\t}\n+\n+\treturn -1;\n+}\n+\n+static int\n+init_vhost_queue_ops(int vid)\n+{\n+\tif (builtin_net_driver) {\n+\t\tvdev_queue_ops[vid].enqueue_pkt_burst = builtin_enqueue_pkts;\n+\t\tvdev_queue_ops[vid].dequeue_pkt_burst = builtin_dequeue_pkts;\n+\t} else {\n+\t\tif (dma_bind[vid2socketid[vid]].dmas[VIRTIO_RXQ].async_enabled)\n+\t\t\tvdev_queue_ops[vid].enqueue_pkt_burst = async_enqueue_pkts;\n+\t\telse\n+\t\t\tvdev_queue_ops[vid].enqueue_pkt_burst = sync_enqueue_pkts;\n+\n+\t\tif (dma_bind[vid2socketid[vid]].dmas[VIRTIO_TXQ].async_enabled)\n+\t\t\tvdev_queue_ops[vid].dequeue_pkt_burst = async_dequeue_pkts;\n+\t\telse\n+\t\t\tvdev_queue_ops[vid].dequeue_pkt_burst = sync_dequeue_pkts;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static inline int\n+vhost_async_channel_register(int vid)\n+{\n+\tint rx_ret = 0, tx_ret = 0;\n+\n+\tif (dma_bind[vid2socketid[vid]].dmas[VIRTIO_RXQ].dev_id != INVALID_DMA_ID) {\n+\t\trx_ret = rte_vhost_async_channel_register(vid, VIRTIO_RXQ);\n+\t\tif (rx_ret == 0)\n+\t\t\tdma_bind[vid2socketid[vid]].dmas[VIRTIO_RXQ].async_enabled = true;\n+\t}\n+\n+\tif (dma_bind[vid2socketid[vid]].dmas[VIRTIO_TXQ].dev_id != INVALID_DMA_ID) {\n+\t\ttx_ret = rte_vhost_async_channel_register(vid, VIRTIO_TXQ);\n+\t\tif (tx_ret == 0)\n+\t\t\tdma_bind[vid2socketid[vid]].dmas[VIRTIO_TXQ].async_enabled = true;\n+\t}\n+\n+\treturn rx_ret | tx_ret;\n+}\n+\n+\n+\n /*\n * A new device is added to a data core. First the device is added to the main linked list\n * and then allocated to a specific data core.\n@@ -1567,6 +1680,8 @@ new_device(int vid)\n \tuint16_t i;\n \tuint32_t device_num_min = num_devices;\n \tstruct vhost_dev *vdev;\n+\tint ret;\n+\n \tvdev = rte_zmalloc(\"vhost device\", sizeof(*vdev), RTE_CACHE_LINE_SIZE);\n \tif (vdev == NULL) {\n \t\tRTE_LOG(INFO, VHOST_DATA,\n@@ -1589,6 +1704,17 @@ new_device(int vid)\n \t\t}\n \t}\n \n+\tint socketid = get_socketid_by_vid(vid);\n+\tif (socketid == -1)\n+\t\treturn -1;\n+\n+\tinit_vid2socketid_array(vid, socketid);\n+\n+\tret = vhost_async_channel_register(vid);\n+\n+\tif (init_vhost_queue_ops(vid) != 0)\n+\t\treturn -1;\n+\n \tif (builtin_net_driver)\n \t\tvs_vhost_net_setup(vdev);\n \n@@ -1620,16 +1746,7 @@ new_device(int vid)\n \t\t\"(%d) device has been added to data core %d\\n\",\n \t\tvid, vdev->coreid);\n \n-\tif (dma_bind[vid].dmas[VIRTIO_RXQ].dev_id != INVALID_DMA_ID) {\n-\t\tint ret;\n-\n-\t\tret = rte_vhost_async_channel_register(vid, VIRTIO_RXQ);\n-\t\tif (ret == 0)\n-\t\t\tdma_bind[vid].dmas[VIRTIO_RXQ].async_enabled = true;\n-\t\treturn ret;\n-\t}\n-\n-\treturn 0;\n+\treturn ret;\n }\n \n static int\n@@ -1647,22 +1764,9 @@ vring_state_changed(int vid, uint16_t queue_id, int enable)\n \tif (queue_id != VIRTIO_RXQ)\n \t\treturn 0;\n \n-\tif (dma_bind[vid].dmas[queue_id].async_enabled) {\n-\t\tif (!enable) {\n-\t\t\tuint16_t n_pkt = 0;\n-\t\t\tint pkts_inflight;\n-\t\t\tpkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);\n-\t\t\tint16_t dma_id = dma_bind[vid].dmas[VIRTIO_RXQ].dev_id;\n-\t\t\tstruct rte_mbuf *m_cpl[pkts_inflight];\n-\n-\t\t\twhile (pkts_inflight) {\n-\t\t\t\tn_pkt = rte_vhost_clear_queue_thread_unsafe(vid, queue_id,\n-\t\t\t\t\t\t\tm_cpl, pkts_inflight, dma_id, 0);\n-\t\t\t\tfree_pkts(m_cpl, n_pkt);\n-\t\t\t\tpkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid,\n-\t\t\t\t\t\t\t\t\t\t\tqueue_id);\n-\t\t\t}\n-\t\t}\n+\tif (dma_bind[vid2socketid[vid]].dmas[queue_id].async_enabled) {\n+\t\tif (!enable)\n+\t\t\tvhost_clear_queue_thread_unsafe(vdev, queue_id);\n \t}\n \n \treturn 0;\n@@ -1887,7 +1991,7 @@ main(int argc, char *argv[])\n \tfor (i = 0; i < nb_sockets; i++) {\n \t\tchar *file = socket_files + i * PATH_MAX;\n \n-\t\tif (dma_count)\n+\t\tif (dma_count && get_async_flag_by_socketid(i) != 0)\n \t\t\tflags = flags | RTE_VHOST_USER_ASYNC_COPY;\n \n \t\tret = rte_vhost_driver_register(file, flags);\ndiff --git a/examples/vhost/main.h b/examples/vhost/main.h\nindex e7f395c3c9..2fcb8376c5 100644\n--- a/examples/vhost/main.h\n+++ b/examples/vhost/main.h\n@@ -61,6 +61,19 @@ struct vhost_dev {\n \tstruct vhost_queue queues[MAX_QUEUE_PAIRS * 2];\n } __rte_cache_aligned;\n \n+typedef uint16_t (*vhost_enqueue_burst_t)(struct vhost_dev *dev,\n+\t\t\tuint16_t queue_id, struct rte_mbuf **pkts,\n+\t\t\tuint32_t count);\n+\n+typedef uint16_t (*vhost_dequeue_burst_t)(struct vhost_dev *dev,\n+\t\t\tuint16_t queue_id, struct rte_mempool *mbuf_pool,\n+\t\t\tstruct rte_mbuf **pkts, uint16_t count);\n+\n+struct vhost_queue_ops {\n+\tvhost_enqueue_burst_t enqueue_pkt_burst;\n+\tvhost_dequeue_burst_t dequeue_pkt_burst;\n+};\n+\n TAILQ_HEAD(vhost_dev_tailq_list, vhost_dev);\n \n \n@@ -87,6 +100,7 @@ struct dma_info {\n \n struct dma_for_vhost {\n \tstruct dma_info dmas[RTE_MAX_QUEUES_PER_PORT * 2];\n+\tuint32_t async_flag;\n };\n \n /* we implement non-extra virtio net features */\n@@ -97,7 +111,19 @@ void vs_vhost_net_remove(struct vhost_dev *dev);\n uint16_t vs_enqueue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n \t\t\t struct rte_mbuf **pkts, uint32_t count);\n \n-uint16_t vs_dequeue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n-\t\t\t struct rte_mempool *mbuf_pool,\n-\t\t\t struct rte_mbuf **pkts, uint16_t count);\n+uint16_t builtin_enqueue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\t\tstruct rte_mbuf **pkts, uint32_t count);\n+uint16_t builtin_dequeue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\t\tstruct rte_mempool *mbuf_pool,\n+\t\t\tstruct rte_mbuf **pkts, uint16_t count);\n+uint16_t sync_enqueue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\t\t struct rte_mbuf **pkts, uint32_t count);\n+uint16_t sync_dequeue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\t\tstruct rte_mempool *mbuf_pool,\n+\t\t\tstruct rte_mbuf **pkts, uint16_t count);\n+uint16_t async_enqueue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\t\t struct rte_mbuf **pkts, uint32_t count);\n+uint16_t async_dequeue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\t\tstruct rte_mempool *mbuf_pool,\n+\t\t\tstruct rte_mbuf **pkts, uint16_t count);\n #endif /* _MAIN_H_ */\ndiff --git a/examples/vhost/virtio_net.c b/examples/vhost/virtio_net.c\nindex 9064fc3a82..2432a96566 100644\n--- a/examples/vhost/virtio_net.c\n+++ b/examples/vhost/virtio_net.c\n@@ -238,6 +238,13 @@ vs_enqueue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n \treturn count;\n }\n \n+uint16_t\n+builtin_enqueue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\t\tstruct rte_mbuf **pkts, uint32_t count)\n+{\n+\treturn vs_enqueue_pkts(dev, queue_id, pkts, count);\n+}\n+\n static __rte_always_inline int\n dequeue_pkt(struct vhost_dev *dev, struct rte_vhost_vring *vr,\n \t struct rte_mbuf *m, uint16_t desc_idx,\n@@ -363,7 +370,7 @@ dequeue_pkt(struct vhost_dev *dev, struct rte_vhost_vring *vr,\n \treturn 0;\n }\n \n-uint16_t\n+static uint16_t\n vs_dequeue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n \tstruct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count)\n {\n@@ -440,3 +447,10 @@ vs_dequeue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n \n \treturn i;\n }\n+\n+uint16_t\n+builtin_dequeue_pkts(struct vhost_dev *dev, uint16_t queue_id,\n+\tstruct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count)\n+{\n+\treturn vs_dequeue_pkts(dev, queue_id, mbuf_pool, pkts, count);\n+}\n", "prefixes": [ "v8", "5/5" ] }