From patchwork Fri Dec 14 15:38:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Maximets X-Patchwork-Id: 48886 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B59A21B5F1; Fri, 14 Dec 2018 16:38:21 +0100 (CET) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 23E651B591 for ; Fri, 14 Dec 2018 16:38:19 +0100 (CET) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20181214153819euoutp02b48b6ad011416ebc978f9f7e8a0bcfcb~wPSR9ZUyO1976319763euoutp02R for ; Fri, 14 Dec 2018 15:38:19 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20181214153819euoutp02b48b6ad011416ebc978f9f7e8a0bcfcb~wPSR9ZUyO1976319763euoutp02R DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1544801899; bh=1bAVMWf08ht+tcJ8FPitdMgYhsZdc40dx9mkqGgl9U0=; h=From:To:Cc:Subject:Date:References:From; b=dPxO7sB+35Rha9P2GgDff0MFxVnzVXkZCQTxlXTNTj5uqMbOzq5fK/qFs3KpTT1g1 4CCxY3IjJPINVma9p2osAggu5JCku0gqOjuoJA59aoyztngFuT03B8Apw72YvOYyn0 fHXO/+O//csmWz+8UKusSBiWnnMPs1jpvxztPLVI= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20181214153818eucas1p2cbee1ebc0908f1ce31972a54428bf734~wPSRhhcr63176331763eucas1p22; Fri, 14 Dec 2018 15:38:18 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 2A.C6.04294.A6EC31C5; Fri, 14 Dec 2018 15:38:18 +0000 (GMT) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20181214153817eucas1p19a41cdd791879252e1f3a5d77c427845~wPSQ0B7gs0231502315eucas1p1s; Fri, 14 Dec 2018 15:38:17 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20181214153817eusmtrp16d16d12ae45bde545b4b01ea39b9e162~wPSQlSL-12683526835eusmtrp1B; Fri, 14 Dec 2018 15:38:17 +0000 (GMT) X-AuditID: cbfec7f4-84fff700000010c6-81-5c13ce6aa2dc Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id AF.69.04128.96EC31C5; Fri, 14 Dec 2018 15:38:17 +0000 (GMT) Received: from imaximets.rnd.samsung.ru (unknown [106.109.129.180]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20181214153816eusmtip1cb20bde00933486ff5dfeadd98b7c1d3~wPSP381dr2727827278eusmtip16; Fri, 14 Dec 2018 15:38:16 +0000 (GMT) From: Ilya Maximets To: dev@dpdk.org, Maxime Coquelin , "Michael S . Tsirkin" , Xiao Wang Cc: Tiwei Bie , Zhihong Wang , jfreimann@redhat.com, Jason Wang , xiaolong.ye@intel.com, alejandro.lucero@netronome.com, Ilya Maximets Date: Fri, 14 Dec 2018 18:38:12 +0300 Message-Id: <20181214153812.3878-1-i.maximets@samsung.com> X-Mailer: git-send-email 2.17.1 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmphleLIzCtJLcpLzFFi42LZduzned2sc8IxBktmilqc+7SMyeLdp+1M Flfaf7JbLLv0mcni3JqlLBbHOvewWPz/9YrVYmvDfyaL/c8Ps1v8eWNqsfniJCYHbo9fC5ay eize85LJY3r3Q2aP9/uusnn0bVnFGMAaxWWTkpqTWZZapG+XwJVxbk8/c8Ex04p52/qYGxj7 dLoYOTkkBEwkvvS+YOxi5OIQEljBKPF7WSsbSEJI4AujxPJzDhCJz4wSPdN2sMB0PH22hBki sZxR4sWWKSwQzg9GibXrusHa2QR0JE6tPgI2V0Sgm1Hi1dPPbCAOs8BzoJYv99hBqoQFPCV+ PboGZrMIqEo8/d8AZvMKWEl8WD6bEWKfvMTqDQfA9kkIfGaTWHzgGztEwkWid9U3qKOEJV4d 3wIVl5E4PbkHKl4vcb/lJSNEcwejxPRD/5ggEvYSW16fA2rgADpJU2L9Ln2IsKPEv9PXWEDC EgJ8EjfeCoKEmYHMSdumM0OEeSU62oQgqlUkfh9czgxhS0ncfPcZ6gIPiZsbNjBCwjFWYlPD AuYJjHKzEHYtYGRcxSieWlqcm55abJSXWq5XnJhbXJqXrpecn7uJEZg2Tv87/mUH464/SYcY BTgYlXh4MyYJxQixJpYVV+YeYpTgYFYS4Q1rFY4R4k1JrKxKLcqPLyrNSS0+xCjNwaIkzlvN 8CBaSCA9sSQ1OzW1ILUIJsvEwSnVwNj2UudAzIQTqQrNkamTXnFMPtzAxj5Pr+bWHvsIrrjg mVGByy2e55cu/DZl/qM30ssn9N9yLFnD2v4o4p0Ua+u82OvuC7Pmb7PU25VTyRj8PSomRUzk CltWsFV/5d50CZkTZotMVjUaZ/9wPd2jXDhp5fZepeeNgovnP2WKz6qSDw89XiHnrcRSnJFo qMVcVJwIAHZ6meUXAwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrNLMWRmVeSWpSXmKPExsVy+t/xu7qZ54RjDPauUrY492kZk8W7T9uZ LK60/2S3WHbpM5PFuTVLWSyOde5hsfj/6xWrxdaG/0wW+58fZrf488bUYvPFSUwO3B6/Fixl 9Vi85yWTx/Tuh8we7/ddZfPo27KKMYA1Ss+mKL+0JFUhI7+4xFYp2tDCSM/Q0kLPyMRSz9DY PNbKyFRJ384mJTUnsyy1SN8uQS/j3J5+5oJjphXztvUxNzD26XQxcnJICJhIPH22hLmLkYtD SGApo0TP6pesEAkpiR+/LkDZwhJ/rnWxQRR9Y5T4f72LHSTBJqAjcWr1EUaQhIhAP6PE7us3 wUYxC7xllOibdAOsSljAU+LXo2tgNouAqsTT/w1gNq+AlcSH5bMZIVbIS6zecIB5AiPPAkaG VYwiqaXFuem5xUZ6xYm5xaV56XrJ+bmbGIEBu+3Yzy07GLveBR9iFOBgVOLhPTBFKEaINbGs uDL3EKMEB7OSCG9Yq3CMEG9KYmVValF+fFFpTmrxIUZToOUTmaVEk/OB0ZRXEm9oamhuYWlo bmxubGahJM573qAySkggPbEkNTs1tSC1CKaPiYNTqoFRIKCTsVDQqWeO5qpd9VvEeg1PBEUH M+TYGR6aHbXmz2OpRylNAfv99pq23zCO+LRV5ktgqgTTPNZtTRe5rsa6JBZ0LGfvcFsftE5p Y8C/M59mzorQcJa0vKsdURPmuvrE9ZIOvxdpkv/viUSlHDNxubpc0uJh/FyrHa/WFuaf5TXf ZJ6esEiJpTgj0VCLuag4EQCYPXOWbgIAAA== X-CMS-MailID: 20181214153817eucas1p19a41cdd791879252e1f3a5d77c427845 X-Msg-Generator: CA X-RootMTR: 20181214153817eucas1p19a41cdd791879252e1f3a5d77c427845 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20181214153817eucas1p19a41cdd791879252e1f3a5d77c427845 References: Subject: [dpdk-dev] [PATCH] net/virtio: add platform memory ordering feature support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" VIRTIO_F_ORDER_PLATFORM is required to use proper memory barriers in case of HW vhost implementations like vDPA. DMA barriers (rte_cio_*) are sufficent for that purpose. Previously known as VIRTIO_F_IO_BARRIER. Signed-off-by: Ilya Maximets --- Based on "[RFC] net/virtio: use real barriers for vDPA". RFC --> PATCH: * Dropped vendor-specific hack to determine if we need real barriers. * Added VIRTIO_F_ORDER_PLATFORM feature definition and checking. Note: Patch to change the name of the feature from VIRTIO_F_IO_BARRIER to VIRTIO_F_ORDER_PLATFORM is not merged yet: https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg04114.html drivers/net/virtio/virtio_ethdev.c | 2 ++ drivers/net/virtio/virtio_ethdev.h | 3 ++- drivers/net/virtio/virtio_pci.h | 7 ++++++ drivers/net/virtio/virtio_rxtx.c | 14 ++++++------ drivers/net/virtio/virtqueue.h | 35 +++++++++++++++++++++++++----- 5 files changed, 48 insertions(+), 13 deletions(-) diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index cb2b2e0bf..5ae7a9650 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -1474,6 +1474,8 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features) if (virtio_negotiate_features(hw, req_features) < 0) return -1; + hw->weak_barriers = !vtpci_with_feature(hw, VIRTIO_F_ORDER_PLATFORM); + if (!hw->virtio_user_dev) { pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); rte_eth_copy_pci_info(eth_dev, pci_dev); diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h index e0f80e5a4..c098e5ac0 100644 --- a/drivers/net/virtio/virtio_ethdev.h +++ b/drivers/net/virtio/virtio_ethdev.h @@ -34,7 +34,8 @@ 1u << VIRTIO_RING_F_INDIRECT_DESC | \ 1ULL << VIRTIO_F_VERSION_1 | \ 1ULL << VIRTIO_F_IN_ORDER | \ - 1ULL << VIRTIO_F_IOMMU_PLATFORM) + 1ULL << VIRTIO_F_IOMMU_PLATFORM | \ + 1ULL << VIRTIO_F_ORDER_PLATFORM) #define VIRTIO_PMD_SUPPORTED_GUEST_FEATURES \ (VIRTIO_PMD_DEFAULT_GUEST_FEATURES | \ diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h index e961a58ca..e2f096185 100644 --- a/drivers/net/virtio/virtio_pci.h +++ b/drivers/net/virtio/virtio_pci.h @@ -128,6 +128,12 @@ struct virtnet_ctl; */ #define VIRTIO_F_IN_ORDER 35 +/* + * This feature indicates that memory accesses by the driver and the device + * are ordered in a way described by the platform. + */ +#define VIRTIO_F_ORDER_PLATFORM 36 + /* The Guest publishes the used index for which it expects an interrupt * at the end of the avail ring. Host should ignore the avail->flags field. */ /* The Host publishes the avail index for which it expects a kick @@ -240,6 +246,7 @@ struct virtio_hw { uint8_t use_simple_rx; uint8_t use_inorder_rx; uint8_t use_inorder_tx; + uint8_t weak_barriers; bool has_tx_offload; bool has_rx_offload; uint16_t port_id; diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c index cb8f89f18..66195bf47 100644 --- a/drivers/net/virtio/virtio_rxtx.c +++ b/drivers/net/virtio/virtio_rxtx.c @@ -906,7 +906,7 @@ virtio_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); num = likely(nb_used <= nb_pkts) ? nb_used : nb_pkts; if (unlikely(num > VIRTIO_MBUF_BURST_SZ)) @@ -1017,7 +1017,7 @@ virtio_recv_mergeable_pkts_inorder(void *rx_queue, nb_used = RTE_MIN(nb_used, nb_pkts); nb_used = RTE_MIN(nb_used, VIRTIO_MBUF_BURST_SZ); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); PMD_RX_LOG(DEBUG, "used:%d", nb_used); @@ -1202,7 +1202,7 @@ virtio_recv_mergeable_pkts(void *rx_queue, nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); PMD_RX_LOG(DEBUG, "used:%d", nb_used); @@ -1365,7 +1365,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) PMD_TX_LOG(DEBUG, "%d packets to xmit", nb_pkts); nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh)) virtio_xmit_cleanup(vq, nb_used); @@ -1407,7 +1407,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) /* Positive value indicates it need free vring descriptors */ if (unlikely(need > 0)) { nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); need = RTE_MIN(need, (int)nb_used); virtio_xmit_cleanup(vq, need); @@ -1463,7 +1463,7 @@ virtio_xmit_pkts_inorder(void *tx_queue, PMD_TX_LOG(DEBUG, "%d packets to xmit", nb_pkts); nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh)) virtio_xmit_cleanup_inorder(vq, nb_used); @@ -1511,7 +1511,7 @@ virtio_xmit_pkts_inorder(void *tx_queue, need = slots - vq->vq_free_cnt; if (unlikely(need > 0)) { nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); need = RTE_MIN(need, (int)nb_used); virtio_xmit_cleanup_inorder(vq, need); diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h index 26518ed98..6b9055a1f 100644 --- a/drivers/net/virtio/virtqueue.h +++ b/drivers/net/virtio/virtqueue.h @@ -19,15 +19,40 @@ struct rte_mbuf; /* - * Per virtio_config.h in Linux. + * Per virtio_ring.h in Linux. * For virtio_pci on SMP, we don't need to order with respect to MMIO * accesses through relaxed memory I/O windows, so smp_mb() et al are * sufficient. * + * For using virtio to talk to real devices (eg. vDPA) we do need real + * barriers. */ -#define virtio_mb() rte_smp_mb() -#define virtio_rmb() rte_smp_rmb() -#define virtio_wmb() rte_smp_wmb() +static inline void +virtio_mb(uint8_t weak_barriers) +{ + if (weak_barriers) + rte_smp_mb(); + else + rte_mb(); +} + +static inline void +virtio_rmb(uint8_t weak_barriers) +{ + if (weak_barriers) + rte_smp_rmb(); + else + rte_cio_rmb(); +} + +static inline void +virtio_wmb(uint8_t weak_barriers) +{ + if (weak_barriers) + rte_smp_wmb(); + else + rte_cio_wmb(); +} #ifdef RTE_PMD_PACKET_PREFETCH #define rte_packet_prefetch(p) rte_prefetch1(p) @@ -312,7 +337,7 @@ void vq_ring_free_inorder(struct virtqueue *vq, uint16_t desc_idx, static inline void vq_update_avail_idx(struct virtqueue *vq) { - virtio_wmb(); + virtio_wmb(vq->hw->weak_barriers); vq->vq_ring.avail->idx = vq->vq_avail_idx; }