From patchwork Wed Oct 25 00:27:25 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongseok Koh X-Patchwork-Id: 30841 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 83D911B808; Wed, 25 Oct 2017 02:27:47 +0200 (CEST) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00063.outbound.protection.outlook.com [40.107.0.63]) by dpdk.org (Postfix) with ESMTP id 81DD71B2F0; Wed, 25 Oct 2017 02:27:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=pTo8tGPp7eVASVwWNxipM+IaTGOxBxcXGIN8/HpOAmM=; b=sEhfdscnbP/v0MQPNsH0nbDn5MOCGfkR1zocacZ6LZqe1Y5s2URfEhUIm6t81cHFZRqhbgiBI7lFR1z28AakDzHBPbafbfd92Wk0ir2b+WzGhOgDpdVNOmsQRI45tHW21CT+K1ac28roBcqCJuFdWvE9/GQu6nMyfXyJ6S/iBBM= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from mellanox.com (209.116.155.178) by HE1PR0501MB2042.eurprd05.prod.outlook.com (2603:10a6:3:35::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.178.6; Wed, 25 Oct 2017 00:27:39 +0000 From: Yongseok Koh To: adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com Cc: dev@dpdk.org, Yongseok Koh , stable@dpdk.org, Sagi Grimberg , Alexander Solganik Date: Tue, 24 Oct 2017 17:27:25 -0700 Message-Id: <49c39b44917c35ecaabf06f5f920d0f7e0ed0b6b.1508891141.git.yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171022080022.13528-1-yskoh@mellanox.com> References: <20171022080022.13528-1-yskoh@mellanox.com> MIME-Version: 1.0 X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: YQXPR0101CA0059.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:14::36) To HE1PR0501MB2042.eurprd05.prod.outlook.com (2603:10a6:3:35::20) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aaca270b-56e6-48bc-2897-08d51b3f33e0 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(48565401081)(4534020)(4602075)(4627075)(201703031133081)(201702281549075)(2017052603199); SRVR:HE1PR0501MB2042; X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2042; 3:rCmF+qLSMxJ+mSrfKQOzrVLepVfcko4nU8hl/QBEjRGQFIyftaw869+b5oPkyt6XlxuSjPTq+FGOHYbLqULaBu+Tz3DNqai3yn9QYqweMIAxiAxIDTUJriphB8grR+xyQmc06jBLKLomfZ14HIBn+egfaVy3//t3tLqk9WMSMB3gAimGqhH3c3nWpNQaHokF7NmgCr9VQz4Cpn1hC+uNuOsKHJzY5AkAenbcG6483mEqqK0L5pcp+SbCzv/jwHd8; 25:a3PZrM/pbW/csTOvqvyvRAsd0/sVndkBwOgFik1DJ+K+i3LMbkXn3DIzTSWxSqZTYDbaMT0g1S/kblLFOpcO6DSk1e97ATwQilm9BzbGRBK8D0Biu/b3a//9LSk8D3yZuxR0oq+K3B95NZvgqGQTzDlKHaEk8fPA04E1c3lCbUi7KRKH1/WqUSSNA6Tg562MZB07wtoiD4o/r+gYvjIE2pcIE6N7fWkdiplK1Ix7TmiXnAmA2RwzcCkY0j810MhzhoQ5wSoJKPEMfJ/nweC17CJNepYL44P+4l6VJodAA3QAWDKG43JVbSNY+dWDthaBNPT2S77soM4oUQMkQcuxKlGWcHaZtR8bWcMxSDVZDlM=; 31:KOvQDxnbA9Gijf8lDcsLVmcUjvIA0cT1OX/t3iF2hKU1XWEmZbo+LluEcp5x1aYfQjCdwkwzoLPd6JKyRnQQF3gvVfHbHHKG1hRMsVgovY4+QGSYxPthT01sEVypexWQxNlW9IP5z6AnrgwbJdaKKeDaIvszZGg8evnBJJ+ujnHHljafF46vF1Tufk4rIjPBGm4ZswXkYGchJHpoIJJkH6+NbXpfsrHqab7Tw0wPz0c= X-MS-TrafficTypeDiagnostic: HE1PR0501MB2042: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2042; 20:BBfyMgQecVUBMGVV+N5dEiFpbxPL+I8cxqg6ghM3ItAbZJ1kYM4BCwmjE6NWrzUl22Ia7Ox8BskDbBvfHdElMHOzfBmQ4FQuewgs3qoxvLNp7Ak74DDcnmcbWNyBU/1dKHuArU/8BUJ7NfQPpgpMFn/h1/cnvI5H7lyYKjJoaOHQdus2jupyoHALVEktMIzvlGHgK3qlKTRaNExlPMtfaHKiz5D3/1ZUK8C++7uXQZ9HXomHKZwTeiP2aMCevtbPe1m+BRy4Or+AT9qnPR7C6Z8WWiir1wmnhExNrJNLQIEngQ0bXQN7VZ5kfdQPNMYK/Kc3QYvE9mqBbnMjMd7qch8gnkuL6CHvyXHLol9/qZBjQRpB3gEObn9kwRRbSuZ7RhpJM9YHvlMjwfzNWQccYUylmQMa8CQgIr2qWOM3Cq4TXg6v4tamQT6a9UolrPbPKYUb7HOU3mz05CJxdCa3sYfrPAAR1xkdBLsuMC10mBUugeoZ1SA21T78LzuM3VPK; 4:j3mmD/IR4PvbNJjNP14vsAoSdWzc3OEAr/inERNzKU3+lG3MDEPK5czGUiMKp6H2b7OImlfEIewhkakJoxIUtmIHXeo8qIoyhFZENgiM/dXw9HgIubycBWSW0Q0YeINaHXWNUM2vMSnbReoehiftw014/H/VAER9g3k3pKVH7z2sP1MZ5Ra5mXgxoyuBj9yFijJspks9kXCcig5SM0prasbkCfl+iBFXlQDnodFlHWXZhAtUA8NjRl/WM5ppUYr21dRkCg4upU6b15yfZk09gw== X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(93001095)(100000703101)(100105400095)(3002001)(10201501046)(3231020)(6055026)(6041248)(20161123564025)(20161123558100)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:HE1PR0501MB2042; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:HE1PR0501MB2042; X-Forefront-PRVS: 0471B73328 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(39860400002)(376002)(199003)(189002)(2950100002)(50986999)(16586007)(5660300001)(101416001)(55016002)(47776003)(39060400002)(76176999)(105586002)(54906003)(6666003)(316002)(86362001)(33646002)(53936002)(66066001)(106356001)(189998001)(81166006)(2906002)(97736004)(8676002)(5003940100001)(81156014)(36756003)(69596002)(8936002)(4326008)(3846002)(6116002)(50226002)(68736007)(21086003)(50466002)(478600001)(48376002)(305945005)(25786009)(16526018)(118296001)(7736002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0501MB2042; H:mellanox.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR0501MB2042; 23:JLsKKFqhXgZ8B5K0MQaGg2YP23vg4xnB/eQib2Q?= LIi122DuHmhu+Cea3zBHAUu5bHX8uMVFi4bRNkGpolU39QFHB8xlfGjQ6rmUxRStHue3LlBojh2vqdjvgJD/VTom7o1NIr5pgzvhWsU1mfs58a3dj47ByrZy5+zL3sG57N8bzp+M8p0kO/yg0mBp6/9Tm3Xr3XquGRYHrOi30oelLm8ADozELBHaLI8hODDnZsBy+dZhweooRlMQ02aB7FnIhjrnzFgalNV077ICT5CLZaLkhgi4XeMDe9L80JJP+xBSU3OlUtyeOjRkeHFPc/NPB7WGPtuAqF0QjBRvGQ8i+LD2fvlsnVcZydND6RYWF7hTKa+k4E+akRYSoybEh+23wNlL4H0IkazFq/YkJs/TNZa5OhEFsendsN5iVNoLVrSw6Q9yXPSdcPUh0jyPNMrkZ8RC/b3IB0mHUyAFieaISt8499vlqer+wxwqOYr1WEoh5v51Mot2wTMCONgnWEgZ+RDWuZsjBnWpQCKqUFZLyY2fAwdFGV9jFuoxG7C5aJcuE1teBXZ327UliThTMOQkDs4QVOQNbPRZ0mC9PgFwevXbfcWcmTn3Z9ISJCyv+rdWOsaYVdpGMkBro8z4coYj8kuSAD4fCI1aehHl1GAqNat6SxQ4m7KtM0OcV4M1jpPCGIcFYy7AkUGfO82Hl8/DqlwpAOU2c0tPqrpejVZF3wcWvVWOhd52XCNP0q/YWVE3PESUCs4MtUPv/ChFUE8J4PUTeJlnFcHR+oUDLLuWfCqc+tdUqUKifMk29cVQIJqcqmbvE2TIwxOytmGOyPuTMTiouTxfWOWqq+hKjwaWsjnnyPD/Fwjoj4nZZduXPIszC/SpTDq4g4vrMbkDLFxtQ2WF1ubgmhKFzN5ci0wEn0jG6DSPSQc4eReg+NyQeAV6uY8c7yKaVIPZ+rTmisllOikQioOr0evLkiTvPurLATwwOtFLFDLipQ16yo6M0RGqd2w0WOPysVVWtjr2csl+fLn5ARUzBPDVVZqgdCTU840peweobuJEcBBaslQvppmyC1IOVscG8oYYh4EikR5Tgm1v3qrZytgkpBJT4Gcmd/PRrwMmYXm7QYFaOSYDxzWl1hJaa7ThKd4PF3FCYQAXX X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2042; 6:ag0cb6VznJuKpBcK453zBMFAqqQ0Ujm3ZU+LDmPz1SQ7lnj4A5Lk0shHexOlLBol4mjHPseIH4U5LhyLf0Y+0rJmLqGUFPfPzhp3nkx7vg+SLbZgSrQ20gXaNPlMlVYg4tarTTLqWN8cwR3evJY9QyJOeeb+ke8N2KNp27zm2aaFxrm5zDkDRg92rbKAAyk9rV5+k70lF5CkSC0Z3ruo4Tatkopd30+BT8U2JQIDiGGVhQWji0twyahnMYhmqlOhqTIRXpXKyu27gXruh/7K5bjYwlmlgn3U8ZwrN7+KSaiahQpQ9BZ2UjfMUwrQNayY6Yw5ab85l+WMcwn9tb1k8nMVDM6Xb9tv22a9FGIe3FQ=; 5:+KMBDeMJkFWNnhMVTT6/J8JZ+zLuNPoIA3sJKxp+iL/HJc3iW8y1PKpkq/tvQfr0JmCFbw1C7/Swz2mYqLHbCFOxxIT45f/DI0aSetDHDtYDk/hWdZ+5z7oTYYPzY1vGW3DmBOm+50vSYEwlDJf9gTLd5q1iIxordHKGIgHYcVY=; 24:BBxQetGQx8LG3rCsgMnJng/jhxudKJd8blij6SBVXqKa9R8/R6aj7oLi1jgDKHkTqYhs6HTFLsd5kT1YXZITJZpR/X1RecaIbUk1bmNWUow=; 7:cVegS9Unh02idZGB63vBjeXQZseJxNDogsmg2CQb6P4U12m1JPOuK7crKD5u6hGZvZDsRqtDR7wJkh1EmIExOqvSt2iHvqM7sit273VJq2wjw8wXlxkE5SjDE0BCf3xTqdIlaM7D4UU6+JNiu6uXDXK0Vn6Jb3w0WJj6Kj4N5qXurAcOQF2AfvtAPEnFeGlgmQ/pU7Q2Q0N/tz1ndYKZXdPoJvkTdwD9u0qzltGBl2t5N/my7TtqsiK1GnWOBXoq SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2017 00:27:39.0634 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: aaca270b-56e6-48bc-2897-08d51b3f33e0 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0501MB2042 Subject: [dpdk-dev] [PATCH v2] net/mlx5: fix Tx doorbell memory barrier X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Configuring UAR as IO-mapped makes maximum throughput decline by noticeable amount. If UAR is configured as write-combining register, a write memory barrier is needed on ringing a doorbell. rte_wmb() is mostly effective when the size of a burst is comparatively small. Revert the register back to write-combining and enforce a write memory barrier instead, except for vectorized Tx burst routines. Application can change it by setting MLX5_SHUT_UP_BF under its own necessity. Fixes: 9f9bebae5530 ("net/mlx5: don't map doorbell register to write combining") Cc: stable@dpdk.org Cc: Sagi Grimberg Cc: Alexander Solganik Signed-off-by: Yongseok Koh Acked-by: Shahaf Shuler Acked-by: Nelio Laranjeiro --- v2: * Add documentation. * Rename functions to minimize changes. doc/guides/nics/mlx5.rst | 17 +++++++++++++++++ drivers/net/mlx5/mlx5.c | 2 -- drivers/net/mlx5/mlx5_rxtx.h | 23 +++++++++++++++++++++-- drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 2 +- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 2 +- 5 files changed, 40 insertions(+), 6 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index d24941a22..085d3940c 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -171,6 +171,23 @@ Environment variables This is disabled by default since this can also decrease performance for unaligned packet sizes. +- ``MLX5_SHUT_UP_BF`` + + Configures HW Tx doorbell register as IO-mapped. + + By default, the HW Tx doorbell is configured as a write-combining register. + The register would be flushed to HW usually when the write-combining buffer + becomes full, but it depends on CPU design. + + Except for vectorized Tx burst routines, a write memory barrier is enforced + after updating the register so that the update can be immediately visible to + HW. + + When vectorized Tx burst is called, the barrier is set only if the burst size + is not aligned to MLX5_VPMD_TX_MAX_BURST. However, setting this environmental + variable will bring better latency even though the maximum throughput can + slightly decline. + Run-time configuration ~~~~~~~~~~~~~~~~~~~~~~ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 89fdc134f..fcdcbc367 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1037,8 +1037,6 @@ rte_mlx5_pmd_init(void) * using this PMD, which is not supported in forked processes. */ setenv("RDMAV_HUGEPAGES_SAFE", "1", 1); - /* Don't map UAR to WC if BlueFlame is not used.*/ - setenv("MLX5_SHUT_UP_BF", "1", 1); /* Match the size of Rx completion entry to the size of a cacheline. */ if (RTE_CACHE_LINE_SIZE == 128) setenv("MLX5_CQE_SIZE", "128", 0); diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index ea037427b..d34f3cc04 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -578,15 +578,18 @@ mlx5_tx_mb2mr(struct mlx5_txq_data *txq, struct rte_mbuf *mb) } /** - * Ring TX queue doorbell. + * Ring TX queue doorbell and flush the update if requested. * * @param txq * Pointer to TX queue structure. * @param wqe * Pointer to the last WQE posted in the NIC. + * @param cond + * Request for write memory barrier after BlueFlame update. */ static __rte_always_inline void -mlx5_tx_dbrec(struct mlx5_txq_data *txq, volatile struct mlx5_wqe *wqe) +mlx5_tx_dbrec_cond_wmb(struct mlx5_txq_data *txq, volatile struct mlx5_wqe *wqe, + int cond) { uint64_t *dst = (uint64_t *)((uintptr_t)txq->bf_reg); volatile uint64_t *src = ((volatile uint64_t *)wqe); @@ -596,6 +599,22 @@ mlx5_tx_dbrec(struct mlx5_txq_data *txq, volatile struct mlx5_wqe *wqe) /* Ensure ordering between DB record and BF copy. */ rte_wmb(); *dst = *src; + if (cond) + rte_wmb(); +} + +/** + * Ring TX queue doorbell and flush the update by write memory barrier. + * + * @param txq + * Pointer to TX queue structure. + * @param wqe + * Pointer to the last WQE posted in the NIC. + */ +static __rte_always_inline void +mlx5_tx_dbrec(struct mlx5_txq_data *txq, volatile struct mlx5_wqe *wqe) +{ + mlx5_tx_dbrec_cond_wmb(txq, wqe, 1); } #endif /* RTE_PMD_MLX5_RXTX_H_ */ diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index 4cb7f2889..61f5bc45b 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -345,7 +345,7 @@ txq_burst_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, uint16_t pkts_n, txq->wqe_ci += (nb_dword_in_hdr + pkts_n + (nb_dword_per_wqebb - 1)) / nb_dword_per_wqebb; /* Ring QP doorbell. */ - mlx5_tx_dbrec(txq, wqe); + mlx5_tx_dbrec_cond_wmb(txq, wqe, pkts_n < MLX5_VPMD_TX_MAX_BURST); return pkts_n; } diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index e9819b762..a53027d84 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -344,7 +344,7 @@ txq_burst_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, uint16_t pkts_n, txq->wqe_ci += (nb_dword_in_hdr + pkts_n + (nb_dword_per_wqebb - 1)) / nb_dword_per_wqebb; /* Ring QP doorbell. */ - mlx5_tx_dbrec(txq, wqe); + mlx5_tx_dbrec_cond_wmb(txq, wqe, pkts_n < MLX5_VPMD_TX_MAX_BURST); return pkts_n; }